Android Things HandBot

Introduction

Innovation studio Deeplocal teamed up with Android Things to create HandBot, a DIY robotic hand that can play games and react to your actions. Using machine vision and Tensorflow, Handbot recognizes your hand gestures so it can take you on in rock-paper-scissors, mirror your hand movements, or play a memory game.

Making a HandBot requires the right hardware, software, and a bit of do-it-yourself assembly. We'll show how we built ours, but feel free to do it your own way. Once completed, your HandBot can "ro-sham-bo" all day long.

Bill of Materials

• (1x) Pico.iMX7Dual Starter Kit

• (1x) Power Supply

• (3x) Dotstar LEDs

• (1x) Servo Hat

• (1x) Audio Amplifier

• (1x) 4-ohm Speaker

• (5x) Finger Servo

• (3x) Wrist and Forearm Servo

• (5x) Finger Servo Horn

• (1x) Wrist Servo Horn

• (2x) Forearm Servo Horn

• (5x) Servo Extension Cables

• (2x) Nylon Sheet 1/32-in

• (2x) Acrylic Sheet 1/8-in

• (1x) Power Switch

• (1x) Power Jack

• (1x) Audio Cable

• Super glue

• Nuts and bolts, heat set inserts (see attachment "Hardware BOM")

Estimated Parts Cost: $490 (+ an Android Things starter kit)

Estimated Build Time: 7 hours

Wiring schematics and file attachments are located at the bottom of this page.

Overview

Android Things HandBot is made up of two main parts: the base and the hand itself. The palm of the hand contains five servos that move the fingers and one servo in the wrist. The base contains two servos for forearm movement and all other electronics including the LED ring, the Pico board, and the camera.

The camera sits in the middle of the LED ring. The camera sees a gesture being presented above the base then reacts with movement and lighting animations.

Assembly

1. Cut the acrylic pieces. We used a laser cutter to cut ours from the attached files.

2. Cut the nylon pieces. We used a laser cutter to cut ours from the attached files.

3. Install #2-56 heat-set inserts into the holes marked below. Use the tip of a soldering iron to heat and push the metal inserts into the holes, melting the plastic slightly. Push the inserts all the way through so the threads are fully engaged in the plastic.

4. Install #8-32 heat-set inserts into the holes marked below. Use the tip of a soldering iron to heat and push the metal inserts into the holes, melting the plastic slightly. Push the inserts all the way through so the threads are fully engaged in the plastic.

5. Prepare the blue servo horn for the wrist joint. Take the double sided aluminum servo horn and cut the ends down to the inner-most mounting hole. Sand down the cuts so that there are no rough edges.

6. In each of the small finger servo horns, cut a shallow slot across the top lip. This will allow the string to pass through later on.

7. Assemble the five finger pulleys. Glue both pieces together and glue onto the finger servo horn, so that the top of the horn sits flush with the pulley surface, and the slots are aligned.

After the glue dries, file a shallow groove into the outer surface for the string to lay in. The groove should be big enough to fit about three full turns of string.

8. Mount the five small servos into the palm panel using #4-40 screws and nuts. The hand is shown below with the palm facing down, thumb to the left.

Orient the index and ring finger servos with the wire facing down (towards the wrist). Orient the middle and pinky finger servos with the wire facing up (towards the fingers). Orient the thumb servo with the wire facing towards the thumb.

9. At this point, we recommend labeling the loose end of each cable, so you don't lose track later. We also added some plastic cable sheathing to clean up the bundle of wires.

10. Mount a pulley onto each servo using the screws included with the servo.

11. Glue the wrist servo horn onto the acrylic wrist plate, with the spline adapter sitting down in the hole.

12. Dry-fit the wrist plate to the palm assembly. Install a #2-56 nut and bolt into the t-slot as shown below. Glue the nut in place. Then remove the bolts and wrist plate and set aside.

13. Install the four nylon standoffs onto the back of the main palm plate.

14. Glue the two string guides into the main palm plate.

15. Group together the three nylon pieces for each finger. Note the size variations: there are 1x small, 2x medium, and 1x large sets.

16. For each finger, fold the fingertip around and under. Reference the picture in step 21 while folding. Rivet each side in place with a short nylon rivet.

17. Next, rivet one side of each nylon string guide to its matching hole. Make sure the scored lines are facing down for proper folding later.

18. Fold over each string guide and rivet the other side to the matching hole.

19. Finally, rivet the finger joints closed. Use the longer nylon rivets for the lower two joints where three layers of nylon are being joined.

20. Repeat for all four fingers and the thumb. When assembling the thumb, make sure that the hole in the string guide faces towards the thumb tip.

21. Screw each finger onto the back of the main palm plate using pan-head #2-56 screws.

22. Slot together the four side plates of the wrist and glue at the joints.

23. Mount the wrist servo into the top plate of the wrist using #4-40 nuts and bolts.

24. Slot the top wrist plate onto the rest of the wrist and glue at the joints.

25. Tie a piece of string to the pair of slots inside each fingertip. Route the string down the inside of the finger through both holes, and through the slot in the string guide. Wrap it tightly around the pulley, then lay the string in the slot and anchor the end underneath the servo horn screw.

26. Rivet one end of each nylon strip to the palm cover as shown.

27. Starting from the center, crease and fold both sides of the palm cover inwards as shown. Rivet the other end of the wide strip to the matching hole on the left side - this strip will form the back of the knuckles. Leave the narrow strip hanging loose and the small tabs down the front open for now.

28. Slide the nylon palm cover over the palm, with the thumb through the large hole. The narrow strip should cover the front of the screw holes under the fingers. Use four #2-56 nylon screws to secure the back of the cover to the nylon standoffs. Snap in the last four rivets (three on the front, and the second rivet on the narrow strip).

29. Fold and rivet the nylon wrist cover. Slide over the wrist assembly.

30. Mount the forearm servo horns onto the bottom of the wrist using #6-32 screws as shown. These screws should go through both the acrylic and the nylon cover, securing the cover in place.

31. Press the acrylic wrist plate onto the wrist servo, and secure with the included screw.

32. Attach the hand assembly to the wrist plate using the t-slot screws and nuts.

33. Route the servo cables into the wrist assembly through the hole in the back. Inside the wrist assembly, plug the servo extension cables into each servo.

34. Slot together the acrylic pieces for the base as shown. Glue all joints, excluding the base plate and the two long tapered pieces that lay down the middle (these need to be removable later).

35. Glue one corner bracket into each corner of the base, inset to leave about a 1/8-in gap from the bottom. This will allow the bottom base plate to sit flush with the lower edges of the base assembly when it is screwed into these brackets. Note that a scrap of cardboard makes for a good spacer to support the brackets while the glue dries.

36. Mount the forearm servos to the two long tapered acrylic pieces using #8-24 screws. Make sure the servo mounting tabs are "outboard" of the acrylic pieces.

37. Mount the NXP Pico board and audio amplifier board to the base plate using 2-56 screws and standoffs. Plug the servo hat into the NXP Pico board.

38. Solder the speaker to the audio amplifier and glue above the speaker grille, then attach the base plate to the assembly using #8-32 screws.

39. Install the power jack and switch into the back of the base assembly. Run power wires to each component according to the attached wiring diagram.

40. Route the audio aux cable and servo cables according to the attached wiring diagram.

41. Assemble the camera mount. First, mount the camera onto the acrylic using #2-56 screws. Fold and route the cable underneath, then glue the middle and top plates onto the assembly.

42. Add the LED ring to the camera assembly. We built a custom PCB using the attached Gerber files - check out OSHpark.com or your local hackerspace to do this. Alternatively, you can use a DotStar LED strip wrapped in a circle. We suggest cutting a frosted acrylic ring to act as a light diffuser.

43. Plug the camera cable into the NXP Pico, and connect the LED signals according to the attached wiring diagram.

44. Before final assembly, install the software and calibrate the forearm servos (see detailed instructions below).

45. Remove the two long tapered inner acrylic from the base. Install the hand and wrist assembly into the base by pressing each of the forearm servos into the respective horns. Plug the servo cables into their respective channels on the hat, then slot everything back into the base assembly.

46. Fold and rivet the nylon base cover, and install over the base. Done!

Installing the Software

The board's firmware is written and deployed entirely through Android Studio. It uses the Pico i.MX7D camera module to capture photos, Tensorflow for gesture detection, and Android Things hardware drivers to control the servo motors (finger, wrist, and forearm), LED ring, and sound.

1. Flash Android Things onto the NXP development board using this guide.

2. Clone the software repository onto your computer.

3. Import the project into Android Studio. (File > New > Import Project)

4. Deploy the application. (Run > Run 'app')

Calibrating the HandBot

Because mechanical systems (like these motors and gearboxes) aren't always perfect, you'll need to calibrate the HandBot's servos before you can play RPS with it.

1. Before attaching the forearm to the two forearm servo motors, we'll want to make sure they're moving in tandem. On startup, the forearm motors will travel to their zero (i.e. upright) position. Once they reach zero position, attach the plastic servo horns (that came with your servo motor) to the two forearm motors and make sure the horns are parallel. We'll use these to make sure the two motors are synchronized.

2. On startup, the forearm servos will move to 30º and back to 0. If the motors aren't moving in sync, adjust the ForearmController.DEFAULT_MOTOR_SYNC_OFFSET until the two forearm servos are moving in tandem (incrementing by 5º to start).

Troubleshooting

No servo motor movement/no servo power—make sure the pwm driver board is getting power and that the servo pins are connected to the pwm driver board in the correct orientation and to the pin map found in BoardDefaults.java.

No sound is playing during RPS or SimonSays —make sure the audio amplifier board is getting 5V power from the pwm board and the speaker is connected to the amplifier.

No Tensorflow output in logs (no "Results") —make sure the camera is connected correctly. You should see successful initialization on startup in Logcat.

No forearm movement or jerky forearm movement —Repeat the calibration step. If the two forearm motors aren't moving in sync, then they're working against each other.

Error: "cannot open camera 0 without camera permission" when deploying the application? Restart NXP board— it's a "dangerous permissions" situation (See "Permissions" section <a href="https://developer.android.com/things/sdk/index.html">here</a>).