Using Digital Puppetry to Control a Robot Arm

The term "digital puppetry" refers to several distinct techniques that all translate the manipulation of a physical input device into digital commands. Your computer mouse is a very rudimentary example of this, as the cursor mirrors the physical movement of the mouse. If you expand that idea, you can build an input device that mimics the structure of the digital model and that allows for perfect 1:1 control. The same idea is applicable to robotics, as demonstrated by Jelle Vermandere's robot arm controlled by a smaller robot arm.

This project follows the same concepts developed by pioneers like Jim Henson, who used digital puppetry to animate Waldo C. Graphic (a virtual character rendered in real time) in the late '80s. Vermandere can manipulate his small robot arm and a digital model of that robot arm on his PC will mirror the movement. At the same time, a larger robot arm that Vermandere constructed in the past will mimic those movements in the real world.

Conventional controllers almost always suffer from the fact that the input doesn't match the output. Pushing a joystick to the left, for instance, is an abstraction of the corresponding movement of the robot. This digital puppetry lets Vermandere move the small input robot in exactly the way that he wants the large output robot to move.

The small robot, the digital model, and the large robot all look identical. The small robot may be scaled-down, but its joint angles all correspond to the joint angles of the large robot. To use the small robot as an input device, Vermandere just needed to monitor those joint angles. That was a straightforward task, as the small robot uses servo motors to actuate its joints. Servo motors have built-in encoders or (as is the case here) potentiometers for angle feedback. An Arduino Nano 33 BLE board both controls those servos and monitors their angles. It then passes that angle data to the PC. The PC, in turn, animates the digital model to mimic the angles and controls the larger robot arm.

While small input and output errors can add up to result in imperfect control, this digital puppetry technique allows for easy and intuitive control of every joint. As a bonus, Vermandere can still use the small robot arm as a robot, since it has servos in its joints.