SwarmTouch: Handheld Interaction with Multiple Drones at Once

One drone is easy enough to control, with hours of practice of course, but operating two or more simultaneously is — if not impossible — extremely difficult. As reported here, a team of researchers at Skolkovo Institute of Science and Technology, or Skoltech, in Russia have developed a swarm control system that interfaces with one’s hand and fingertips.

The system, called “SwarmTouch,” tracks markers on the operator’s wrist using an OptiTrack motion capture system with eight cameras covering a 5x5x5m area. This allows a swarm of drones that are also tracked in the same visual manner to react to arm motions. An Arduino Uno handles the wrist/hand unit, which is connected to the central computing system via an XBee radio. Vibration motors on each of the pilot’s fingers provide sensory feedback.

Each drone is linked together with a virtual spring-mass-damper system, where the human hand controller “pushes” one drone into the others, which react in turn. This is seen clearly in the first part of the video below, where if you imagine each one paired with an unseen mechanical system, the swarm’s movement makes sense. As the swarm contracts, the vibration feedback goes from the pinky finger to the thumb, and it expands in response to arm movements, vibration propagates from the middle finger out.

Both the interface method as well as the way the drones react to one another are interesting developments in and of themselves. the full research paper is available here.