Unfolding Space Allows the Blind to See with Their Hands

Interaction designer Jakob Kilian has developed a device that allows the blind to see with their hand using a tool that he calls Unfolding Space. As he explains, “The ‘Unfolding Space’ project deals with sensory substitution: A phenomenon by which the function of one missing or dysfunctional sensory modality is replaced (substituted) by stimulating another one. The outcome is a prototype that projects a 3D picture, generated by a special camera, as vibration patterns on the back of the hand.”

The Unfolding Space glove enables the blind to identify their surroundings, detect obstacles, and orient themselves within an area via haptic vibration. The glove is outfitted with PMDTec’s CamBoard pico flex depth camera, which captures the surroundings in a 3D pointcloud. The 3D-generated image is then processed by an onboard Raspberry Pi 3B+ and haptically projected to the back of the wearer’s hand using vibrational patterns supplied by nine LRA vibration actuators. The location of the vibration on the hand denotes the position of an object in space, while the strength of the vibration represents its distance.

Wearers equipped with the Unfolding Space glove can explore their surroundings, locate and avoid obstacles, and orient themselves in unfamiliar areas. Like riding a bike, the vibrational patterns and intensity are learned over time, as they have to be actively interpreted by analyzing the tactile stimulus, which is scientific jargon for the brain getting used to the input. The Unfolding Space glove is still undergoing development; however Kilian has made his device open source, so anyone can recreate his current glove design using the files (code for the glove is coming soon) found here.