Cyborg Crafts: Vibrotactile Tongue Vision

Introduction

This project was submitted, and accepted to, the IEEE World Haptics Conference 2021 Student Innovation Challenge. Very big thank you to the organizers for supplying the hardware!

The Vibrotactile Tongue Vision (VTTV) system is an open-source programmable tongue display unit (TDU) that augments visual and auditory experiences through haptic feedback. VTTV is designed to be low floor and wide walls, enabling those without embedded systems skills to fabricate a TDU, controller, and craft tactile experiences easily. Experienced engineers, researchers, makers, and artists can hack VTTV to be an actuator in their projects.

TDUs typically use electrotactile stimulation to display information to the user. These systems produce sensations by generating voltage pulses from electrodes to the user’s tongue. These commercial TDU systems are closed source, prohibitively expensive, and compromise haptic resolution for user comfort. Cyborg Crafts proposes an alternative to electrotactile TDUs, a novel system that uses ERM motors for actuation. Cyborg Craft’s goal is to enable anyone to fabricate their own TDU using easily accessible materials with affordable tools.

What Do We Mean by “Craft Tactile Experiences”?

Crafting is the act of skillfully creating an object or an experience such as a story or narrative. There is a tradition of sharing crafted sensory experiences, from the sharing of a mixtape/playlist; to sharing photos from a vacation. These experiences are often passive and static; one cannot feel the explosions of fireworks in a photo or feel the crashing of tidal waves in a video. Vibrotactile Tongue Vision prioritizes the crafting of tactile actuation that can be connected to other sensory experiences. Similar to associating scents with specific experiences, one can create a tactile experience that will be associated with static experiences.

Through the creation and dissemination of VTTV, we will introduce a new genre of human augmentation: sensory crafting. Sensory crafting is the crafting of a sensory experience to be shared from one human's sensory modality (e.g., visual, auditory) to another through a mediating device. Unlike previous devices where the user reacts to transduced signals from sensors, VTTV encourages users to reflect on their own sensory experiences and then program actuation to be shared. In addition to a sensory crafting device, VTTV is also an open-source, hackable tongue display unit that can be utilized as a haptic output for various research and maker applications.

Embedding the TDU in Candy

Cyborg Crafts has also experimented with encasing the tongue display unit in various types of chocolates and gummies with great success. While they do dampen some of the haptic feedback from the motors, they increase the crafting aspect by enabling users to share taste, as well as tactile, crafted experiences!

We haven't quite formed a system for encasing the TDU, but here's what we've had success with:

Here are the molds that we've had the most successes with.

We've used gummy bears, chocolate chips, chocolate bars, and gummy candy powder mixes (Figure 2, 3, & 4) with much success. For the store-bought gummies & chocolate, all you need to do is carefully melt them in a microwave, pour them into the mold (make sure to leave space), then place the TDU into the mold until it's submerged (Figure 1). For the powder mixes, you follow the same process but pour the mixture into the mold before it hardens, and then place in the refrigerator.