Teaching Rules of Thumb

When learning a new skill, we typically begin by observing someone else that already has prior experience with it. By closely studying the techniques, strategies, and nuances demonstrated by experienced individuals, we gain valuable insights and understanding that can accelerate our own skill acquisition. But seeing is not everything, especially when it comes to tasks that require rapid or fine muscle movements, like playing a musical instrument or performing a delicate surgery, for example.

In saying that we have to “get a feeling for something,” we may be making a far more literal statement than we realize. The sense of touch is crucial in learning many types of skills. But whereas it is simple to watch a video online or listen to a podcast, for example, to learn about any number of topics, it is not so easy to experience what it feels like to correctly perform a task. The instructor can certainly give some advice and provide some rules of thumb, but this is not at all the same as actually experiencing the sensations.

Tactile feedback systems do exist that might help to serve this role. However, there are no ways to transfer the sensations experienced by one individual to another, at least not with the sort of fidelity that is required to teach someone a complex task that requires fine motor control. To move beyond the relatively low resolution tactile feedback systems used in today’s virtual and augmented reality applications, innovations are needed.

A team at MIT’s CSAIL has been at work on exactly this problem, and has recently published the results of their research. They have developed a multifunctional smart glove that can both record and reproduce tactile sensations. With these capabilities, it is possible to transfer tactile sensations from one individual to another, allowing them to experience what it is like to perform a task correctly firsthand, and even provide promptings to help them along as they learn.

Each glove is custom-made for its wearer with the help of a digital embroidery machine. These machines are capable of embedding electronic components into textiles as they are produced. In this case, piezoresistive force sensors and arrays of vibrotactile actuators were embedded into the gloves to record and reproduce tactile sensations, respectively. A glove can be made for a new user in about ten minutes.

As it turned out, this glove alone was insufficient to reproduce tactile sensations accurately. This is because everyone perceives haptic feedback in a different way. To accommodate these differences, the researchers developed a simple machine learning algorithm that was trained to adapt haptic sensations to each individual. In this way, the recorded sensations could be tailored to each individual before they were transferred. Only about fifteen seconds of training data is needed from a new user for the algorithm to learn how the haptic feedback should be presented to them.

In a series of experiments, users of the smart gloves were trained to play the piano, or play a few different video games. It was demonstrated that the haptic feedback helped individuals to learn faster than those not using the glove. It was also shown that the haptic feedback preprocessed by the machine learning algorithm led to greater success rates than raw feedback directly transferred from another user.

Aside from the obvious use as a teaching aid, the team also sees a number of opportunities in virtual and augmented reality, and also in robotic teleoperation for their technology. As they improve their system to accommodate additional use cases, they also intend to explore the possibility of extending the haptic feedback to guide the feet, hips, and other body parts.