A Soft Exoskeletal Robo-Glove Can Play the Piano — And Can Help Stroke Patients Relearn to Do It Too

Researchers from the Florida Atlantic University, Boise state University, and the University of Florida College of Medicine have developed a smart glove which could help rehabilitate patients after a stroke — to the point of restoring their ability to play a musical instrument and of "feeling" mistakes as they happen.

"Playing the piano requires complex and highly skilled movements, and relearning tasks involves the restoration and retraining of specific movements or skills," explains senior author Erik Engeberg, PhD and professor at Florida Atlantic University, of the team's creation. "Our robotic glove is composed of soft, flexible materials and sensors that provide gentle support and assistance to individuals to relearn and regain their motor abilities."

When a person suffers a stroke or other injury to their brain, they can lose full control of their limbs — and regaining fine dexterity, such as that required to play a musical instrument, is a difficult process. The glove, which takes the form of a custom-molded exoskeleton with pneumatic actuators in the fingertips and 16 "taxel" sensors on each fingertip providing tactility when touching objects or surfaces while also mapping the pressure across the glove's surface, aims to help.

"While wearing the glove, human users have control over the movement of each finger to a significant extent," Engeberg notes. "The glove is designed to assist and enhance their natural hand movements, allowing them to control the flexion and extension of their fingers. The glove supplies hand guidance, providing support and amplifying dexterity. We found that the glove can learn to distinguish between correct and incorrect piano play. This means it could be a valuable tool for personalized rehabilitation of people who wish to relearn to play music."

The glove itself is produced in a single molding, using a 3D-printed mold tailored to each specific user. The hardware is then fed into a Teensy 4.1 microcontroller through an op-amp circuit, feeding data to a Robot Operating System (ROS) network and through to Simulink for visualization and storage. The songs used in the experiments were also programmed in Simulink, which drove the pneumatic actuators to guide the wearer's movement based on errors noted by a machine learning classification algorithm.

"Adapting the present design to other rehabilitation tasks beyond playing music, for example object manipulation, would require customization to individual needs," cautions lead author Maohua Lin, PhD and adjunct professor at Florida Atlantic University.

"This can be facilitated through 3D scanning technology or CT scans to ensure a personalized fit and functionality for each user. But several challenges in this field need to be overcome. These include improving the accuracy and reliability of tactile sensing, enhancing the adaptability and dexterity of the exoskeleton design, and refining the machine learning algorithms to better interpret and respond to user input."

The team's work has been published in the journal Frontiers in Robotics and AI under open-access terms.

Main article image courtesy of Alex Dolce/Florida Atlantic University.