MagTouch Combines a Magnetic Ring with Machine Learning to Let a Smartwatch Detect Specific Fingers

User interaction researchers at the Korea Advanced Institute of Science and Technology (KAIST) and University of Virginia have developed a means for a smartwatch to determine which of three fingers is being used on its display — greatly increasing the number of interactions possible on the small screen.

"Completing tasks on smartwatches often requires multiple gestures due to the small size of the touchscreens and the lack of sufficient number of touch controls that are easily accessible with a finger," the team explains of the core issue with smartwatch interaction. "We propose to increase the number of functions that can be triggered with the touch gesture by enabling a smartwatch to identify which finger is being used. We developed MagTouch, a method that uses a magnetometer embedded in an off-the-shelf smartwatch. It measures the magnetic field of a magnet fixed to a ring worn on the middle finger."

While the magnet ring is only worn on one finger, it allows the smartwatch - using the magnetometer it already has as part of its on-board inertial measurement unit (IMU) — to determine which of three fingers have been used: "For example," the researchers explain, "if the touch point is at the left side of the right hand, then the contact was made with the index finger."

Raw magnetometer readings alone, however, proved insufficient to accurately determine finger use - in particular when a user faced a different direction, altering the magnetometer's background readings. The solution: Computational Ambient Magnetic Field Eliminator, or CAME. "When the magnet ring is not nearby, the ambient magnetic field is measured and stored," the team explains.

"When the magnet ring approaches the magnetic field around the smartwatch, distorts the magnetic field. Subsequently, CAME subtracts the saved ambient magnetic field data from the measured magnetic field. As a result, only the magnetic field of the magnet ring remains. The concept is simple, but requires elaborate computation because the watch may move and rotate during the ambient magnetic field subtraction process."

Combined with a machine learning classifier, and using existing data from the touch-screen display itself, accuracy improved dramatically: "By combining the measured magnetic field and the touch location on the screen," the researchers note, "MagTouch recognises which finger is being used. The tests demonstrated that MagTouch can differentiate among the three fingers used to make contacts at a success rate of 95.03%."

The team's work, which does not require modifications to the hardware of the watch, has been published under open-access terms in the ACM Digital Library as part of the Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20).