Three Magnetometers, a Fingertip Magnet, and Clever Mathematics Offer Cheap, Accurate 3D Tracking
Toyota's research arm works on a flexible, low-cost, and non-intrusive way to track a wearer's finger through 3D space.
Kent Lyons of the Toyota Research Institute has developed a finger tracking system based on a wearable magnetometer and fingertip-mounted magnets — allowing for full three-dimensional tracking through a novel processing system.
"Obtaining a signal useful for continuous pointing input is still an open problem for wearables," Lyons explains. "While magnetic field sensing is one promising approach, there are significant limitations. Our key contribution in this work is a simulation of a system that tracks a magnet in 3D while also accounting for the ambient magnetic field. The simulated sensor data is processed and the position and rotation is determined by using magnetic field equations, a particle filter and a kinematic model of the hand."
The sensor hardware itself is nothing particularly new: Magnetometers are worn on the back of the hand, while the finger to be tracked is fitted with a permanent magnet worn attached to the fingernail so as not to get in the user's way. "The magnet is a passive component," Lyons adds. "It requires no power to generate the magnetic field and we do not need to place a battery on the finger."
The problem comes in processing data from the sensor in order to actually track the wearer's finger, despite the presence of environmental magnetic fields and calibration and noise issues within the sensors themselves. The solution: Novel algorithms and a forward kinematic model of the wearer's hand, which was shown to dramatically increase accuracy.
There's a catch, however: While a prototype wearable was created, the algorithm itself was only tested using simulated data. "We constructed a prototype," Lyons notes, "but our focus here was on developing our algorithm and our latency was not good enough for user tests and lacked the ability to measure ground truth adequately."
The full paper is available under open-access terms following the project's presentation at the ACM International Symposium on Wearable Computers 2020 (ISWC '20).
