"Watch my Painting" Concept Turns the Back of Your Hand Into an Art Tablet for Your Smartwatch

Researchers from Leibniz University Hannover have developed a way to more easily use a pen-based input with a small-screen smartwatch — by turning the back of the wearer's hand into drawing space.

"Smartwatches can be used independently from smartphones, but input tasks like messaging are cumbersome due to the small display size. Parts of the display are hidden during interaction, which can lead to incorrect input," the researchers explain of the problem they sought to solve. "For simplicity, instead of general text input a small set of answer options are often provided, but these are limited and impersonal. In contrast, free-form drawings can answer messages in a very personal way, but are difficult to produce on small displays."

"To enable precise drawing input on smartwatches we present a magnetic stylus that is tracked on the back of the hand. In an evaluation of several algorithms we show that 3D position estimation with a 7.5x20mm magnet reaches a worst-case 6% relative position error on the back of the hand. Furthermore, the results of a user study are presented, which show that in the case of drawing applications the presented technique is faster and more precise than direct finger input."

The system works by combining a cylindrical magnet, touch sensor, six degrees of freedom (6DoF) inertial measurement unit (IMU), and a battery into a pen-like stylus. Data from the touch sensor and IMU are combined with data from the smartwatch's internal magnetometer to infer the position, angle, movement, and skin contact of the pen — and translate the movements into on-screen drawings.

The team's work isn't the first to look at pulling interaction away from the screen, nor at using a smartwatch's magnetometer to sense more than magnetic north: Four years ago engineers at Dartmouth developed WristWhirl, which turned the wearer's wrist into a joystick; late last month the MagTouch project outfitted smartwatch users with a magnetic ring and used magnetometer data to determine which of three fingers was touching the display.

The concept has been published on the Leibniz University Hannover website as part of the proceedings of the ACM CHI Conference on Human Factors in Computing Systems 2020 (CHI'20), under open-access terms