SeamPose Turns Shirt Seams Into Smart, Wireless Capacitive Sensors for Pose Estimation

Researchers at Cornell University have come up with a way to track a user's pose using smart clothing — combining "sensing seams" with a Nordic Semiconductor nRF52840-based microcontroller and battery pack to accurately track the wearer's movements in a project dubbed SeamPose.

"In SeamPose, we repurposed [fabric] seams as capacitive sensors in a shirt for continuous upper-body pose estimation," the researchers explain of their work. "Compared to previous all-textile motion-capturing garments that place the electrodes on the clothing surface, our solution leverages existing seams inside of a shirt by machine-sewing insulated conductive threads over the seams. The unique invisibilities and placements of the seams afford the sensing shirt to look and wear similarly as a conventional shirt while providing exciting pose-tracking capabilities."

The smart shirt developed by the team turns eight of the seams, already present in the clothing to join its various pieces of fabric together, into capacitive sensors by replacing the usual cotton thread with insulated conductive thread. These sensors are linked to a Seeed Studio XIAO nRF52840, a compact development board built around the Nordic nRF52840 microcontroller, through a carrier board that includes two Texas Instruments FDC2214 signal conditioners and a 290mAh battery.

Signals captured by the garment are transmitted over Bluetooth Low Energy to a nearby computer, which runs the readings through a deep learning model to provide pose estimation. In testing, through experimentation with 12 participants wearing the smart garment, SeamPose was able to deliver 3D pose estimation good to a mean per-joint position error (MPJPE) of just 6cm (around 2.4") — a result that, the researchers say, could mean "paving the way for everyday pose-tracking smart clothing."

More information on the project is available on the Seeed Studio blog, while the team's work has been published in the Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology (UIST '24) under open-access terms.