Face-Mic Turns Your Virtual Reality Headset Into a Permissionless Speech-Monitoring Microphone
Using on-board sensors to turn head movements during speech into recognized words and numbers, Face-Mic highlights privacy problems.
Researchers at Rutgers University, Shanghai Jiao Tong University, Texas A&M, and the University of Tennessee have published a paper describing a family of vulnerabilities in popular virtual and augmented reality (VR/AR) headsets could leave their users at risk of attack — by allowing ne'er-do-wells to eavesdrop on them, turning the headset into a face-mounted microphone.
“Face-Mic is the first work that infers private and sensitive information by leveraging the facial dynamics associated with live human speech while using face-mounted AR/VR devices," Yingyin "Jennifer" Chen, associate director of Rutgers' WINLAB, explains. "Our research demonstrates that Face-Mic can derive the headset wearer's sensitive information with four mainstream AR/VR headsets, including the most popular ones: Oculus Quest and HTC Vive Pro."
That it's possible to tap into a microphone to listen in on private conversations is nothing new, but what Face-Mic proves is that devices you wouldn't normally associate with audio recording can be turned into functional microphones — in this case, a VR/AR headset which is capable of tracking the wearer's movements to a point sensitive enough it can reveal private information.
"By analyzing the facial dynamics captured with the motion sensors," Chen says, "we found that both cardboard headsets and high-end headsets suffer security vulnerabilities, revealing a user’s sensitive speech and speaker information without permission."
While a number of commercial VR/AR headsets have integrated microphones for voice control and in-app chat, accessing it typically requires elevated permissions, which must be granted by the wearer. Accessing the motion sensors, however, can be done invisibly — and provided enough data for the team to infer the user's gender and to capture spoken digits and words including passwords, credit card numbers, phone numbers, birth dates, and more.
"Given our findings," Chen warns, "manufacturers of VR headsets should consider additional security measures, such as adding ductile materials in the foam replacement cover and the headband, which may attenuate the speech-associated facial vibrations that would be captured by the built-in accelerometer/gyroscope."
The team's work has been published under closed-access terms in the Proceedings of the 27th Annual International Conference on Mobile Computing and Networking (MobiCom '21).