MeMic Only Has Ears for You

Recent advances in artificial intelligence (AI) have enabled the development of all sorts of useful tools that can make us more efficient and add convenience and enrichment to our lives. But in some cases, getting people to use these tools can be a tall order. A major barrier to adoption lies in the types of data that these tools need to collect. In order for that shiny, new AI-powered gadget to do its job, it needs to be supplied with a steady diet of data — and that may include video or audio that could reveal some very private information.

Consider the applications made possible by utilizing an always-listening microphone on a wearable AI device. With this combination, a variety of health monitoring tools and ultra-customized personal assistants can be created. But in order to reap these benefits, one must be willing to let everything that they, and anyone around them, say (and every other sound as well!) be recorded before likely being transmitted over the public internet to an unknown server farm in the cloud.

That is, of course, a hard pill to swallow for many people that could otherwise benefit from these tools. But thanks to the work of a group of engineers at the MIT Media Lab, always-listening microphones may soon become more acceptable. They developed a prototype wearable device called MeMic that serves to activate a microphone only when the wearer of the device is speaking. This prevents the conversations of bystanders, and other unrelated noises, from being recorded.

The hardware design is very simple and relies on an off-the-shelf Seeeduino XIAO nRF52 Sense BLE microcontroller development kit and a small LiPo battery for power. This provides the horsepower for the necessary computations, and also an inertial measurement unit (IMU). The IMU rests against the body and collects motion data as the wearer moves.

A simple algorithm was developed, which runs locally on the nRF52 microcontroller, to interpret the IMU data and determine when the wearer is speaking. First, a bandpass filter narrows the frequency range of incoming signals to only those that correspond with voice-related frequencies. Next, a trio of features (root mean square, zero crossings, and rate of change of root mean square) are calculated from the z-axis acceleration data. Thresholds, which were determined through an iterative testing process, are then applied to these features to determine if the MeMic user is speaking.

When the individual is speaking, MeMic enables the microphone and also turns on a small light. This light indicates that recording is in process, and is intended to give users and bystanders confidence that their privacy is being protected as intended.

To evaluate their methods, the team built a MeMic into a pendant necklace, glasses, earbuds, and a snug-fit necklace. These devices were given to a dozen participants in a trial that were asked to wear them and engage in a variety of activities. In general, the participants noted that they had less privacy-related concerns when using MeMic with a wearable recorder, indicating that the system makes wearable recorders more socially acceptable.

While MeMic may ease some concerns, it is not perfect. It will still capture the speech of bystanders if they are talking at the same time as the wearer of the device, for example. MeMic also requires a tight mechanical coupling between the device and the user which could be uncomfortable when worn for long periods of time. In the future, the researchers plan to explore the use of contact-based microphones and also conduct some longer-term trials to resolve these issues.