Wake word detection

Team members:

• Yusen Lin (yl162)
• Siyu Xie (sx18)
• Weiwei Zhou (wz37)

Project Description:

Our team select the topic 'wake word detection', according to Chapter 7 in TinyML. This project design a micro device, who can detect and classify some wake words in speech. When people say 'yes' to the device, the LED flash light will turn green for several seconds. When people say 'no', it will turn red. And when people say anything else, it will turn blue.

We use tensorflow lite to train the model and upload the model to the micro device. The micro device we used is Arduino NANO 33 Sense board.

How to train the model:

The model was trained on a dataset called the Speech Commands Dataset (https://oreil.ly/qtOSI). This consists of 65, 000 one-second long utterances of 30 short words, crowdsourced online.

The structure of the program:

Main loop

Like the “hello world” example, our application runs in a continuous loop. All ofthe subsequent processes are contained within it, and they execute continually, asfast as the microcontroller can run them, which is multiple times per second.

Audio provider

This component captures raw audio data from the microphone. Since the meth‐ods for capturing audio vary from device to device, this component can be over‐ridden and customized.

Feature provider

The feature provider converts raw audio data into the spectrogram format thatour model requires. It does so on a rolling basis as part of the main loop, providing the interpreter with a sequence of overlapping one second windows.

TF Lite interpreter

The interpreter runs the TensorFlow Lite model, transforming the input spectrogram into a set of probabilities.

Model

The model is included as a data array and run by the interpreter. The array islocated in tiny_conv_micro_features_model_data.cc (https://oreil.ly/ZQsR0).

Recognize commands

Since inference is run multiple times per second, the recognize commands class aggregates the results and determines if, on average, a known word was heard.

Command responder

If a command was heard, the command responder uses the device’s output capa‐bilities to let the user know. Depending on the device, this could mean flashing an LED, or showing data on an LCD display. It can be overriden for different device types.

Run the model on the micro device:

1. In arduino IDE, include tensorflow lite library.

2. Open the micro_speech project.

3. Plug in the board, adjust the type and the port.

4. Compile and upload the code to the board.