[TinyML] Wake Word Detection

The aim of this project is to try training a new model that could be used to handle independent word detection task on an Arduino Nano 33 board. The process of detection is based on the existing code of Wake Word Detection here. The model training code is based on the instructions of this notebook.

I have also followed the instructions from Chapter 7: Wake-Word Detection: Building an Application and Chapter 8: Wake-Word Detection: Training a Model to build and train my own model. Thanks for the detailed and in-depth instructions that have saved me lots of time.

Prepare for Training

I have chosen 'cat' and 'dog' as my target words. The training steps and corresponding learning rates have been kept the same with the default settings. The following code shows the configuration in training period.

WANTED_WORDS = "cat,dog"

TRAINING_STEPS = "12000,3000"
LEARNING_RATE = "0.001,0.0001"

Training Model

Training the model is easily done by executing the following python script with corresponding arguments given.

tensorflow/tensorflow/examples/speech_commands/train.py

Freeze the Graph

In this section, we will combine relevant training results (graph, weights, etc) into a single file for inference. This process is known as freezing a model and the resulting model is known as a frozen model/graph, as it cannot be further re-trained after this process. Freezing the graph could be easily done by executing this python script with corresponding arguments given.

tensorflow/tensorflow/examples/speech_commands/freeze.py

Deploy the Model

To deploy my model onto the Arduino device, I changed the following parts of the code and complied it using Arduino IDE. The were several warnings but nothing matters. After uploading the code to my device, we can take a look on how it works for the work detection task.

In arduino_command_responder.cpp:

if (is_new_command) {
    TF_LITE_REPORT_ERROR(error_reporter, "Heard %s (%d) @%dms", found_command, score, current_time);
    // If we hear a command, light up the appropriate LED
    if (found_command[0] == 'c') {
        last_command_time = current_time;
        digitalWrite(LEDG, LOW);  // Green for cat
    }
    if (found_command[0] == 'd') {
        last_command_time = current_time;
        digitalWrite(LEDR, LOW);  // Red for dog
    }
    if (found_command[0] == 'u') {
        last_command_time = current_time;
        digitalWrite(LEDB, LOW);  // Blue for unknown
    }
}

In micro_features_micro_model_settings.cpp:

const char* kCategoryLabels[kCategoryCount] = {
    "silence",
    "unknown",
    "cat",
    "dog",
};

In micro_features_model.cpp:

const unsigned char g_model[] DATA_ALIGN_ATTRIBUTE = { 0x20, 0x00, 0x00, 0x00, 0x54, ... }
const int g_model_len = 18720;

Demo

These pictures show how it works on the Arduino Nano 33 board.

"Dog"

"Cat"