TinyML: Wake World Detection (yr15)

Team member:

Yunchuan Ran (yr15)

Goal:

After running some simple projects in the previous chapters. It is interesting to try to train my own model and replace the model with the Wake World Detection example. Therefore, the goal is to implement embedded speech recognition applications. I will train the model in Colab and replace the old model with this new model, and then upload the application to the Arduino Nano 33 Sense BLE.

Task:

According to the goals, the main tasks are: (1) Training the model. (2) Freeze the graphics. (3) Convert the model to a C array. (4) Replace the model in our project, modify the corresponding settings and code. (5) Upload the project to the Arduino board.

Approach:

1. Training model:

I used Colab to train the model, because Colab is a clould based notebook, which means I can code in any machines only if the machine is connected with internet. And the most important thing is that Colab is very easy to learn and set Tensorflow Lite dependencies. There are some important settings of the training process that I want to give out. I seperate the training process into two parts, one part with more training steps (15000) and bigger training rate (0.001), the other one with less tranining steps (3000) and smaller training rate (0.0001). This special setting aims to train the model fast while can still maintain high accuracy.

2. Freeze the graph:

After training the model, we need to combine the weights and biases information with the model's input to produce model's output. Because we use TensorFlow Lite, but we trained the model in TensorFlow. So, after freeze the graph, we still need to convert the graph into TensorFlow Lite version. After these steps, we will get a 'tflite' format file.

3. Convert model into C array:

Install xxd and use xxd to convert 'tflite' format file into 'cc' format file. (we can do this by running '!xxd -i /path/to/tflite file/ > /path/to/cc target file/')

4. Replace model and modify codes:

There are three things to do here: (1) replace the array information and length value in 'micro_features_model.cpp' with the new model and the new model's length. (2) Go to 'micro_features_micro_model_setting.cpp' to replace 'yes' with 'on' and 'no' with 'off'. (3) Go to 'arduino_command_responder.cpp' to modify the 'yes' and 'no' judge codes with the following codes:

// replace yes to on
if (found_command[0] == 'o' && found_command[1] == 'n') { 
...
// replace no to off
if (found_command[0] == 'o' && found_command[1] == 'f') {
...

5. Upload project into Arduino board:

Use Arduino IDE to open the project codes, then connect Arduino Nano 33 Sense Ble board with computer. Choose Arduino board and port in IDE, then click Upload. If there is no error appear, then it means we upload the project correctly.

Result: