TinyML Course #7 Go Tinier! Cortex M0+ Posture Recognition

This is the last article of the TinyML course series – in the previous articles we have discussed how to train and deploy machine learning models for audio scene classification, anomaly detection, speech recognition and other tasks to Wio Terminal, a compact production-ready development board from Seeed studio. All of the articles were published on Hackster.io and for videos, you can have a look at TinyML course with Wio Terminal playlist on YouTube.

UPDATED 03/29/2022. I try my best to keep my articles updated on a regular basis and based on your feedback from YouTube/Hackster comments section. If you'd like to show your support and appreciation for these efforts, consider buying me a coffee (or a pizza) :) .

Wio Terminal while being convenient for experiments due to presence of multiple built-in sensors and a case, might be a bit too bulky for some applications for example, wearables. In the last project we’ll go even Tinier and use Seeeduino XIAO boards – namely the original XIAO and newer XIAO 2040 and will briefly mention soon-to-be-released XIAO BLE.

Specks-wise, original XIAO at the time of launch was likely the smallest M0 development board available and was packing quite a punch for its size with ARM® Cortex®-M0+ 48MHz microcontroller(SAMD21G18) and 256KB Flash, 32KB SRAMOriginal XIAO – as you can see it is a bit larger than a thumb nail.

Later RP2040 chip arrived and delivered even better specs Cortex M0+ design. Both are quite capable of running the tiny neural network we will have for this project, but if you have some more demanding applications it does make sense to choose XIAO RP2040 over the original XIAO.

As a software engineer I, like many of you I’m sure, spend a lot of time in front of the glowing screen on my chair. And later in the day it becomes difficult to maintain a proper pose.

If only there was a way to make a device that could learn your specific body position for proper and wrong poses and warn you when you slouch too much or go into “Python pose”… Wait a moment, there is!

The best sensor for the task that will provide the data for machine learning model is obviously accelerometer. All of the XIAO series boards, being very small do not come equipped with accelerometer sensor. While we could use a XIAO expansion board for development and testing, it eliminates the low-footprint advantage XIAO boards have. If you are going to create your own product, the better option would be to create your own custom PCB board for the chip or SoM. I asked our hardware engineer to design a simple carrier board for XIAO, that would include LIS3DH accelerometer, a buzzer and a battery connector with a power switch.

Then we used Seeed studio Fusion service to print some PCBA samples – for that go to https://www.seeedstudio.com/fusion_pcb.htmland upload Gerber files, which contain the PCB design and choose the proper parameters for the board, such as number of layers, base material, minimum drill hole size and so on. We can see that a simple 2-layer board is approximated to cost 4.9 USD for 10 pieces plus shipping cost.

If you’d like to repeat the experiment without custom PCB, you can connect Grove LIS3DH accelerometer module to XIAO expansion board and start collecting the data. I collected 3 data samples for each posture, 60 seconds each with device attached to t-shirt on my back.

For each sample, I maintained the same pose, but included some arm, head and torso movements to simulate normal activity.

I have chosen 5 seconds time window with window shift of 1 second and Flatten processing block, since we are dealing with very slow moving data. A very plain fully connected network provided a good accuracy. Here is the link to public version of the Edge Impulse project.

Some improvement can be made by collecting more data and making sure proper and improper postures can be recognized with some variations in device positioning on the clothes. Since the device is thought to be individual usage device it does not need to generalize to different people’s postures and can be easily re-trained. You can check how well it detects your postures after training in Live classification tab.

After you’re satisfied with accuracy download the resulting model as Arduino library and copy it to your Arduino sketches/libraries folder. You can download sample code, that would collect 5 second sample, performs the inference and turn on the buzzer if one of the improper poses are detected.

void loop()
{

    ei_printf("Sampling...\n");

    // Allocate a buffer here for the values we'll read from the IMU
    float buffer[EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE] = { 0 };

    for (size_t ix = 0; ix < EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE; ix += 3) {
        // Determine the next tick (and then sleep later)
        uint64_t next_tick = micros() + (EI_CLASSIFIER_INTERVAL_MS * 1000);

        lis.getAcceleration(&buffer[ix], &buffer[ix+1], &buffer[ix + 2]);
        buffer[ix + 0] *= CONVERT_G_TO_MS2;
        buffer[ix + 1] *= CONVERT_G_TO_MS2;
        buffer[ix + 2] *= CONVERT_G_TO_MS2;

        delayMicroseconds(next_tick - micros());
    }

    // Turn the raw buffer in a signal which we can the classify
    signal_t signal;
    int err = numpy::signal_from_buffer(buffer, EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE, &signal);
    if (err != 0) {
        ei_printf("Failed to create signal from buffer (%d)\n", err);
        return;
    }

    // Run the classifier
    ei_impulse_result_t result = { 0 };

    err = run_classifier(&signal, &result, debug_nn);
    if (err != EI_IMPULSE_OK) {
        ei_printf("ERR: Failed to run classifier (%d)\n", err);
        return;
    }

    // print the predictions
    ei_printf("Predictions ");
    ei_printf("(DSP: %d ms., Classification: %d ms., Anomaly: %d ms.)",
        result.timing.dsp, result.timing.classification, result.timing.anomaly);
    ei_printf(": \n");
    for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) {
        ei_printf("    %s: %.5f\n", result.classification[ix].label, result.classification[ix].value);
    }
#if EI_CLASSIFIER_HAS_ANOMALY == 1
    ei_printf("    anomaly score: %.3f\n", result.anomaly);
#endif
    
  if (result.classification[1].value > ALARM_THRESHOLD || result.classification[2].value > ALARM_THRESHOLD)
  {     
  tone(BUZZER_PIN, 523, 250);
  delay(250);
  noTone(BUZZER_PIN);
  delay(250);  
  tone(BUZZER_PIN, 523, 250);
  delay(250);  
  noTone(BUZZER_PIN);    
  }

}

Since it is relatively slowly changing data and we do not need fast response times, normal sequential inference pipeline suits this application well.

A step above would be to use the newest XIAO BLE and connect the device to user’s smartphone, which would allow for better alerts, statistics and so on.

Soon-to-be released Seeeduino BLE comes in two versions: BLE and BLE Sense. Both version are based on Nordic Semiconductor nRF52840 SoC, but Sense version is also equipped with IMU sensor and PDM Microphone, which makes it a great choice for TinyML projects, such as the one described in this article. Stay tuned to price announcement (the price is known now, it is 9.9 USD for BLE and 15.99 USD for BLE Sense)!

Hope you enjoyed my TinyML course video series and learned a lot about Machine Learning on microcontrollers. Tinkering with Machine Learning on smallest of devices has been truly an eye-opening experience for me, after executing much larger ML models on SBCs and servers. If you would like to continue learning on that topic, have a look at free courses by Coursera and Edge impulse and also Codecraft, a graphical programming environment that allows creating TinyML application and deploying them to Wio Terminal.