Fall Detection and Heart Rate Monitoring

Overview

Falls can result in physical and psychological trauma, especially for the elderly. In order to improve the quality of life of these patients this project presents the development of a fall detection and heart rate monitoring system. The wearable device obtains information from accelerometer and heart rate sensor and sends them to the AWS IoT Core. The fall detection is done offline on the chip using a machine learning algorithm.

Setup environment

Download MPLAB X IDE from here https://www.microchip.com/en-us/development-tools-tools-and-software/mplab-x-ide.

Clone AVR IoT Out of the box AWS sensor node example application using the command below:

git clone https://github.com/microchip-pic-avr-solutions/avr-iot-aws-sensor-node-mplab.git

We will be updating this repository code base to suit with the desired application.

Open project MPLAB X IDE > File > Open Projects...

Production > Build Project

After successful build we can flash the AVR-IOT by clicking on the Run Project button as shown below:

We need to setup Wifi to connect to the AWS IoT. When we connect the AVR-IoT board to the computer it appears as CURIOSITY in the file browser as shown below.

We need to open "CLICK-ME.HTM" file which opens up in a default internet browser as shown below.

Here we should fill in the wifi ssid and credentials and download configuration. We should copy the downloaded configuration file to the CURIOSITY directory and in few seconds the board is connected to the internet.

Since we would like to see the data in the private AWS Cloud account, we need to create an account and login to AWS console and go to the AWS IoT Core.

Please follow this link to provision the device to send the data to the private AWS cloud. In the AWS IoT > Test, we can subscribe the topic "<node_name>/sensors" and can see if the data is coming as below:

{
   "Light": 7
   "Temp": 26.75
}

At this point we have a working application connected to the AWS IoT.

For the fall detection and heart rate monitoring we need two sensors:

1. Accelerometer (ADXL345)

2. Heart Rate Sensor (MAX10305)

I have ported the code for the above two sensors from the Arduino libraries to work with AVR-IoT. All the included libraries and code are available in the GitHub repository.

Also, to have Arduino's millis() equivalent function in AVR I have written my own function.

#include <avr/io.h>
#include <avr/interrupt.h>
#include <util/atomic.h>
#include "millis.h"

volatile static uint32_t timerMillis;

// CPU frequency in Hz
#define F_CPU 3333333   // 20MHz/6 (CPU Clock prescalar=6)
//#define PERIOD_VALUE 0xC  // Overflow after 1 ms = 0.001s, (0.001 * F_CPU / 256) - 1, (TCA0 prescalar=256)
#define PERIOD_VALUE 0xD3
// Initializes the use of the timer functions by setting up the TCA timer.
void initTimer()
{
    /* enable overflow interrupt */
    TCA0.SINGLE.INTCTRL = TCA_SINGLE_OVF_bm;

    /* set Normal mode */
    TCA0.SINGLE.CTRLB = TCA_SINGLE_WGMODE_NORMAL_gc;

    /* disable event counting */
    TCA0.SINGLE.EVCTRL &= ~(TCA_SINGLE_CNTEI_bm);

    /* set the period */
    TCA0.SINGLE.PER = PERIOD_VALUE;

    /* set clock source (sys_clk/256) */
    //TCA0.SINGLE.CTRLA = TCA_SINGLE_CLKSEL_DIV256_gc | TCA_SINGLE_ENABLE_bm; 
    TCA0.SINGLE.CTRLA = TCA_SINGLE_CLKSEL_DIV16_gc | TCA_SINGLE_ENABLE_bm; 

}

// TCA overflow handler, called every millisecond.
ISR(TCA0_OVF_vect)
{
    timerMillis++;
    // Clear the interrupt flag (to reset TCA0.CNT)
    TCA0.SINGLE.INTFLAGS = TCA_SINGLE_OVF_bm;
}

// Gets the milliseconds of the current time.
uint32_t millis()
{
    uint32_t m;

    ATOMIC_BLOCK(ATOMIC_RESTORESTATE)
    {
        m = timerMillis;
    }

    return m;
}

Connecting Sensors to the AVR-IoT board

The heart rate and accelerometer sensors are connected over I2C. A Sparkfun Qwiic adapter is used to connect the sensors. The Qwiic adapter pins arrangements (GND, 5V, SDA, SCL) are exactly same (see below) as on the AVR-IOT boards so it was really easy to stick them together.

Heart Rate Monitoring

This is a simple step and the MAX30105 sensor is connected to the AVR-IOT over I2C connection. The code calculates the average BPM (Beats per minute) and sends to the cloud. We can monitor the heart rate and set a threshold if it goes above or below the safe range it should make an alert.

Fall Detection machine learning model selection

This is a bit complicated. We cannot reliably detect only using the accelerometer raw readings. But we can do it using machine learning. I wanted to use TensorFlow Lite but the AVR-IOT board has only 6KB of RAM and more than 3KB is being used by the AWS IoT Node application. Another option could be using the AWS lambda to do the heavy lifting of the Tensorflow Lite. But I always wanted it to do at the edge. So doing neural network based machine learning is not feasible for such a small memory device but we can do tree based for example Decision Tree or Random Forest classifiers. But again it was a problem. The machine learning model needs at least 2 seconds of 3-axis accelerometer data to reliably predict. If we downsample the 3 axis accelerometer floating point data to 64 samples per second, we need 4 x 3 x 64 x 2 = 1536 bytes to keep the 2 seconds data in a ring buffer. Also, to use the snapshot of the data to extract features we need another 1,536 bytes. So the total is more than 3KB and it would not fit in the RAM. One way to cope up with this problem is to keep the data in a 16-bit (2 bytes) signed integer instead of 4 bytes floating point and multiply it by 100 to keep the precision up to two decimal places. While doing feature extraction we divide the data points by 100 and convert back to the floating point. Using this strategy we need only 1,536 bytes and that would fit in the available RAM. Another problem was the Random Forest model keeps all weights in the code and the available space in the flash memory is only 8 KB since AWS Node application uses almost 93% of the flash. So the Random Forest model can not be stored in the flash. While exploring small size model, I came across a recent algorithm SEFR: A Fast Linear-Time Classifier for Ultra-Low Power Devices which uses only one weight for each feature. I gave it a try and it is tiny by design it could easily fit into the available flash.

Training dataset for fall detection

I used the SisFall: A Fall and Movement Dataset which is a dataset of falls and activities of daily living (ADLs) acquired with accelerometer. It consists of 19 ADLs and 15 fall types performed by 23 young adults, 15 ADL types performed by 14 healthy and independent participants over 62 years old, and data from one participant of 60 years old that performed all ADLs and falls. I used Edge Impulse Studio for feature extraction purpose only. The training was done on the extracted feature using scikit-learn Python library. The trained model was ported to the plain C code using micromlgen Python library. The SEFR algorithm currently supports only binary classification. The SisFall accelerometer data collected using ADXL345 sensor were divided into two classes "Fall" and "Move" and converted to m/s^2 and uploaded to the Edge Impulse Studio. The training was done on the 4 features listed below calculated from the raw data:

• Root-mean square (RMS)
• Standard deviation
• Skewness
• Kurtosis

We can see in the Edge Impulse Studio's Feature explorer (below) the "Fall" and "Move" classes have clear separation using the extracted features.

The scripts to upload the data to the Edge Impulse Studio and training Jupyter notebook are available in the Github repository (Please see the Code section for the link). The extracted features are downloaded from the Edge Impulse Studio in the numpy array format and trained using the scikit-learn. The SEFR algorithm are already ported to the EloquentArduino python library which can be used from scikit-learn.

Complete application

Clone the application repository:

git clone https://github.com/metanav/AVRIoT.X.git

Build Configuration

Since this application is using 2 sensors data buffers and code we need to change the build optimization level to "s" in the Project > Properties > XC8 Compiler (as shown below) so that the complete application can be flashed without any memory limitations.

Build and run the application while connecting AVR-IOT board. Go to the AWS IoT > Test and subscribe the MQTT topic. You would be able to see the incoming heart rate data as BPM and fall detection as 0 (fall detected) or 1 (no fall, normal activity).

{
    "Avg BPM": 58,
    "Fall": 0
}

Fall Detection Testing Demo

Conclusion

This project presents a proof-of-concept device especially for elderly people to monitor their health and well-being. This project also showcases that a simple algorithm can be used to solve complex problems and can be run on low-powered resource-constrained devices.