Medical Alert System Using TinyML Fall Detection and BLE

The Problem

With one in four seniors experiencing a fall every year, falling is a serious and dangerous obstacle in life. Out of the annual 36 million senior falls, there are around 950, 000 hospitalizations and 32, 000 deaths. The risk and consequences of falling serve as a significant hindrance to the freedom of older adults.

While medical alert services like Life Alert offer possible solutions, they have fundamental problems that need to be remedied. Medical alert services either charge hefty fees up to 100 dollars a month, have a tremendous initial cost, or have some combination of both. Also, many insurance companies do not cover medical alert services. The Apple Watch's fall detection service is fantastic, but not everyone can afford one. The other issues of most services include not being able to leave a particular base station at home and being pointless if the person is either unconscious or cannot move to press the button.

Fall Detection and Why TinyML

In the event that someone is unconscious or is unable to press the button, automatic fall detection is a requirement. However, traditional heuristic models of fall detection fail and are more of a disappointment than a feature. Machine Learning allows for a somewhat reliable fall detection that can help in these situations.

I am not the first who has had this idea (i.e., the existence of fall detection datasets). Other research papers suggest using a machine learning model either in the cloud or running on local devices: methods that would be too resource-intensive, waste battery life, and lack reliability. Offloading the processing doesn't really work as it requires the data to be sent from the device through wireless communication: significantly reducing the reliability and portability. Constant wireless communication would significantly impact battery life, and no real device could stream data 24/7.

TinyML is the solution to all of these problems as it moves the neural net from the cloud/external devices. Tools like TensorFlow Lite Micro make running such a model much easier than before. With TinyML, the system is a lot less complex. Communication is reduced from a constant stream to only when a fall is detected; it no longer needs that reliable of a connection. The latency is significantly reduced, and the device respects the user's privacy. Now the only limitation is whether the smartphone can send the message.

How Does it Work?

Every two seconds, the TensorFlow Lite Micro model is used to predict whether the user has fallen or not. If the user has not fallen, nothing happens. If a fall was predicted or if the button was pressed, a buzzer turns on, and the 30 second grace period. During the grace period, the user can cancel the alert by pressing the button. If the alert is not canceled, a message is sent to the user's phone using Bluetooth Low Energy. The phone app, which runs in the background, then sends an SMS message with the user's location to the emergency phone number that was saved in the app. After receiving the message, the emergency contact can assist the user as they have their location. If a mistake was made—which should be rare with the grace period—the user can always text the emergency contact and tell them it was a mistake.

New Functionality

The device does not have to be limited to seniors. Kids can wear it at amusement parks. It also makes you feel safe completing daily activities like hiking or leaving your home. In addition, if you were to be attacked and unable to call, the device can alert others for you. The device's purposes aren't restricted to medical reasons but can be a general alert.

The Model

The best model for fall detection would be a Conv-LSTM—mixing a convolutional neural network with an LSTM—allowing the model to get the context around a fall. However, LSTMs don't work with TensorFlow Lite Micro, so I just went with a CNN with a window size of around two seconds of accelerometer data.

The model is mostly trained on the SmartFall dataset. When I first ran the model on real data, it wasn't good: common movements like waving, shaking, clapping, or just moving hands around were detected as falls. To mitigate this, I collected a decent amount of my own data, trying to mimic the movements that were typically mistaken as falls. I also augmented the data by rotating it on the x, y, and z axes by random angles to allow for better generalization and for the detection to work in many different orientations. Without the augmentation, I get a validation accuracy of 98 to 99 (with a worse actual accuracy), while I get around 95% with the augmentation. Check out the Google Colab to see exactly how I trained my model.

On the Arduino, the accelerometer data is downsampled to a frequency of 31.25 Hz as that was the frequency the SmartFall database was collected in. When I first ran the model on the Arduino, I used to have a quantized version; however, I found that just converting the unquantized model would actually allow for much better accuracy. Since the model is unquantized, the program sketch is pretty big but still runs fine on the Nano BLE 33 Sense.

The Hardware

The hardware is quite simple and consists of a LiPo battery, Arduino Nano BLE 33 Sense, buzzer, and transistor to control the buzzer. The Arduino Nano BLE 33 Sense is great for a prototype as it supports TinyML and BLE; however, the BLE 33 Sense has many features on the board that are not needed for this project (taking up space and battery life).

The custom enclosure was designed in Fusion 360, and 3D printed to fit all of the parts. The enclosure is bigger than I would like, and I wear it using a velcro strap. Still, these are relatively small inconveniences: a better design can easily be made, and the code will only need minor adjustments. A future design will fit all of the components on one board, significantly reducing the device's size. Other wearable projects have reached extremely small sizes, and fall detection has the same potential. The cost is kept low, around 30 dollars, but it would cost around 20 dollars if a final product were ever produced.

The Phone App

The fall detection is just that: detection. Without some device that communicates with the outside world and brings help, it is just pointless. Most medical alert systems use some base station that is positioned somewhere in the home, but I use a phone app. It makes the device portable anywhere where there is cellular, and SMS is already a well-established form of communication.

The app's UI is really simple, and all the user has to do is enter an emergency phone number to alert when they fall. The app saves the phone number, but they can still change it. The main part of the app runs as a foreground service, meaning that it runs without the app having to open. The foreground service—using the BLESSED library—handles it all: it scans, connects, reconnects, and listens to the device using BLE. When the device notifies the app of a fall, it uses Google Play Services to get the current location and texts the coordinates as well as a link to Google Maps to the phone number saved.

Demonstration

Problems and Future Solutions

If you punch really fast or shake as hard as you can, the fall detection still activates. While the model isn't ready to be released as a widescale product, it still works great for fall detection. Making a new dataset to make a product-ready model would be its own little mini-project, but it is worth it for the massive potential an improved and reliable model would provide. I have already started collecting my own data, but I still need more samples and a larger range of people before it is complete. The current design is also a little bulky, and the battery life doesn't last forever, but these can easily be fixed by making a custom PCB for the project: all of the components would fit on one board, and more energy-efficient parts could be used. Even with all of its faults, the TinyML Fall detection makes this medical alert system have the potential to rank among the best. The low cost, edge technology, and portability are unrivaled.

Current Improvements

I've improved on the design by making a schematic of a new PCB. The new design still uses an nrf52840 but replaces the accelerometer with the ultra low-power Bosch BMA400. The smart functionalities of the BMA400 will allow for the nrf52840 to sleep more, saving much more power.

The PCB is a four copper layer circle with a radius of 12.5 mm, making it significantly smaller than the Nano 33. Also, since it is a circle, it can fit into smaller enclosures than a rectangular shape. I would order and assemble the PCB, but many components are currently out of stock.