Robot Fall Detection with Edge Impulse

The Idea

Imagine creating a simple robot that can navigate around a room on its own using some kind of distance sensor. It does well at first, since it can dart between the legs of a table and avoid running into the wall. But there is a problem: it is prone to falling off the edge of the stairs. After falling down, it continues to attempt to run as normal, which wastes power and can cause more damage. Normal methods of checking if a fall has happened involves checking if an accelerometer has gone above a certain threshold, but this can give false positives. Instead, a model can be trained to recognize the difference between a normal movement and an actual fall with far greater accuracy than a simple threshold.

Why Is This Important?

This kind of technology has wider-reaching implications beyond just seeing if a robot has fallen down. Think of a product such as Life Alert. This could be greatly improved by making it run a basic ML model to more accurately determine if a person could be in distress. It could also be used in a field that deals with transportation of goods, as drivers or warehouse managers could be notified that a certain item has fallen and possibly been damaged during shipping.

The Robot

Designing a robot chassis from scratch and gathering the necessary components takes a long time and can become very costly. Rather, using a kit is a much faster and cheaper option. I went with one of Elegoo's Smart Car V3 Kits, which contain everything needed for a robotic car, including motors, a motor controller, rechargeable batteries, and a servo mount. The kit is solidly constructed from laser-cut acrylic panels and milled aluminum pieces. If you're curious about how to assemble it, you can view on of my other projects here.

Hardware Setup

The brain of this project is the Arduino Nano 33 BLE Sense, which contains a plethora of very useful sensors. Since I wanted to detect falls, I used the onboard IMU to measure accelerations. The robot is able to navigate autonomously with an HC-SR04 mounted on a servo motor that can pan it to the left and right. The two pairs of motors are driven by a single L298N driver, and power is provided with two 3.7V LiPo batteries.

Gathering Data

Accelerometer readings are generated at consistent intervals by the sensor, which limits how many times it can be read. The Arduino_LSM9DS1 library is responsible for providing a simple interface to talk with the sensor via I2C. Accelerometer measurements are taken at regular intervals, along with distance values from the HC-SR04. Once a scan has taken place, the maximum value is found and the car moves in that direction. IMU readings get outputted over USB to a host computer that is running the edge-impulse-daemon via Edge Impulse's CLI. For information on how to set that up, visit their documentation page here.

Recognizing a Fall

After there is plenty of data of both falling and benign samples, it's time to train the model. I went with the basic 3-axis input and a standard Keras model. You can also see the settings I used below to filter and smooth the data.

Testing the Model

So after training, it becomes necessary to test the model and ensure that it is accurate. I let the robot run a bit and collected some additional data, including a fall. Then after running the live classifier I can observe that it was successful in identifying the difference. The model can then be downloaded as an Arduino library and run on the BLE Sense.

You can find the dataset and model information on this project's public Edge Impulse page here.