Fall Detection

In this project, we install Jetson Nano with a camera in a house and detect if an elderly person has fallen down using a neural network on the Jetson Nano, and once a fall is detected, it sends a text message to a caregiver.

The purpose of doing this project is elderly people have weak bones which makes them fall down a lot and since they tend to live alone, it might take several hours before it is known that they have fallen. So we want to use AI on Jetson Nano to detect automatically falls.

For this to work, we need to process video data in real time which is about 10 image frames per second. Since neural networks typically require a lot of computation this rules out most popular neural network architectures. However recently an architecture called MobilenetV2 was proposed that can process images in real time. However this architecture is for image classification but not fall detection. So we had to modify this architecture to make it work for fall detection. The accuracy is around 95%.

The algorithm has two stages. In the first stage, a Raspberry PI camera connected to the Jetson Nano captures the video stream and this video stream is read by the block that computes the optical flow. This optical flow is passed into a neural network which outputs a binary output of fall or no-fall. When a fall occurs, a text message is sent via Amazon Web Services Simple Notification Service.

The neural network used is a variant of the MobilenetV2 architecture. We took the original MobilenetV2 architecture which is pretrained on the ImageNet dataset. This network takes RGB images as inputs and predicts this input to be 1 out of 1000 classes. Input images are of size 224x224x3 and output is a binary vector of size 1000.

We modified the input and output layers of the network network to take optical flow of size 224x224x20 as input and outputs vectors of size 101. This is trained on the UCF101 dataset which has around 14000 videos with 101 different action categories. This lets the weights of the network learn optical flow. Then we modify the final layer of the network to generate output vector of size 2. We retrain the final layer of this network on the URFD fall dataset. Now the network is able to distinguish between falls and no falls

I couldn't save the BOM in the "Things" tab (possibly a bug in hackster) so I'm writing it down here.

Complete BOM
Detail the hardware, software and/or tools used.

Jetson Nano - The Neural Network is run on this

Raspberry PI camera - https://www.raspberrypi.org/products/camera-module-v2/

Keras, Tensorflow - The neural network is written using these libraries

Opencv - Used to compute optical flow which is then fed into the neural network

Amazon Web Services Command Line Interface - Used to send text messages to a caregiver when a fall occurs.