How It's Helpful

Dan Thyer • Posted by Sushma Jampani

Published June 24, 2024

Fall Alert System with YOLOV8 Pose Estimation & Alexa

The Verbal Echo Dot can trigger an alert when a fall is detected through a live video stream, enabling remote notifications for caretakers.

BeginnerFull instructions provided6 hours98

Fall Alert System with YOLOV8 Pose Estimation & Alexa

Things used in this project

Hardware components

Amazon Alexa Echo Dot

Amazon Alexa Amazon Echo

Camera (generic)

Amazon Alexa Echo Buttons

Internet connected Computer/WIFI enabled Smart device

Particle Argon

AITRIP 8 Pack 3V 1 Channel Relay Power Switch Module with Optocoupler Opto Isolation High Level Trigger for IOT ESP8266 Development Board

USB hub to provide ports to power the particle and 2 echo buttons.

Batter Eliminator

Hand tools and fabrication machines

Soldering iron (generic)

Solder Wire, Lead Free

Story

Fall Detection and Alert System

Imagine a smart home system that watches over your loved ones and sends an alert so you can attend to them immediately. Our fall detection system uses pose estimation technology to monitor movements and detect falls in real-time. Alerts are instantly sent via Alexa announcements, ensuring caregivers can act quickly.

How It's Helpful:

This system offers peace of mind by providing real-time alerts, allowing caregivers to respond quickly. It's especially beneficial for those caring for individuals with conditions like Rett syndrome, where falls can have serious consequences. Integrating with Alexa devices ensures timely, clear, and accessible alerts throughout the home. This innovation enhances safety and provides caregivers with personal time, knowing their loved ones are being monitored.

Checklist of Items to Buy:

[ ] Multiple cameras for full room coverage
[ ] Amazon Echo with battery base
[ ] Amazon Echo Dot
[ ] Microcontroller similar to the IoT brain on the DUO lift
[ ] Soldering equipment
[ ] Echo Photon Bridge hardware
[ ] Battery Eliminator
[ ] Enclosure Box
[ ] USB hub
[] Velcro

Other Open-Source S/W:

Pose Estimation Model:

Develop an algorithm that identifies fall and pre-fall movements using skeletal coordinates from a pose estimation model.
Identify the location of specific key points (e.g., joints) as 2D [x, y] or 3D [x, y, visible] coordinates with confidence scores.
Process data from multiple cameras to cover the entire room without blind spots.
Validate the algorithm’s accuracy in detecting potential falls based on the key points and their relation to each other.

Amazon Echo Integration:

Integrate the fall detection system with Amazon Echo devices to send announcements.
Use Amazon Echo with a battery base and Echo Dot to provide notifications.
Ensure hardware integration to trigger announcements, bypassing standard API limitations with donated hardware.
Solder and configure the necessary microcontroller for triggering announcements, similar to the IoT brain on the DUO lift.
Test announcements to ensure they are clear and prompt.

Echo Photon Bridge:

This code receives messages for fall detection and alerts when Parnika is out of camera range. It also enables Echo messaging with smart home integration using the Echo Photon Bridge. Alexa recognizes our device as a light switch, allowing notifications to be turned on or off. The firmware sends a message by using a relay to trigger the switch on the Echo button.

Additionally, we also got a USB hub to provide ports to power the particle and 2 echo buttons.

Echo Buttons: Thank you, Jeremy Proffitt, for donating these Echo buttons for a greater cause.

USB Hub:

We got an enclosure too. All will fit inside, and the beautiful part isthat it will have a power cord attached but nothing else, and we will never need to open it because nothing will run on batteries. We got all the components for the enclosure at Lowes. All of the inside components are velcroed in with super-strong Velcro. I think it will be safe to ship, and it looks great for Rama's house!

Real-time Alerts:

Ensure real-time processing and alerting through Alexa devices.
Trigger an alert within seconds of detecting potential fall movements.
Send notifications to all Alexa devices in the home, including Amazon Echo and Echo Dot.

System Reliability:

Conduct extensive testing in a home environment to minimize false positives and negatives.
Ensure the system works continuously without interruptions.

Why Use Hardware Components? To avoid hackers getting into your Internet system,

Code Overview

This code receives fall detection messages and alerts when the subject is out of camera range, enabling Echo messaging with smart home integration using the Echo Photon Bridge. The firmware triggers notifications by using a relay to activate the Echo button switch.

User Feedback:

They appreciate the real-time alerts and the peace of mind it provide.

Additionally Collect feedback from caretakers to improve the system.

Future Enhancements will include:

Provide a way for caretakers to report issues or suggest improvements.

Implement feedback-driven updates to enhance system performance.

Team and Acknowledgments

Thankyou Jeremy Proffitt for Donating these Echo buttons for a greater cause.

Special thanks go out to the Rehrig India Team and the RTC Rehrig team in Dallas, Texas, for their important contributions to the project's success.

Schematics

Code

source code for particle firmware

// This #include statement was automatically added by the Particle IDE.

#include <EchoPhotonBridge.h>
 
// Include Particle Device OS APIs

#include "Particle.h"
 
// Let Device OS manage the connection to the Particle Cloud

SYSTEM_MODE(AUTOMATIC);
 
// Show system, cloud connectivity, and application logs over USB

// View logs with CLI using 'particle serial monitor --follow'

SerialLogHandler logHandler(LOG_LEVEL_INFO);
 
int _outOfRangeOutput = D7;

int _outFallDetect = D0;

int _mode = 1;   // 0 = do not send announcement,  1 = send announcement

EchoPhotonBridge epb;
 
int fallDetectOnOff(int device, bool onOff, String rawParameters) {

    if(onOff == TRUE) {

        _mode=1;  

    } else {

        _mode=0;

    }

    return 0;

}
 
// setup() runs once, when the device is first turned on

void setup() {

    RGB.control (true);

    //Particle.function("fall_detection", fallDetectionHandler);

    Particle.function("out_of_range", outOfRangeHandler);

    Particle.function("fall_detection", fallDetectionHandler);

    epb.addEchoDeviceV2OnOff("fallDetect", &fallDetectOnOff);

    pinMode(_outOfRangeOutput,  OUTPUT);

    pinMode(_outFallDetect,  OUTPUT);

    WiFi.setCredentials("ACT101500027428", "93032849");

    WiFi.setCredentials("Rehrig-Guest", "Welcome!");

    WiFi.setCredentials("DanPhone", "LogicalIoT");

}
 
// loop() runs over and over again, as quickly as it can execute.

void loop() {

    if (_mode == 1) {

        RGB.color(0,255,0);

        delay(500);

        RGB.color(0,100,0);

        delay(500);

    }

    else {

        RGB.color(255,0,0);

        delay(500);

        RGB.color(100,0,0);

        delay(500);

    }
 
}
 
int outOfRangeHandler(String data) {

    if (_mode == 1) {

        digitalWrite(_outOfRangeOutput, HIGH);

        delay(500);

        digitalWrite(_outOfRangeOutput, LOW);

        return 1;

    }

    return 0;

}

int fallDetectionHandler(String data) {

    if (_mode == 1) {

        digitalWrite(_outFallDetect, HIGH);

        delay(500);

        digitalWrite(_outFallDetect, LOW);

        return 1;

    }

    return 0;

}

Credits

Dan Thyer

Posted by

Sushma Jampani

Thanks to Dan Thyer.

Fall Alert System with YOLOV8 Pose Estimation & Alexa