Published February 17, 2016 © MIT

A fall detection system based on Arduino, Windows and Azure

A fall detection system based on Arduino, Windows 10 and Microsoft Azure

AdvancedFull instructions provided2 hours46,661

Best Project - Rest of the World

World’s Largest Arduino Maker Challenge

A fall detection system based on Arduino, Windows and Azure

Things used in this project

Hardware components

Arduino UNO

HC-06 Bluetooth Module

Adafruit ADXL345 accelerometer module

Male-male and male-female jumper wires

Mini-breadboard

Lumia 1520 or other Windows Phone 10 compatible device

ORICO Bluetooth module

Software apps and online services

Microsoft Windows 10 IoT Core

Microsoft Azure

Microsoft Visual Studio 2015

Arduino IDE

Story

Introduction:

Falls those leads to fatal injury have become a great challenge that cannot be neglected for elderly people. In the last few years, different kinds of approaches have been proposed in fall detection area, which can be explained and categorized into three types: wearable device based, ambience sensor based and vision based [1]. First of all, wearable devices usually take advantages of embedded sensors to detect the motion and location of the body, such as accelerometer, magnetometer and gyroscope [2, 3]. And the cost of wearable device based approach is quite low, as well as the installation and operation is not complicated for the elderly. Secondly, ambience based approach always use pressure sensors to detect and track body. This solution is also cost-effective and easy-deployment [4, 5]. However, the possibility of sensing objects other than human bodies posts a remarkable challenge to the detection accuracy of this approach. Last but not least, vision based solution makes full use of deployed cameras to monitor all the objects within the range, including human bodies [6, 7]. There is less intrusion into people’s daily life than the above two approaches, while the observation space is limited and ubiquitous monitoring can’t be achieved.

In this project, a surveillance system based on Arduino, Windows 10 and Microsoft Azure for real-time fall detection is proposed. Raw data of three-dimensional accelerometer are provided by Arduino with ADXL345. Windows 10 device makes use of this information to get the orientation of the subject by efficient data fusion and fall detection algorithms. Microsoft Azure services and Mobile/PC application are also designed to achieve seamless data processing, analyzing, storing and acquiring any time from any place as long as they have access to the Internet. The system architecture is shown is Fig. 1 as follows.

Fig.1: System Architecture

References

[1] L. Jian and T. E. Lockhart, "Development and Evaluation of a Prior-to-Impact Fall Event Detection Algorithm," Biomedical Engineering, IEEE Transactions on, vol. 61, pp. 2135-2140, 2014.

[2] P. Pierleoni, A. Belli, L. Palma, M. Pellegrini, L. Pernini, and S. Valenti, "A High Reliability Wearable Device for Elderly Fall Detection," Sensors Journal, IEEE, vol. 15, pp. 4544-4553, 2015.

[3] R. Freitas, M. Terroso, M. Marques, J. Gabriel, A. Torres Marques, and R. Simoes, "Wearable sensor networks supported by mobile devices for fall detection," in SENSORS, 2014 IEEE, 2014, pp. 2246-2249.

[4] Z. Xiaodan, H. Jing, G. Potamianos, and M. Hasegawa-Johnson, "Acoustic fall detection using Gaussian mixture models and GMM supervectors," in Acoustics, Speech and Signal Processing, 2009. ICASSP 2009. IEEE International Conference on, 2009, pp. 69-72.

[5] L. Yun, K. C. Ho, and M. Popescu, "Efficient Source Separation Algorithms for Acoustic Fall Detection Using a Microsoft Kinect," Biomedical Engineering, IEEE Transactions on, vol. 61, pp. 745-755, 2014.

[6] M. Yu, S. M. Naqvi, A. Rhuma, and J. Chambers, "One class boundary method classifiers for application in a video-based fall detection system," Computer Vision, IET, vol. 6, pp. 90-100, 2012.

[7] E. E. Stone and M. Skubic, "Fall Detection in Homes of Older Adults Using the Microsoft Kinect," Biomedical and Health Informatics, IEEE Journal of, vol. 19, pp. 290-301, 2015.

Principle of Fall Detection with Posture Recognition

In this project, the fall detection algorithm is designed according to Bharadwaj Sreenivasan‘s Fall-detection-in-Android. There are two modules in the algorithm, posture recognition and fall detection. Firstly, we read the x, y and z values form the accelerometer. Then we compute the L2 norm of the x, y, and z. This is used by posture recognition and fall detection module.

In the Posture recognition module, the user postures are classified into three basic postures: sitting, standing and walking. The values of “y” is applied a threshold to find out the orientation. Using the rate of variation of L2 norm w.r.t mean acceleration due to gravity (9.8) we classify the data into walking or just transition between states.

The fall detection module searches for particular pattern in the signal. Fig.2 represents a typical pattern in L2 norm during a fall event.

Fig.2: Fall Pattern

If the difference between consecutive minima and maxima is greater the 2g, the output is decided as a fall. The final decision on event of a fall is based output of both posture recognition and fall decision module. When a fall is detected, decision from posture recognition module tells us whether it’s a false alarm. If the state is still walking, the fall event decision can be discarded. Table 1 summarizes the final decision.

Tab. 1: Decision on fall

Prerequisites

In this project, for Arduino device, the Bluetooth communication is achieved by HC-06, which is a cheap, small packaged module. For windows 10 device, if you use Surface Pro and Lumia 1520, you will not need Bluetooth module. However, if you use Raspberry Pi or MBM as Windows 10 IoT Devices, you will need Bluetooth dongle. Refer to our MVP David Jones’s Generic Serial Bluetooth with Win 10 IoT-Core RPI2 to setup the environment.

Note: It will take 2-3 hours to complete the prerequisites if you don’t have Windows 10 and Visual Studio 2015 already setup but most of that time is unattended.

What You'll Need

Parts:

1. Arduino Uno and standard A/B USB cable

2. HC-06 Bluetooth module

3. ADXL345 accelerometer module

4. Male-male and male-female jumper wires

5. Mini-breadboard

6. Lumia 1520 or other Windows Phone 10 compatible device

7. ORICO Bluetooth module

Project Instructions

Step 1: Connect the accelerometer module ADXL345 to the Arduino

Time: 5 minutes

Parts: Arduino Uno R3, ADXL345, jumper wires, mini-breadboard

Tools: N/A

The ADXL345 is a small, thin, low power, 3-axis MEMS accelerometer with high resolution (13-bit) measurement at up to +-16 g. Digital output data is formatted as 16-bit twos complement and is accessible through either a SPI (3- or 4-wire) or I2C digital interface.

The ADXL345 supports both SPI and I2C, we are just going to talk I2C here in this project. I2C is a 2-wire serial connection, so we need to connect the SDA (Data) and SCL (Clock) lines to Arduino UNO for communication. On Arduino UNO, SDA is on analog pin 4, and SCL is on analog pin 5. Make sure both VCC and the CS pins on the sensor are connected to 3.3v. We connect CS to 3.3V to tell the sensor we will be using it as an I2C device, and not an SPI device.

Fig. 3: Sketch for ADXL345 Connection

Step 2: Connect Bluetooth module

Time: 5 minutes

Parts: Bluetooth module, jumper wires

Tools: N/A

In this project, we use HC-06 as the Bluetooth module - it is much cheaper than BlueSMiRF Silver module from SparkFun. Furthermore, HC-05 has the ‘full’ firmware on it, many AT commands, and can be both master and slave module. The HC-06 firmware on the other hand only can be a slave device, with very limited AT commands. The table below shows all the HC-06 firmware commands with the response:

Tab. 2: HC-06 firmware AT commands

First of all, plug the Bluetooth module into the breadboard to make connections easier. To power the module, run a jumper from the Arduino’s 5V power rail to the module's Vcc pin. And run a male-male jumper from the module's GND pin to any of the GND pins on the Arduino. To establish the serial communications, connect a male-male jumper from the Bluetooth module's TX pin to the Arduino's RX pin (digital pin 0) and another from the module's RX pin to the Arduino's TX pin (digital pin 1).

Note: You will not be able to upload sketches from the Arduino IDE to the Arduino when you have the Bluetooth's TX and RX pins connected to the Arduino. Disconnect them when you need to upload a sketch and reconnect them once the sketch has been successfully uploaded. In the sketch below, it is shown that HC-05 is used. Actually, as it is explained above, both HC-05 and HC-06 are OK.

Fig. 4: Sketch for Bluetooth Module Connection

After the Bluetooth and accelerometer connection, the real thing is shown like this:

1 / 2 • Fig. 5: Arduino Uno with Bluetooth and Accelerometer

Step 3: Pairing Bluetooth module with Windows 10 Device

Time: 5 minutes

Parts: Bluetooth module, Windows 10 device, ORICO Bluetooth module(Optional)

Tools: N/A

The Bluetooth module runs the SPP (Serial Protocol over Bluetooth) protocol. So any device supporting SPP can connect to it. On a Windows 10 device, this looks like a virtual COM port. Before connecting, make sure the module is powered and ready to pair. The red LED on the module indicates the status: blinking means “ready to pair”, while steady on means “paired”.

On Windows 10 for PC, just navigate to Settings->Devices->Bluetooth, make Bluetooth open, and then “HC-06” module will be shown on the screen. Tap to pair the device, and enter the device’s paring code. Please do remember that the default pairing code is 1234, and the default serial port speed is 9600 Baud. Then, the Bluetooth page is shown like this:

Fig. 6: Bluetooth Pairing on Windows 10 PC

For Windows 10 for Mobile, navigate to Settings->Devices->Bluetooth, and do the same steps described in the above section. And the paired device page is shown like this:

Fig. 7: Bluetooth Pairing for Windows 10 Mobile

For Windows 10 IoT Core device, for example Raspberry Pi 2, MBM and Dragon Board 410c, the pairing process is much different. Firstly, for Raspberry Pi 2 and MBM who don’t have on-board Bluetooth module, we should buy a Bluetooth USB dongle and plug in. Just refer to the Hardware Compatibility List and choose a verified Bluetooth USB dongle.

Fig.8: Verified Bluetooth USB Dongle for Windows 10 IoT Core

In this project, we choose a ORICO BTA-403 module.

If you try this project on Dragon Board 410c, then you don’t need a Bluetooth dongle because Dragon Board 410c has an on-board Bluetooth module.

Plug the ORICO BTA-403 Bluetooth USB dongle in the USB interface of Raspberry Pi 2 as follows.

Fig.9: Raspberry Pi 2 with ORICO BTA-403 Bluetooth USB Dongle

Once the ORICO BTA-403 Bluetooth USB dongle is plugged in, it will show messages on the default app screen of Windows 10 IoT Core device.

Fig.10: Information of Bluetooth module that display on Windows 10 IoT Core device

For Bluetooth pairing on Windows 10 IoT Core device, refer to the Pairing a BLE Device and GATT Attribute Table Dump Tool page and follow the “Step 1 – Step 6” described in the “Pairing the Sensortag” Section. Here we should use SSH to connect to the Windows IoT Core and run the command line tool to configure the device.

Note: do remember that the BR mode should be chosen on the Main Menu. And the default pairing code is 1234.

After successful pairing, the Bluetooth Page of the device is shown like this:

Fig. 11: Bluetooth Pairing on Windows 10 IoT Core

Step 4: Create the Azure Storage Account

Time: 10 minutes

Parts: N/A

Tools: Azure Account

In this project, we will need Azure Storage Table to store the data from Windows 10 IoT Core device. So, we can create it as follows.

(1) In the Azure Management Portal (https://manage.windowsazure.com), click the on the "STORAGE" icon along the left to view your existing storage accounts (if any), then click the "+NEW" button in the lower left corner.

(2) In the "NEW" panel, select "DATA SERVICES" | "STORAGE" | "QUICK CREATE". Complete the fields of “URL, Location/Affinity Group and Replication”, then click the "CREATE STORAGE ACCOUNT" button.

(3) Wait for the new storage account's status to show as "Online".

Fig.12: Create Azure Storage Account

(4) Select the newly created Storage account, and then click MANAGE ACCESS KEYS at the bottom of the page. Copy the Storage account name and one of the access keys.

Fig.13: Storage Account Name and Access Key

Step 5: Create Azure Storage Table

We use Azure Storage Explorer (It a third party open source tool available on codeplex: https://azurestorageexplorer.codeplex.com) to create the Azure Storage Table.

(1) Go to add account and fill the account name and access key. Once the account established, you can access the storage that shows blob containers, Queues and tables.

(2) Select the table and click the new button and create a new table named “Accelerometer Table” as follows.

Fig.14: Create Azure Storage Table

Step 5: Download and Deploy the Code

Time: 10 minutes

Parts: N/A

Tools: N/A

First of all, now that we have everything wired up for Arduino, download the “ADXL345_HC06.ino” project from (provided in the code section below) and load it into the Arduino IDE on your PC. Be sure to unplug the HC-06 module's TX/RX jumper wires and then upload the sketch. Reconnect the TX/RX jumper wires to the Arduino's digital 0 and digital 1 pins. Plug the power and the application will run on Arduino Uno.

Secondly, download the “GenericBluetoothSerialUWApp” project from github. Use Visual Studio 2015 with Update 1 to load “GenericBluetoothSerialUWApp”. This app is designed with the help of our MVP David Jones’s Generic Serial Bluetooth with Win 10 IoT-Core RPI2. If you want this UWP app running on PC, choose x86 or x64 with local device debugging. If you want this UWP app running on Windows 10 Mobile, choose ARM with Device debugging. If you want this UWP app running on Windows 10 IoT device, for example, Raspberry Pi 2, make sure you set the “Remote Debugging” point to your device.

Next, open MainPage.xaml.cs in Solution Explorer and locate to the “dataTransferTick”. Add your storage account name and access key that saved in section 4.

Fig.15: Add Account Information

After pairing the HC-06 with your Windows 10 device, press F5 from Visual Studio to launch the Universal Windows Platform app.

Fig.16: The Startup Screen of the App

Double tap the “HC-06” Symbol, you will find the Id and Name display on the very top of the application as follows.

Fig.17: The Bluetooth Module Information Shown on the App

Soon, you can see that the state of the LED on HC-06 will change from blinky to steady. And the “Start Rev” button is enabled as soon as the communication channel of Bluetooth module is established.

Fig.18. Start Recv Button

Click Start Recv to enable data receive. Then you will see the Stop Recv, Process Data and Upload Data button are enabled. Just click Process data button to run the fall detection algorithm. You will see the data on Recvd TextBlock and the state of the person who carry the device.

Fig.19: Result of the Fall Detection Algorithm

You still can find the data will be displayed on the debug windows of Visual Studio as follows.

Fig. 20: Debug information shown on Output Window of Visual Studio

Then, click the Upload button to upload the data to Azure. The data will be transmitted every 3 seconds. If you want to change the time interval, you can modify the code in “AzureButton_Click” as follows.

private void AzureButton_Click(object sender, RoutedEventArgs e) {

timerDataTransfer = ThreadPoolTimer.CreatePeriodicTimer(dataTransferTick, TimeSpan.FromMilliseconds(Convert.ToInt32(3000)));

}

Soon, you can see the data in AccelerometerTable as follows

Fig.21: Accelerometer Table

Thirdly, download the “FallDetectionClient” from github. Use Visual Studio 2015 with Update 1 to load this project. This application is built with Windows Universal Temple, so you can run it on Windows 10 PC/Mobile, as well as Windows 10 IoT Core device. Next, open StorageSensor.cs in Solution Explorer and locate to the “_accountName” and “_key”. Add your storage account name and access key that saved in section 4.

Now you should be able to press F5 from Visual Studio: The app will deploy and start, and you should see this on the device output.

Fig.22: Fall Detection Client UWP App

Code

Arduino Sketch (ADXL345_HC06.ino)

#include <Wire.h>  
#include <String.h>  

#define Register_ID 0
#define Register_2D 0x2D
#define Register_X0 0x32
#define Register_X1 0x33
#define Register_Y0 0x34
#define Register_Y1 0x35
#define Register_Z0 0x36
#define Register_Z1 0x37


int ADXAddress = 0xA7 >> 1; 
int reading = 0;
int val = 0;
int X0, X1, X_out;
int Y0, Y1, Y_out;
int Z1, Z0, Z_out;
double Xg, Yg, Zg;

void setup()
{
  Serial.begin(9600);
  Wire.begin(); 
  delay(100);
  Wire.beginTransmission(ADXAddress);
  Wire.write(Register_2D);
  Wire.write(8);
  Wire.endTransmission();
}


void loop() {
  double tempx = 0;
  char strx[5];
  double tempy = 0;
  char stry[5];
  double tempz = 0;
  char strz[5];
  
  String gy;

  Wire.beginTransmission(ADXAddress);
  Wire.write(Register_X0);
  Wire.write(Register_X1);
  Wire.endTransmission();
  Wire.requestFrom(ADXAddress, 2);
  if (Wire.available() <= 2);
  {
    X0 = Wire.read();
    X1 = Wire.read();
    X1 = X1 << 8;
    X_out = X0 + X1;
  }

  Wire.beginTransmission(ADXAddress);
  Wire.write(Register_Y0);
  Wire.write(Register_Y1);
  Wire.endTransmission();
  Wire.requestFrom(ADXAddress, 2);
  if (Wire.available() <= 2);
  {
    Y0 = Wire.read();
    Y1 = Wire.read();
    Y1 = Y1 << 8;
    Y_out = Y0 + Y1;
  }

  Wire.beginTransmission(ADXAddress);
  Wire.write(Register_Z0);
  Wire.write(Register_Z1);
  Wire.endTransmission();
  Wire.requestFrom(ADXAddress, 2);
  if (Wire.available() <= 2);
  {
    Z0 = Wire.read();
    Z1 = Wire.read();
    Z1 = Z1 << 8;
    Z_out = Z0 + Z1;
  }

  Xg = X_out / 256.00; 
  Yg = Y_out / 256.00;
  Zg = Z_out / 256.00;
 
  if(Xg>0)
  {
    gy = String("")+ "x=+"+Xg;
  }
  else
  {
    gy = String("")+ "x="+Xg;
  }
  
  if(Yg>0)
  {
    gy += String("")+ "y=+"+Yg;
  }
  else
  {
    gy += String("")+ "y="+Yg;
  }
  
  if(Zg>0)
  {
    gy += String("")+ "z=+"+Zg+",";
  }
  else
  {
    gy += String("")+ "z="+Zg+",";
  } 
  
  Serial.print(gy);

}

Credits

Jiong Shi

13 projects • 85 followers

Microsoft Azure MVP

Comments

Awards

Best Project - Rest of the World

World’s Largest Arduino Maker Challenge

A fall detection system based on Arduino, Windows and Azure