The Protector - a life saving wearable

Story

Many of us like to go for a walk, whether it is to do some exercise, to shop at the market, or for a simple walk in the park, whatever our case may be, we are exposed to the weather conditions of the moment, if we walk in a prairie we probably don't have to worry about the quality of the air we breathe, but this is not the case in town, stores or even our office.

To take care of ourselves in these spaces that can have an impact on our health, I have created "The Protector", a device that is always with us, going almost unnoticed, but keeping a constant record of certain parameters that will allow us to know if we are in a safe place or If we are fine, these include temperature, humidity, atmospheric pressure, air quality, CO2 concentration, and even what we are doing at the moment, if we are standing still, walking, running, in a car, bicycle and last but not least, detecting if we have suffered a fall.

The project is accompanied by a custom-designed mobile application in which we will be able to see all the mentioned variables in real time, and also see the records of important moments such as exposure to highly polluted air, having been in very high places or the fact that we fell.

All this will be recorded with the exact time, the conditions and the place where it occurred.

The Hardware

How does it work?

Using the integrated motion sensor unit (BHI260AP) in the Nicla Sense ME, the system identifies automatically known activities as:

• Standing still.
• Walking.
• Running.
• Riding a bicycle.
• Being in a car.

For custom activity detection as falling down, a motion recognition model on Edge Impulse was created for this specific case.

With the integrated pressure sensor (BMP390) the system is able to measure absolute atmospheric pressure and so, high altitude conditions.

Using the BME688 sensor, it can measure:

• Temperature.
• Humidity.
• Index of Air Quality.
• CO2 concentration.

All this data is sent to a smartphone through BLE and is shown on a dashboard, and if a fall detection or a low pressure condition is sent, the app will record the situation including other variables status and the exact location to be then reviewed. (I am trying to send an SMS or email after a fall is detected).

Training the Edge Impulse model

Remember that this model is trained mainly to detect falls.

I developed a method to trained my model with accelerometer data (x, y, z) from the Nicla Sense ME wirelessly, taking advantage of it’s BLE capability, for this I created a code that sends the data to an ESP32 board connected to the PC and using the Edge Impulse data forwarder, upload the samples directly to my model training dataset.

This was very important because that way I could record my movements without any cable connected to the Nicla board that could ruined my readings, and also allow me to record hard and sudden movements as falls.

I recorded 10 seconds samples saying several times the keywords and then split them into 1 second windows.

My dataset consists on +4 minutes of each event:

• Walking.
• Standing still.
• Falling.

This project is public on Edge Impulse so you can clone it, modify it and test it!

Deploy the model back to the microcontroller

• In the Deployment section, create an Arduino library, enabling the EON compiler (int8):

• Unzip the downloaded library.

💡 Note: The Nicla Sense ME has limited (64 kB) SRAM, this will cause the board to crash on due to the size of stack reserved memory by the ArduinoBLE and ArduinoBHY2 libraries (on Arduino IDE), after a week on trying to fix this problem, I suddenly thought in compiling the code with PlatformIO and this solved the problem (apparently PlatformIO compilation is more memory friendly).

• Set the PlatformIO environment to program Nicla Sense ME board.
• Open the PlatformIO project attached.
• Copy the model library to the /lib folder of the project:

• Build the code and flash the Arduino board.
• Let the board to be fully charged.

Mobile app download and setup

• Go to Play Store and download “The Protector” app.

• Open it once downloaded.
• As you can see, the dashboard and graphs are empty, to fill them with some data, let’s pair the project with the app.
• In the right upper corner, touch on the BT icon, and select the device called “The Protector”.
• After few seconds, data will show up.
• Hook the Nicla Sense ME to your K-Way jacket.
• We are almost ready to go.

Understanding the APP functions

Sensor tab:

• Let us pair the app with the hardware.
• Here we can see a live stream of the environment variables being measured by the Nicla Sense ME onboard sensors.
• Also, we can enable the graphs of each variable to compare them in time.

History tab:

• Here we can see our real time location using the smartphone GPS.
• We can see the Alert history.
• If we select an Alert event, it will show the location where it occurred and the sensor data recorded in that moment.

Settings tab:

• Here you can enable the notifications of harmful events like a bad IAQ, low pressure because high altitude or after falling.
• You can enable the LED of the Nicla Sense ME so it can give you a feedback of the air quality with a very simple color code.

🟩Green - Good IAQ

🟨Yellow - Slightly polluted air

🟥Red - Highly polluted air.

• We can delete the history logs of alerts.

Real use case and demonstration

In a normal situation, The Protector will log the temperature, humidity, pressure, IAQ, CO2 levels and activity.

If you are exposed to a polluted place, the app will notify you and will record the exact moment and location data.

If you suddenly fall down, the app will send a message to your emergency contact through WhatsApp with your location and will store the event.

Video demo

Here I show you the project working on video: