.
Overview.
To fight obesity, especially during this Covid-19 period where people will stay indoor most of the time, an activity intensity tracker would be beneficial for those who want to keep their body fit.
I am going to build a device connecting to the sensors, which will read the sensors' data and compute the activity intensity level (high, low, medium) and send this data via bluetooth UART connection to a mobile device displaying a real time time series logs at nRF toolbox mobile app.
With sensors connected, nRF5340 DK app will read, compute and send the data to a host device (mobile phone running nRF Toolbox) for real time display. At the same time, power consumption will be profiled by the Power Profiler Kit II.
.
How it's built.
First I made a nRF5340 DK app to read the ADXL335 sensor data (x, y, z) and use the Edge Impulse's Data Forward to do the accelerometer data capturing. With the Edge Impulse Studio and data collected, I trained a model and built a C++ library. The model can do motion classifications of low, medium and high
movement.
Based on the Edge Impulse firmware for nRF52840 DK / nRF5340 DK GitHub project, I added the ADXL335 sensor code and the model library from Edge Impulse Studio. The app can stream classification results using the Nordic BLE UART Service. The streaming classification results can be displayed on the terminal connecting via USB or on mobile device using the nRF Toolbox app.
Here is a picture of the final setup:
.
.
Setup nRF5340 DK and its app development environment.
I followed the nRF Connect SDK tutorial to setup the nRF5340 DK and its development environmont.
.
Data Collection.
nRF5340 DK wrote sensor values over a serial connection, and the Data Forwarder collected the data (x, y, z), signed the data and sent the data to the ingestion service of Edge Impulse.
Refer to the code (prj.conf (data collection) and main.c (data collection))
attachment section for the code that I used to accomplish this.
.
// add photo
.
Build modelThe next few pictures how to build the modle using Edge Impulse Studio.
.
Create impulse
.
Generate features
.
Config NN Classifier
.
Live classification
.
DeploymentAt Edge Impulse's Deployment panel, build the C++ library
:
.
Build motion classifiction streaming app.
There is an Edge Impulse example firmware already available for the nRF5340 DK with the Nordic BLE UART capability. I extended it by adding the sensor reading feature and streaming motion classification output.
Specifically, two sensor related files (ei_adcsensor.h and ei_adcsensor.cpp
) and the classification output streaming code (ei_run_impulse.cpp
) were add to the above example firmware. For the code, please refer the code attachment of this project.
Connecting with USB port, you can type AT+HELP
to see a list of commands. To run impulse classification output streaming, type AT+RUNIMPULSE
.
.
.
Display with nRF Toolbox and nRF Connect.
Below are the screen captures of the nRF Toolbox's UART interface and nRF logs. The PLAY button will invoke the AT+RUNIMPULSE
command, STOP button sends te 'b
' command, INFO button sends the AT+HELP
command, 1 button sends the AT+SENSORS?
command and 2 button sends the AT+DEVICEINFO?
command. Please refer to the Edge Impulse example firmware for the meaning of these commands.
When the AT+RUNIMPULSE
is sent, the app will stream the motion classification out to the terminal and nRF Connect logs, if those are connected to the nRF5340 DK.
.
..
Here is a video about the project. Using the nRF Toolbox's UART to connect to the app and press the PLAY button to run the Impulse classification routine. Then connect the nRF5340 DK with the nRF Connect to show the streaming of motion classification ouput.
.
.
Power Profiling with PPK 2.
Below are the screen captures of power measurements while the power profiling that carried out using Power Profilinig Kit 2.
.
.
Summary.
This is a simple prototype. There are lots of improvements to be made. For example, more and better data collection in order to build a better intelligent model for the classification. A 3-D casing can be made to house the board and sensor so the it can be carried around to capture the sensor's data. Of course, the use of solar powered battery to enhance the mobility of the equipment.
.
Currently there are apps that target 10K steps per day as a benchmark of healthy activity lifestyle. With this activity tracking app, we can count the number of 'high' labeled activity as an alternative benchmark.
.
References.
Using Edge Impulse with nRF Connect SDK
Edge Impulse: Continuous motion recognition
Edge Impulse: Nordic Semi nRF5340 DK
Streaming Edge Impulse Classification Results Using the Nordic BLE UART Service
Testing with a mobile device (nRF53 Peripheral UART example)
.
Comments