Developing a "personal trainer" using TinyML

Introduction

According to the Minister of Health of Brazil, in 2020, 36.8% of the country population used to practice some sort of physical activity on their free time.

Seeking a way to improve the feedback a person can receive while doing exercises, a system was developed by integrating TinymL with a small development board that can be attached to a dumbbell.

The development board which will be used in this project is the Seeed XIAO nRF52840 BLE Sense for its compact size, lower power consumption, ability to recharge the battery and built-in accelerometer and gyroscope. Sensors essential for this project.

3D Casing

To be able to bring the whole system inside a gym or in a physical activity environment, it is necessary to create a case to protect the development board from accidental drops while being able to carry along a battery, thus making it possible to operate without using cables.

A simple yet useful casing can be made by using 3D printers.

Data Acquisition

One of the main problems of this project is being able to collect reliable data, since collecting data involves attaching the whole system to the side of a dumbbel that in moving all the time, either up and down, side to side or both all together.

One way to solve this problem is by using bluetooth to transmit the data from the inertial measurement unit (IMU) to a smartphone without the need for cables.

On this project the main focus relies on three exercises, the biceps curl, the bench press and the lateral raise, all of them done using dumbbells. This Choice was based on these exercises working different types of muscles: the biceps, the chest area, and the shoulders respectively.

To receive the data from the microcontroller, it is necessary the use of an android app, for this purpose, an app was developed on the MIT App Inventor 2 website.

The app function is to receive the data, for the necessary amount of time, from the 6 axis of the IMU and store it in either a.csv file or an online google sheets.

Model training

To train this model, we used the Edge Impulse website. First, all the raw data collected was uploaded and pre-processed inside the platform. To generate our features we'll be using a FFT of 256 points length.

The model design will be a Dense Neural Network (DNN) that will 570 neurons on its input layer, two hidden layers with 40 and 20 neurons, a dropout layer of 0.25 ratio, another 10 neurons layer and output layer of 5 neurons, one per each class.

Model testing

By using 20% of the data collected during the data acquisition phase, we can verify how our model will behave with unknown data.

Even though 85% accuracy is not ideal, it is still a good result considering the complexity of the data being analised.

Deploy

After training is complete, we are able to deploy the model in our development board of choice, the Seeed XIAO nRF52840 BLE Sense, an easy way to do that is by using the compiled library provided by the Edge Impulse, after that, we will have the whole system complete ready for deployment.

But, we still need a way to visualize the inferences results, to do that, the same approach made in the data acquisiton phase can be done.

Making inferences

By developing a new app to display inferences, a number of options became available, the inferences results can be displayed on the smartphone screen, by audio and also by the smartphone vibration. The data can also be saved to create a timeline of the whole exercise section and a data analysis can be performed to create graphs, allowing the end user to interact and understand where and when his performance was good or bad.

Conclusion

With the help of the great and realiable board used in this project, we could achieve significant results with our tiny model.

As seen in the model training section, the final model only occupies 1.9K of RAM, which leaves us with the possibility of expanding and improving this project further by capturing more data or data from new and different exercises.

On the project Github repository, you will find all the codes used, as well as the model library and the 3D casing stl files.

Knowing more!

This project is also on Youtube: SciTinyML-23 - Day4 - UNIFEI Personal Trainer

If you want to know more about collecting data through bluetooth, check out the amazing tutorial made by Marcelo Rovai: Sensor DataLogger.

To understand and get to know the development board used in this project, check out this other tutorial made by Marcelo Rovai: TinyML Made Easy: Anomaly Detection & Motion Classification.

Thank you!

We want to express our gratitude to CNPq for sponsoring this project, Seeed Studio for providing us the development boards used here and Cleyton Nogueira, the personal trainer from 4FIT gym.

Also thank you for reading this far, we hope this project helped you somehow or gave you a new idea for the future!