Pull-up Sensor and Counter - Arduino Nano 33 BLE - tinyML

A pull-up is an upper-body strength The pull-up is a closed-chainmovement where the body is suspended by the hands and pulls up. As this happens, the elbows flexand the shoulders adductand extendto bring the elbows to the torso.

This exercise is not easy, therefore many do it wrong. Many times people use too much momentum and body sway to pull themselves up, or they do not use full range of motion - they don't go with chin above the bar and don't fully extend at the bottom. Using incorrect form leads to minimizing the effectivness of the exercise and may lead to injury.

We created an app, that tells you if you are doing the exercise properly with correct form or wrong. The app also counts the number of correct pull-ups made.

Using the sensor

User uses either phone or portable phone charger to power the Arudino Nano 33 BLE board and puts it in the pocket. The application is then opened on computer and connected to the Arduino board via bluetooth. The user goes and performs the exercise. Using board's sensors - the accelerometer, the board then detects when user is doing the pull-ups and if they are performed correctly or not. The number of correct pull-ups is also counted, the application also shows if user is performing pull-ups correctly. The board itselfs changes LED color from green (correct form) to red (incorrect form).

Machine learning model

Using EdgeImpulse we created a model, in which we recorded two main movements:

• Correct pull-up
• Wrong pull-up

Beside those two we recorded more secondary movements like staying still and walking to improve our model to not confuse detection of pull-ups. Machine learning model was exported as Arduino library since we then changed and added to the code.

All movements were recorded with device in the pocket, so the sensor works best when also put in the pocket. It can be attached for example to the belt too, but the accuracy of detection is a bit lowered this way. The model needs to be imported to arduino as a library to work.

Counter

To detect the number of pushups we needed to modify and add to the exported Edge Impulse code. Two additional libraries were used:

• Scheduler (enables us to run multiple functions at the same time)
• Peak detection (enables us to detect peaks in real time)

First we set the edge impusle code to run every 2 seconds, so this means predections for which moevement was performed happens every 2 seconds. Less than that and detection of movement was too inaccurate, more than that would make predictions much more accurate but we risk pull-ups not being detected as pull-ups and thus making counter less accurate. (For example if we set it to 5 seconds and person pefromed 7 seconds of pull-ups, the first 5 seconds would be detected as pull-ups but the second 5 seconds would not since there was 3 seconds of not pull-ups and only 2 seconds of pull-ups).

We get result classification value and classify movement simply by finding the result classification with highest value.

This is all performed in loop() function. Counter is performed in loop2() function, which runs every 100 miliseconds. First we start the peak detection function and tell Scheduler to run the loop2() in the setup():

Scheduler.startLoop(loop2);

peakDetection.begin(10, 1.82, 0.9);

The 3 parametrs in peak detection function represent lag, threshold and influence (more information on how algorithm works linked here). The exact numbers were achieved by trial and error - we found these numbers work best for our intentions.

Inside loop2() function we first calculate the congregated number of all axis of accelerometer data which represents movement (acc variable):

float x, y, z;    
IMU.readAcceleration(x, y, z);
acc = sqrt(x*x + y*y + z*z);

When acc variable remains at 1 it means there is no movement, when it rises below or above it means the movement was detected. We use acc readings and input them as data into peak detection algorithm:

double data = (double)acc;
peakDetection.add(data);
peak = peakDetection.getPeak();
double filtered = peakDetection.getFilt();

The peak variable just outputs -1 and 1 when it detects peaks.

We then just make sure to not count multiple values (multiple 1's for example) when certain peak is detected and according to this we increment our movement counter.

In the main loop() function after the prediction of movement for last two seconds is made we also check the movement counter in last two seconds and if the detected movement is correct pull-ups we increment that value to the correct pull-ups counter.

When certain movement is peformed we just turn on the LED light with red and green colors for inccorect and correct pull-ups respectively:

if (tipVaje == "pravilenVzgib") {
        if (merjenje) test2 = test2 + test1;
        digitalWrite(BLUE, HIGH);
        digitalWrite(RED, HIGH);
        digitalWrite(GREEN, LOW);
      }

    if (tipVaje == "napacenVzgib") {
        if (merjenje) test3 = test3 + test1;
        digitalWrite(BLUE, HIGH);
        digitalWrite(RED, LOW);
        digitalWrite(GREEN, HIGH); 
      }

Application

We created a web application, where we show the correct pull-up counter and on the bottom in the black bar we write the current movement being peformed. The app is composed of two main buttons, start and stop. We also used bright and visible colors to tell the user, if the exercise is correct or wrong as seen on the image above.

To connect our sensor (Arduino Board) we use BLE technology. The application auto detects the board, simplifying it's usage. But there is an option to connect it manually. Experimental web platform features need to be turned on first, to enable the browser to connect via bluetooth. Example for chrome:

chrome://flags/#enable-experimental-web-platform-features

Bluetooth

The arduino model detects the movement. If the movement is the one we are searching for we send the value via bluetooth.

if ((tipVaje == "pravilenVzgib") || (prejsnjiTipVaje == "pravilenVzgib")){
           steviloPravilnih +=1;
          digitalWrite(BLUE, HIGH);
          digitalWrite(RED, HIGH);
          digitalWrite(GREEN, LOW);
          if (central.connected()) vzgib.writeValue(1);
}
if ((tipVaje == "napacenVzgib")){
          steviloNepravilnih +=1;
          if (central.connected()) vzgib.writeValue(2);
          digitalWrite(BLUE, HIGH);
          digitalWrite(RED, LOW);
          digitalWrite(GREEN, HIGH); 
}

Using JavaScript we search for bluetooth devices that matches the board's name. Finding our sensor we connect to it. First we connect to the GATT server, then we use the sensor UUID to connect to its services and its characteristics to access the values:

let serviceUuid = "e267751a-ae76-11eb-8529-0242ac130003";
  if (serviceUuid.startsWith('0x')) {
    serviceUuid = parseInt(serviceUuid);
  }

We also have a reset button (which is avialable on application after user has finished measuring) which goes in reverse direction and sets the variable in Arduino's code to zero:

function reset() {
  pravilni = 0;
  nePravilni = 0;
  var arr = new Int8Array([21,31]);
  return resetValue.writeValueWithResponse(arr).then(response => {
    document.getElementById("stevecPravilni").innerHTML = pravilni;
  });
  
}

The type of pull-ups detected is sent via integer value and then displayed:

if (value < 0) {
    console.log("NAPACEN: " + value)
    nePravilni = Math.abs(value + 1);
    if (prejsnjiZgresen < nePravilni) {
      document.getElementById("vzgib").innerHTML = "Zgrešen"
      document.getElementById("barvaP").style.display = "none"
      document.getElementById("barvaZ").style.display = "block"
    }
    
  }

  else {
    pravilni = value;
    console.log("PRAVILEN: " + value)
    if (prejsniPRavilen < pravilni) {
      document.getElementById("vzgib").innerHTML = "Pravilen"
      document.getElementById("barvaP").style.display = "block"
      document.getElementById("barvaZ").style.display = "none"
    }
    
  }

Working example - Video