Published September 7, 2019 © MIT

Brainium Rehab: Rehabilitation Trainer with Robotic Arm

Rehabilitation system based on SmartEdge Agile which detects rehabilitation movements and gives feedback to the patient.

AdvancedFull instructions provided15 hours2,029

Best Use of AI at the Edge: 1st Place

Bring Intelligence to the Real World

Things used in this project

Hardware components

Avnet SmartEdge Agile Brainium

Robotic arm

Adafruit feather huzzah 32

Standard core wire

8 Channel DC 5V Relay Module with Optocoupler x1.

Software apps and online services

Arduino IDE

Python anaconda

CloudMqtt

Hand tools and fabrication machines

Soldering iron (generic)

Story

Introduction:

There are people in the world that suffer from serious impediments in their arms. These happen because of several illnesses and acute symptoms such as stroke, Guillain-Barré syndrome, paralysis from birth, cerebral palsy, spina bifida, spinal muscular atrophy and several others. These patients have to continuously take rehabilitation therapies, which are expensive:

The prices of these sessions go from $ 650 to $ 1300 USD and sometimes, they are not insurable: https://www.healthline.com/health/cool-sculpting-cost

They take a lot of time, and it is also necessary to go to specific clinics for the service. Which increases the pain and discomfort of the patient.

The clinical devices that perform the rehabilitation therapy are enormous and can not be transported or mobilized easily: https://www.hocoma.com/solutions/armeo-power/ https://www.researchgate.net/figure/Examples-of-robotic-devices-for-motor-training-A-End-effector-type-InMotion-20_fig1_259609214

Solution

Therefore, we decided to create a device capable of helping the patient in his rehabilitation and give him continuous feedback on his therapy. It also has to be economical (to a certain degree and in comparison to the other choices).

System Architecture:

This is the connection diagram of the system:

For this to work we will be setting up the SmartEdge Agile device via MQTT to the Brainium portal, and via its motion sensor train the four movements that we will base our rehab in. From there we will then communicate via MQTT to a computer (could even be a Raspberry pi). In this case a laptop running Python via the paho-mqtt library. From there also via MQTT we will communicate to an ESP32 in order to give the commands required to move the Robotic arm a certain way, based on rehabilitation movements.

Brainium Setup:

Model Training:

Before performing any other task, it is vital to be able to generate a model for elbow rehabilitation. The system can be extended to any rehabilitation therapy but we chose to work with elbow rehab as the first sample.

The four basic movements were programmed for arm rehabilitation (elbow), of which three of them will be used in the final rehabilitation.

Elbow flexo-extension:

Arm Lift (Lateral raise):

Elbow Flexion:

This is the model developed and the number of repetitions for each movement, notice the number of repetitions needed vs the maturity of the model:

And this is the model motion confusion matrix:

Now that we know what we will be working with, lets set up the device and widgets.

Press on create the widget.

Assign the model to the device.

Create a widget as shown in the image.

Finish the widget.

We will get a result like so.

Obtaining Credentials.

Save External access token and User ID.

CloudMQTT Setup:

Create an account in Cloud MQTT.

https://www.cloudmqtt.com/

Copy the credentials for "Server", "User", "Password" and "Port".

Laptop Setup:

We will be using a laptop for this. Install Python Anaconda so that you can easily manipulate the MQTT broker, this had to be done because the Arduino library ESP32 for MQTT does not accept connectivity such as Websocket.

https://www.anaconda.com/distribution/

Then install the paho-mqtt library with the following command:

pip install paho-mqtt

Download the file "Python Files", open "MQTTRehab.py" and input your credentials.

Arm Setup:

Follow the diagram without making a mistake,

WARNING: IF YOU DON'T CONNECT EVERYTHING PROPERLY, YOU CAN MAKE IT SHORT CIRCUIT.

A note on the arm used: This robotic arm model just closes certain circuits to set the operating voltage positive or negative depending on how the circuit is closed. The control just uses a couple switches to achieve this. With the Relay module we can accomplish the same funcion of the controller by closing those circuits, with the added bonus of making it wireless. This diagram is only intended for a Robotic arm that uses DC motors such as the one presented, if your Robotic arm uses servos or other kind of motors the electrical design and microcontroller setup must be different.

After connecting everything, program the ESP32 with the code you can find below inside the "Arduino Files" folder.

Within the "ESP32ARMRehab.ino" file enter the CloudMQTT credentials.

const char* ssid = "YOURSSID";  const char* password =  "YOURPASS";  const char* mqttServer = "m12.cloudmqtt.com";  const int mqttPort = 12345;  const char* mqttUser = "YOURCLOUDMQTTUSER";  const char* mqttPassword = "YOURCLOUDMQTTPASS";

The Final Product:

Here are some pictures of the final product and the build process:

1 / 3 • Robotic arm with module and weight.

1 / 4 • Relay module connected to the ESP

And now time to test everything and produce a DEMO!

Business Opportunity

The global rehabilitation equipment market is set to grow and keep growing at an alarming rate. The reasons for this are the increasing number of disabilities that are derived from non-communicable diseases, favorable reforms in the healthcare sector and an increasing geriatric population coupled with increasing incidences of chronic diseases. In addition, increasing support from the government and technological innovations in the field of rehabilitation equipment which supports the geriatric population and the physically challenged people is also giving an impetus to this market. As previously stated, the high cost of the rehabilitation equipment, rehabilitation therapies along with a high maintenance cost of such devices alongside an increasing cost of general healthcare services (at least in the US) makes it an enduring problem.*1

According to Persistence Market Research, the global rehabilitation equipment market is forecasted to reach a figure of about US$ 13.4 Billion in 2022 and is poised to exhibit a robust CAGR in the period of assessment.

Reiterating on the cost of the therapies and the devices alongside the reduced number or lack of specialists (in relation to the number of patients) there is quite a market for these kind of solutions. Another problem that has to be taken into consideration, is the fact that nowadays and going forward payment will be linked to patient satisfaction and greately related to outcomes. So, patient experience is much more important as a whole. For this project I had the opportunity of actually performing the demonstration to medical specialists and rehabilitation professionals. They immediately found the proposition valuable and at the same time proposed a niche market for the product which is to target it to children. Children are sometimes very difficult to work with, but something such as a Robotic arm has the potential to improve on their attention and adherence to the routine and furthermore on their outcome.

On SDG's

1 / 2 • SDGs focused on

Healthcare is changing, and it has to. Alone the US is reaching almost 20% on its GDP which is unsustainable and the trend that relates payment to patient satisfaction and outcomes will be much more prevalent. Our solution in this case is a combination of two SDGs, the #3 SDG which is to ensure good health and well being for all ages and the #9 which includes to foster innovation. Let's be fair here, even if we advance on health and well being to the limit proposed by the SDG, noncommunicable diseases will always be part of modern life and a sometimes forgoten sector involves orthopaedical-related injuries and rehabilitation. They are both what fills hospitals, and will continue to be for the forseeable future. This solution offers to foster innovation while closing the gap between patient satisfaction and adherence to a treatment for a better outcome. There is seldom a solution like this one for patient satisfaction, and yes a hospital and/or clinic is nowadays filled with technology, but more so related only to critical-care devices and top of the line imaging equipment. This solution focuses much more on the patient as a whole.

Commentary and the future

I think the project came up nicely, but I still want to perform a couple changes. One is regarding the use of a whole laptop to run a MQTT node and Python script. The issue was that Brainium's MQTT was unable to get to the MCU (in this case an ESP32) so a laptop was used as a gateway. This was the easier way to perform this action and follow on the architecture proposed, but as usual there are many alternatives. For example we may be able to do the same thing using a virtual machine hosted on a cloud service such as IBM cloud, AWS or Azure. The other alternative is to use an embedded computer such as a Raspberry Pi in order to create the gateway and make it more of an Edge device. The latter is probably the solution I would go with, as it implies a much lower price and ease of access.

The last thing I would like to intergrate in the project is audio cues. This could be done with any STT platform such as snips or even Amazon Alexa so that the patient is receiving instructions during the therapy.

Thank you for reading, and I hope you liked the project.

References

1.- https://www.persistencemarketresearch.com/market-research/rehabilitation-equipment-market.asp

Code

#define Vcc 13
#define Vee 12
#define Claw 27
#define Wrist 33 
#define Elbow 15
#define Shoulder 32
#define Base 14
#define Ddelay 11

#include <WiFi.h>
#include <PubSubClient.h>
 
const char* ssid = "YOURSSID";
const char* password =  "YOURPASS";
const char* mqttServer = "m12.cloudmqtt.com";
const int mqttPort = 12345;
const char* mqttUser = "YOURCLOUDMQTTUSER";
const char* mqttPassword = "YOURCLOUDMQTTPASS";
bool sw = false;
int counter = 0;
int mov=0;
 
WiFiClient espClient;
PubSubClient client(espClient);
 
void setup() {
  pinMode(Vcc,OUTPUT);  
  pinMode(Vee,OUTPUT); 
  pinMode(Claw,OUTPUT);
  pinMode(Wrist,OUTPUT);
  pinMode(Elbow,OUTPUT);
  pinMode(Shoulder,OUTPUT);
  pinMode(Base,OUTPUT);
  // Turning Off Everything
  digitalWrite(Vcc,LOW); 
  digitalWrite(Vee,HIGH); 
  digitalWrite(Claw,HIGH);
  digitalWrite(Wrist,HIGH);
  digitalWrite(Elbow,HIGH);
  digitalWrite(Shoulder,HIGH);
  digitalWrite(Base,HIGH);
 
  Serial.begin(115200);
 
  WiFi.begin(ssid, password);
 
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.println("Connecting to WiFi..");
  }
  Serial.println("Connected to the WiFi network");
 
  client.setServer(mqttServer, mqttPort);
  client.setCallback(callback);
 
  while (!client.connected()) {
    Serial.println("Connecting to MQTT...");
 
    if (client.connect("ANYNAME", mqttUser, mqttPassword )) {
 
      Serial.println("connected");  
 
    } else {
 
      Serial.print("failed with state ");
      Serial.print(client.state());
      delay(2000);
 
    }
  }
 
  client.publish("/ESPTopic", "Hello from ESP32"); 
  client.subscribe("/ArmTopic"); 
  delay(5000);
  digitalWrite(Vcc,HIGH);
  delay(5000);
  Serial.print("Move Arm Start");
  FmoveP();
  Fmove();
  Fmove();
  Fmove();
  Serial.print("Move Arm Stop");

}
 
void callback(char* topic, byte* payload, unsigned int length) {
 
  String datas="";
  for (int i = 0; i < length; i++) 
  {
    datas+=char(payload[i]);
  }
  Serial.println(datas);
  if(datas=="Sit Flex" && mov==0 && counter<10)
  {
    counter++;
  }
  else if(datas=="Sit Flex" && mov==0 && counter==10)
  {
    mov++;
    counter=0;
    FmoveR();
    SmoveP();
    Smove();
    Smove();
    Smove();
  }
  else if(datas=="Sit Up" && mov==1 && counter<10)
  {
    counter++;
  }
  else if(datas=="Sit Up" && mov==1 && counter==10)
  {
    mov++;
    counter=0;
    SmoveR();
    TmoveP();
    Tmove();
    Tmove();
    Tmove();
  }
   else if(datas=="Sit Inside" && mov==2 && counter<10)
  {
    counter++;
  }
  else if(datas=="Sit Inside" && mov==2 && counter==10)
  {
    mov++;
    counter=0;
    TmoveR();
  }
  
}

void loop() {

  if (!client.connected()) {
    reconnect();
  }
  client.loop();
}

void up(int pin,int Delay)
{
  digitalWrite(Claw,HIGH);
  digitalWrite(Wrist,HIGH);
  digitalWrite(Elbow,HIGH);
  digitalWrite(Shoulder,HIGH);
  digitalWrite(Base,HIGH);
  digitalWrite(pin,LOW);
  digitalWrite(Vcc,LOW);  // +Vcc ON
  delay(Delay);
  digitalWrite(Vcc,HIGH);
  digitalWrite(pin,HIGH);
}

void down(int pin, int Delay)
{
  digitalWrite(Claw,LOW);
  digitalWrite(Wrist,LOW);
  digitalWrite(Elbow,LOW);
  digitalWrite(Shoulder,LOW);
  digitalWrite(Base,LOW);
  digitalWrite(pin,HIGH);
  digitalWrite(Vee,LOW); // -Vee ON
  delay(Delay);
  digitalWrite(Vee,HIGH);
  digitalWrite(Claw,HIGH);
  digitalWrite(Wrist,HIGH);
  digitalWrite(Elbow,HIGH);
  digitalWrite(Shoulder,HIGH);
  digitalWrite(Base,HIGH);
  
}


void FmoveP(void)
{
  down(Shoulder,5000);
  down(Elbow,5500);
}
void Fmove(void)
{
  up(Wrist,3500);
  down(Wrist,3500);
}
void FmoveR(void)
{
  up(Shoulder,5500);
  up(Elbow,6000);
}

void SmoveP(void)
{
  up(Base,6000);
  down(Shoulder,5000);
  down(Elbow,5300);
}
void Smove(void)
{
  up(Elbow,6500);
  down(Elbow,5300);
}
void SmoveR(void)
{
  up(Elbow,6000);
  up(Shoulder,5500);
  down(Base,6000);
  
}

void TmoveP(void)
{
  up(Base,6300);
  down(Shoulder,5000);
  up(Elbow,3000);
  up(Wrist,3500); 
}
void Tmove(void)
{
  up(Elbow,4000);
  down(Elbow,4000);
  up(Elbow,4000);
  down(Elbow,4000);
}
void TmoveR(void)
{
  down(Elbow,3500);
  down(Wrist,3500);
  up(Shoulder,5700);
  down(Base,6300);
}


void reconnect() {
 while (!client.connected()) {
    Serial.println("Connecting to MQTT...");
 
    if (client.connect("ESP32Client2", mqttUser, mqttPassword )) {
 
      Serial.println("connected");  
 
    } else {
 
      Serial.print("failed with state ");
      Serial.print(client.state());
      delay(2000);
 
    }
  }
  client.publish("/ESPTopic", "Hello from ESP32"); 
  client.subscribe("/ArmTopic"); 
}

import uuid
import json
import paho.mqtt.client as mqtt
import time

time.sleep(2)

# Brain MQTT CREDENTIALS

mqtt_user_name = 'oauth2-user'
mqtt_password = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'  # copy and paste here external client id from your account
user_id = 'USERNUMBER'  # copy and paste here your user id
device_id = 'XXXXXXXXXXXXXXXXXXXX'  # copy and paste here your device id
ind=""

alerts_topic = '/v1/users/{user_id}/in/alerts'.format(user_id=user_id)
acc_norm_datasource_topic = '/v1/users/{user_id}/in/devices/{device_id}/datasources/MOTION'.format(
    user_id=user_id,
    device_id=device_id)

ca_cert_path = 'cacert.crt'


def on_connect(client, userdata, flags, rc):
    print('Client 1 Connected with result code {code}'.format(code=rc))

def on_disconnect(client, userdata, rc):
    if rc != 0:
        print ("Unexpected MQTT disconnection. Will auto-reconnect")

def on_message(client, userdata, msg):
    global ind
    global mqttc
    ind= json.loads(str(msg.payload.decode("utf-8")))
    print(ind[0]['name'])
    # CLOUD MQTT CREDENTIALS
    client2 = mqtt.Client()
    client2.username_pw_set("USER", "PASSWORD")
    client2.connect("m12.cloudmqtt.com", 12345)
    client2.publish("/ArmTopic",ind[0]['name'])

def main():    
    client1 = mqtt.Client(client_id="7624876298468192648726348726938", transport='websockets')
    client_iad1=str(uuid.uuid4())
    print(client_iad1)
    client1.on_connect = on_connect
    client1.on_message = on_message
    client1.on_disconnect = on_disconnect
    client1.tls_set(ca_certs=ca_cert_path)
    client1.username_pw_set(mqtt_user_name, mqtt_password)
    client1.connect('ns01-wss.brainium.com', 443)
    client1.subscribe(acc_norm_datasource_topic)
    client1.loop_forever()

if __name__ == "__main__":
    main()

Credits

EdOliver

38 projects • 74 followers

Engineer, Scientist, Maker. Entrepreneur and Futurist.

Thanks to Victor Altamirano.

Comments

Awards

Best Use of AI at the Edge: 1st Place

Bring Intelligence to the Real World

Brainium Rehab: Rehabilitation Trainer with Robotic Arm

Things used in this project

Hardware components

Software apps and online services

Hand tools and fabrication machines

Story

Introduction:

System Architecture:

Brainium Setup:

Widget Configuration:

Obtaining Credentials.

CloudMQTT Setup:

Laptop Setup:

Arm Setup:

The Final Product:

Business Opportunity

On SDG's

Commentary and the future

References

Schematics

System Architecture

Connection diagram for the Robotic Arm

Code

certificate

Brainium rehab for ESP32

MQTTRehab.py

BriniumRehab Github

Credits

EdOliver

Comments

Awards

Embed the widget on your own site

Brainium Rehab: Rehabilitation Trainer with Robotic Arm

Brainium Rehab: Rehabilitation Trainer with Robotic Arm

Things used in this project

Hardware components

Software apps and online services

Hand tools and fabrication machines

Story

Introduction:

System Architecture:

Brainium Setup:

Widget Configuration:

Obtaining Credentials.

CloudMQTT Setup:

Laptop Setup:

Arm Setup:

The Final Product:

Business Opportunity

On SDG's

Commentary and the future

References

Schematics

System Architecture

Connection diagram for the Robotic Arm

Code

certificate

Brainium rehab for ESP32

MQTTRehab.py

BriniumRehab Github

Credits

EdOliver

Comments

Awards

Related channels and tags