Nero: A Smart Glove That Controls Remote Devices

It has become a critical and laborious task to explore all the possible marvels of numerous industries like the healthcare and medication industry, where surgeries involve precision and opinions from experts around the world, in places like chemical manufacturing plants, packaging and production industry, and all areas of commerce with harsh chemical environments. This project is just the solution to all the stated problems. Nero is a leap forward in making all the tasks possible that cannot be completed just with human intervention and skills.

This gesture-controlled smart robotic glove enables us to achieve an economical solution using pre-existing technology. It can be implemented to control and automate machines like robotic arms, conveyor belts, pullies, cranes, and innumerable other devices.

Your imagination is the limit to the efficiency of Nero. This project provides us with remarkable precision, accuracy, and gives liberty to further accessibility and automation.

Nero In Action!

Overview

In this project we basically have 2 major parts, the transmitting side and the receiving side. We make use of the WIZnet Serial to Ethernet board to publish the data received from the arduino to the MQTT broker and a nodeMCU to subscribe to the data at the receiving side. The technical steps for both the ends are given in detail in the next section.

Steps

Transmitting Side

Circuit Connections:

Step 1: Connect the Hall effect sensors(A3144) and the gryoscope sensor(MPU6050) to the arduino as shown below. (Refer to the pinout diagrams for the connections)

o Hall sensors are connected to d9 and d6 of the arduino

Step 2: Connect the Arduino Uno to the WIZ750SR board (with Serial to Ethernet converter) using a RS-232 cable as follows:

o The transmitting pin (pin 11 in this case) of the Arduino Uno is connected to the receiving pin of the WIZ750SR Serial to Ethernet using the RS-232 cable and jumper wires.

o Receiving pin (pin 10 in this case) of the Arduino is connected to the transmitting pin.

o Ground pin of the Arduino is connected to the ground pin of the WIZnet board.

Step 3: Connect the WIZnet board to the network (for MQTT connection) using an Ethernet LAN cable.

Step 4: Connect both boards to the PC to upload the codes.

- Code for Transmitting Side:

Steps to upload codes to the Serial to Ethernet board:

1. Push the boot switch from normal mode to the boot mode.

2. Open the serial port using ISP tool software.

3. Once “serial open complete” is displayed, upload the ‘os.mbed’ complied code.

4. Push the boot switch back to the normal position and reset the wiz board.

5. The data received by the board can be monitored using “COOLTERM”.

//Code to publish data from the Wiznet Board
#include "mbed.h"
#include "MQTTEthernet.h"
#include "MQTTClient.h"
#define ECHO_SERVER_PORT   7
Serial a(D1,D0);
   Serial s(USBTX, USBRX);
int arrivedcount = 0;
void messageArrived(MQTT::MessageData& md)
{
   MQTT::Message &message = md.message;
   printf("Message arrived: qos %d, retained %d, dup %d, packetid %d\n", message.qos, message.retained, message.dup, message.id);
   printf("Payload %.*s\n", message.payloadlen, (char*)message.payload);
   ++arrivedcount;
}
void baud(int baudrate) {
   s.baud(baudrate);
}
int main (void)
{
   baud(9600);
   printf("Wait a second...\r\n");
   char* topic = "Ultrasonic";
   MQTTEthernet ipstack = MQTTEthernet();
   MQTT::Client<MQTTEthernet, Countdown> client = MQTT::Client<MQTTEthernet, Countdown>(ipstack);
   char* hostname = "172.16.73.4";
   int port = 1883;
   int rc = ipstack.connect(hostname, port);
   if (rc != 0)
       printf("rc from TCP connect is %d\n", rc);
   printf("Topic: %s\r\n",topic);
   MQTTPacket_connectData data = MQTTPacket_connectData_initializer;       
   data.MQTTVersion = 3;
   data.clientID.cstring = "parents";
   if ((rc = client.connect(data)) != 0)
       printf("rc from MQTT connect is %d\n", rc);
   if ((rc = client.subscribe(topic, MQTT::QOS0, messageArrived)) != 0)
       printf("rc from MQTT subscribe is %d\n", rc); 
   MQTT::Message message;
   char buf[100];
   while (true) 
   {
       char b;
       int i=0;
       char q[100];
       if(a.readable())
       { 
           for(i=0;i<=1;i++)
           {
           b= a.getc();
           q[i]=b;
           s.printf("%c",q[i]);
           }
       sprintf(buf,"%s",q);
       message.qos = MQTT::QOS1;
       message.retained = false;
       message.dup = false;
       message.payload = (void*)buf;
       message.payloadlen = strlen(buf);
       rc = client.publish("Ultrasonic", message);
       client.yield(2000);
           }
           }
}

Steps to upload code to Arduino Uno:

• Connect the Arduino Uno to the PC.

• Select the correct COM port from Tools->Port.

• Select Arduino Uno from Tools->Boards.

• Compile and upload the code.

//Arduino code to recieve values of the sensors and serially transmit it to the wiznet  board
//defines pins numbers
#include<SoftwareSerial.h>
#include"Wire.h"
//I2Cdev and MPU6050 must be installed as libraries, or else the .cpp/.h files
//for both classes must be in the include path of your project
#include"I2Cdev.h"
#include"MPU6050.h"
//class default I2C address is 0x68
//specific I2C addresses may be passed as a parameter here
//AD0 low = 0x68 (default for InvenSense evaluation board)
//AD0 high = 0x69
MPU6050accelgyro;
int16_tax, ay, az;
int16_tgx, gy, gz;
SoftwareSerialmySerial(10, 11); // RX, TX
inthPin=9;
intvalue1;
intmyInts[2];
constint trigPin = 3;
constint echoPin = 2;
longduration;
intdistance;
inthPin1=6;
inthState1=0;
intvalue;
inthState=0;
intolds=0;
intod;
intolds1=0;
//accelerometer values
intaccel_reading;
intaccel_corrected;
intaccel_offset = 200;
floataccel_angle;
floataccel_scale = 1; // set to 0.01
voidsetup() {
//join I2C bus (I2Cdev library doesn't do this  automatically)
Wire.begin();
 accelgyro.initialize();
 pinMode(trigPin,OUTPUT); // Sets the trigPin as an Output
pinMode(echoPin,INPUT); // Sets the echoPin as an Input
 //put your setup code here, to run once:
Serial.begin(9600);
 //put your setup code here, to run once:
pinMode(hPin,INPUT);
pinMode(hPin1,INPUT);
 mySerial.begin(9600);
 pinMode(LED_BUILTIN,OUTPUT);
olds=olds1=80;
}
voidloop() {
digitalWrite(trigPin,LOW);
delayMicroseconds(2);
// Sets the trigPin onHIGH state for 10 micro seconds
digitalWrite(trigPin,HIGH);
delayMicroseconds(10);
digitalWrite(trigPin,LOW);
// Reads the echoPin,returns the sound wave travel time in microseconds
duration =pulseIn(echoPin, HIGH);
// Calculating thedistance
distance=duration*0.034/2;
// Prints the distanceon the Serial Monitor
accelgyro.getMotion6(&ax,&ay, &az, &gx, &gy, &gz);
 //accelerometer_X_Axis angle calc
 accel_reading= ax;
 accel_corrected= accel_reading - accel_offset;
 accel_corrected= map(accel_corrected, -16800, 16800, -90, 90);
 accel_corrected= constrain(accel_corrected, -90, 90);
 accel_angle= (float)(accel_corrected * accel_scale);
  // put your main code here, to run repeatedly:
 hState=digitalRead(hPin);
 hState1=digitalRead(hPin1);
if(hState != olds ||hState1 != olds1){
 if(hState==LOW && hState1==LOW){
value=1;
olds = hState;
olds1 = hState1;
}
if (hState1==HIGH&& hState==LOW){
value=2;
olds = hState;
olds1 = hState1; }
if (hState1==LOW&& hState==HIGH){
value=3;
olds = hState;
olds1 = hState1;}
if (hState1==HIGH&& hState==HIGH){
value=4;
olds = hState;
olds1 = hState1;
}
}
else{
  if(accel_angle>=0 &&accel_angle<=30)
  {
 value= 5;
 od=value;
  }
 elseif(accel_angle>=30 && accel_angle<70)
 {
 value=6;
 od=value;
 }
 elseif(accel_angle>=70 && accel_angle<=90)
 {
 value=7;
 od=value;
 }
 else{
 if(od==0){
 value=5;
 }
 else{
 value=od;}
 }
}
mySerial.println(value);
Serial.println(value);
delay(1000);
}

Receiving Side

- Circuit Connection

Step 1: Connect three servo motors to the NodeMCU.

Step 2: Position the servo motors according to the picture below.

Step 3: Upload the code to NodeMCU (ESP8266) via PC.

Step 4: Check the output on the Serial Monitors.

- Code for Receiving Side:

Steps to upload the code into NodeMCU:

• Connect the NodeMCU (ESP8266) to the PC.

• Select the correct COM port from Tools->Port.

• Add the needed libraries to the Ardunio compiler. (Refer 1.1)

• Select NodeMCU (ESP8266) from Tools->Boards.

• Compile and upload the code.

/*
This code basically sets up an ESP8266 board as a MQTT client and helps in subscribing to desired topics that are published.
It is required that the board in the 'Tools' section must be selected to NodeMCU and the ESP8266 drivers are installed.
The baud rate must be set at 115200
Here we are operating three servo motors using the nodeMCU based on the values received via the MQTT topic
*/
#include <ESP8266WiFi.h>
#include <PubSubClient.h>
#include <Servo.h>
// Creating objects for the 3 servo motors
Servo servo;
Servo servo1;
Servo servo2;
// The pins we have used for the servo motors are D1,D2 and D3
// Update these with values suitable for your network.
const char* ssid = "CDI";
const char* password = "Cdi*1717";
const char* mqtt_server = "172.16.73.4";
WiFiClient espClient;
PubSubClient client(espClient);
long lastMsg = 0;
char msg[50];
int value = 0;
char old ;
void setup() {
 //Initial Setup Code
 Serial.begin(115200);
 setup_wifi();
 client.setServer(mqtt_server, 1883);
 client.setCallback(callback);
 delay(1000);
}
//Code for connecting to the wifi
void setup_wifi() {
 delay(10);
 // We start by connecting to a WiFi network
 Serial.println();
 Serial.print("Connecting to ");
 Serial.println(ssid);
 WiFi.begin(ssid, password);
 while (WiFi.status() != WL_CONNECTED) {
   delay(500);
   Serial.print(".");
 }
 Serial.println("");
 Serial.println("WiFi connected");
 Serial.println("IP address: ");
 Serial.println(WiFi.localIP());
}
//This function is responsible for subscribing to a topic and receiving the values from it 
void callback(char* topic, byte* payload, unsigned int length) {
 Serial.print("Message arrived [");
 Serial.print(topic);
 Serial.print("] ");
 old = (char)payload[0];
 //detach each time to avoid jitter of the servo motors
 servo2.detach();
 servo.detach();
 servo1.detach();
for (int i = 0; i < length; i++) {
 Serial.println("Received message is :");
 Serial.println((char)payload[i]);
 servo.detach();
 servo1.detach();
 servo2.detach();
 if((char)payload[i] == '1')
 {
   servo.detach();
   servo2.detach();
   servo1.detach();
   servo.attach(0);
   servo.write(89);
   delay(1000);
   servo.detach();
   servo1.attach(4);
   servo1.write(89);
   delay(1000);
   servo1.detach();
   servo2.detach();
   old='1';
   break;
 }
if((char)payload[i] == '2')
 {
   servo.detach();
   servo2.detach();
   servo1.detach();
   servo1.attach(4);
   servo1.write(89);
   delay(1000);
   servo1.detach();
   servo2.detach();
   old='2';
   break;
 }
if((char)payload[i] == '3')
 {
   servo.detach();
   servo2.detach();
   servo1.detach();
   servo.attach(0);
   servo.write(89);
   delay(1000);
   servo.detach();
   servo2.detach();
   old='3';
   break;
 }
if((char)payload[i] == '4')
 {
   servo.detach();
   servo1.detach();
   servo2.detach();
   servo.attach(0);
   servo.write(180);
   delay(1000);
   servo.detach();
   servo1.attach(4);
   servo1.write(180);
   delay(1000);
   servo1.detach();
   servo2.detach();
   old='4';
   break;
 }
 if((char)payload[i]=='5')
 {
   servo2.detach();
   servo2.attach(5);
   servo2.write(179);
   delay(1000);
   servo2.detach();
 }
 if((char)payload[i]=='6')
 { servo2.detach();
   servo2.attach(5);
   servo2.write(89);
   delay(1000);
   servo2.detach();
  }
 if((char)payload[i]=='7')
 { servo2.detach();
   servo2.attach(5);
   servo2.write(2);
   delay(1000);
   servo2.detach();
 }
   servo2.detach();
 }
   servo.detach();
   servo1.detach();
   servo2.detach();
}
//This function is to reconnect once the connection is lost
void reconnect() {
 // Loop until we're reconnected
 while (!client.connected()) {
   Serial.print("Attempting MQTT connection...");
   // Attempt to connect
   if (client.connect("ESP8266Client")) {
     Serial.println("connected");
     // Once connected, publish an announcement...
     // ... and resubscribe
     client.subscribe("Ultrasonic");
   } else {
     Serial.print("failed, rc=");
     Serial.print(client.state());
     Serial.println(" try again in 5 seconds");
     // Wait 5 seconds before retrying
     delay(5000);
   }
 }
}
void loop() {
 if (!client.connected()) {
   reconnect();
 }
 client.loop();
} 

Future Enhancements

We aim at reducing the latency time and developing a client side monitoring application as further improvements to make the project implementable in a versatile range of industries.