Computer Vision + Mechanical Arm

Control a mechanical arm with your movements and gestures!

AdvancedShowcase (no instructions)1,003

Things used in this project

Hardware components

Arduino UNO

SG90 Micro-servo motor

Camera (generic)

Story

We already had constructed this arm before, but our motivation was to ask: "how can we differently control this arm?".

With this in mind, we built a system in Python to recognize our face and hand movements:

Therefore, with the use of the cv2 and MediaPipe libraries, we could understand and classify the fingers:

All left fingers down: turn clockwise

On left finger up: turn anti-clockwise

One right-hand finger up: move the X-axis servo motor.

Two right-hand fingers up: move the Y-axis servo motor.

And so on...

External battery to support the current of all servos

Our communication from Python to the Arduino's system was by UART.

Video of the project working: https://imgur.com/4hC5K3f

I will organize the Python code and I will update it here soon!

Code

Arduino Controller

#include <Servo.h>
#define pinServ1 11
#define pinServ2 10
#define pinServ3 9
#define pinServ4 8

Servo serv1 , serv2 , serv3 , serv4;

int motor1 = 90 ;
int motor2 = 52 ;
int motor3 = 67 ;
int motor4 = 0  ;


char buf;

void setup() {
  Serial.begin(9600);

  serv1.attach(pinServ1);
  serv2.attach(pinServ2);
  serv3.attach(pinServ3);
  serv4.attach(pinServ4);

  
}

void loop() {
  while(Serial.available() > 0)
  {
    buf = Serial.read();

      if(buf == 'A' && motor1 < 170){
        serv1.write(motor1 += 1);
        serv1.detach();
      }
    
       if(buf == 'B' && motor1 > 10){
        serv1.write(motor1 -= 1);
        serv1.detach();
      }
  
      if(buf == 'C' && motor2 < 170){
        serv2.write(motor2 += 1);
        serv2.detach();
      }
    
       if(buf == 'D' && motor2 > 10){
        serv2.write(motor2 -= 1);
        serv2.detach();
      }
  
      if(buf == 'E' && motor3 < 170){
        serv3.write(motor3 += 1);
        serv3.detach();
      }
    
       if(buf == 'F' && motor3 < 10){
        serv3.write(motor3 -= 1);
        serv3.detach();
      }
      if(buf == 'G' && motor4 < 170){
        serv4.write(motor4 += 1);
        serv4.detach();
      }
    
       if(buf == 'H' && motor4 < 10){
        serv4.write(motor4 -= 1);
        serv4.detach();
      }
    delay(15); 

  }

}