Published September 17, 2016

Gesture Controlled Robotic Arm Using Kinect & Arduino

This is a Simulink model to control a robotic arm using gestures that are captured using Kinect; robotic arm developed using servo motors.

IntermediateShowcase (no instructions)3 hours18,469

Gesture Controlled Robotic Arm Using Kinect & Arduino

Things used in this project

Hardware components

Arduino Mega 2560

Microsoft Kinect Sensor

Kinect for XBox 360

Microsoft Xbox Kinect Adapter

Adafruit Servo Motor

SparkFun USB to Serial Breakout - FT232RL

Gripper

Solderless Breadboard Full Size

Jumper wires (generic)

Software apps and online services

Arduino IDE

Microsoft Kinect for Windows SDK v1.6

Microsoft Windows SDK 7.1

MATLAB

MATLAB 2013a

Story

About

This is a Simulink model to control a robotic arm by gestures that are captured using a Kinect sensor. This robot arm is developed using servo motors that replicate the right arm shoulder, elbow and hand movements.

My file at Mathworks File Exchange: https://in.mathworks.com/matlabcentral/fileexchange/58189-gesture-controlled-robotic-arm-using-kinect--simulink-support-for-kinect--and-arduino

Simulation Window in Matlab and the Arm Position According to our Gestures

Software Required

Matlab 2013a or later
Arduino IDE

Components Required

Kinect Sensor
Arduino Board
Servo Motors
Matlab 2013a

This is a project by B.Avinash & J.Karthikeyan

The Zip file contains the Matlab Simulink Support for Kinect and the Simulink model named "GestureControlledRoboticArm" that we have created as well as the Arduino code and wiring diagram. The Simulink Support for Kinect was downloaded from Mathworks File Exchange and the full credits go to "Takashi Chikamasa" for providing it. We do not own the copyrights or have developed the Simulink Support for Kinect.

Simulink Support for Kinect consists of IMAQ, Image, IR, Depth, Motion and Skeleton blocks to achieve real-time image data acquisition from Kinect devices such as Microsoft Kinect or ASUS Xtion PRO under Simulink simulation environment. It works with 32bit/64bit version of MATLAB R2010b or later in Windows 7 32bit/64bit and it requires the following 3rd party software:

C MEX Compiler: Microsoft Visual Studio 2010 Express Edition (VC++) - Microsoft Windows SDK 7.1
To use OpenNI: http://openni.ru/ - OpenNI - PrimeSense NITE - PrimeSensor Module
To use Microsoft Kinect SDK: http://www.microsoft.com/en-us/kinectforwindows/ - Microsoft Kinect for Windows SDK v1.6

For more detailed information, please refer to document in the zip file and help of the blocks.

Optional MATLAB products:

Computer Vision System Toolbox (highly recommended)
DSP System Toolbox (to acquire Kinect Microphone Array data)
Simulink Coder (to generate *.exe file)

Acknowledgements: Kinect Matlab, Simulink Support For Kinect, and Kinect Microsoft Sdk inspired this project.

Awards and Recognitions

"Gesture Controlled Robotic Arm Using Kinect (Simulink Support For Kinect) and Arduino" has been chosen (29/10/2016) as Pick of the Week! - Matlab Simulink Hardware Challenge 2016 : http://blogs.mathworks.com/pick/2016/10/28/2016-matlab-and-simulink-hardware-challenge/
Won the 4th place in "MATLAB International Simulink Hardware Challenge 2016" : https://in.mathworks.com/matlabcentral/fileexchange/58189-gesture-controlled-robotic-arm-using-kinect--simulink-support-for-kinect--and-arduino

Schematics

Code

Arduino Program to Receive Angle Data from Matlab

#include <Servo.h> 
 
Servo myservo1; //Servo Shoulder Vertical Movement
Servo myservo2; //Servo Shoulder horizontal Movement
Servo myservo3; //Elbow Movement
Servo myservo4; //Gripper Movement

void setup() 
{
  Serial1.begin(9600);
  myservo1.attach(9); //Attach Shoulder Vertical Movement Control Signal Pin To 9
  myservo2.attach(10); //Attach Shoulder Horizontal Control Signal Pin To 10
  myservo3.attach(3); //Attach Elbow Movement Control Signal Pin To 3
  myservo4.attach(5); //Attach Gripper Movement Control Signal Pin To 5
}

void loop() 
{
 if(Serial1.available()>0) //Check for any data in Serial Port
 {
   byte con = Serial1.read(); //Read serial data and store it in con
  
   if(con > 0)
   {
   if(con < 50)
   {
   con = (180*con)/50;  //Decrypt the data obtained
   myservo1.write(con); //Write the data to myservo1          
   delay(15);
   }
   }
  
   if(con > 50)
   {
   if(con < 100)
   {
   con = (180*(con-50))/50; //Decrypt the data obtained
   myservo2.write(con); //Write the data to myservo2            
   delay(15);
   }
   }
    
   if(con > 100)
   {
   if(con < 150)
   {
   con = (180*(con-100))/50; //Decrypt the data obtained
   myservo3.write(con); //Write the data to myservo3
   delay(15);
   }
   }
   
   if(con > 150)
   {
   if(con < 200)
   {
   con = (180*(con-150))/50; //Decrypt the data obtained
   if(con<20)
   {
   myservo4.write(180); //Write the data to myservo4
   delay(15);
   }
   if(con>20)
   {
   myservo4.write(60);
   delay(15);
   }
  }
 }
      
      }

}

Credits

Avinash Baranitharan

2 projects • 13 followers

Master of the Possible, an Engineer.

Karthikeyan

1 project • 5 followers

Some what good in arduino.

Thanks to Takashi Chikamasa.

Gesture Controlled Robotic Arm Using Kinect & Arduino