Raspberry Pi as Robot Arm Controller with 3D Gesture Shield

About

This shield is based on Microchip MGC3130 chip, which enables the Raspberry Pi with 3D gesture recognition and motion tracking function. It can capture x y z position information, can also do proximity sensing and touch sensing, support tap and double click.

As shown in the figure below, the recognition area is divided into two parts: the strip area distributed around and a central panel.

The strip areas can sense the orientation change, including the North, South, West, and East. The central area can sense touch, tap, double click, and gestures in the air above. That's why we call it 3D Gesture & Tracking Shield, you don't even need to touch the central area directly, just wave your hand above the central area, and this shield can sense your movements.

Thanks to the Microchip’s patented GestIC® technology, this shield utilizes electrical near-field sensing to detect movements. The shield generates a magnetic field above the central panel when the hand approaches, it will interfere with the magnetic field, and the magnetic field receiver below the shield can detect the change.

This HAT communicates with Raspberry Pi via the I2C interface, also we have reserved a Grove I2C connector in case you need to work with other modules.

What an amazing module, with the help it, you can use gestures to control lights, TV, speakers... Just unleash your imagination and create more magical projects.

We come out the project to control the robot as below, Please enjoy.

Hardware Connection

Here is the hardware connection. For more info, please refer to: http://wiki.seeedstudio.com/3D-Gesture-Tracking-Shield-for-Raspberry-Pi-MGC3130/

Software Configuration

#1. Play uArm Metal with Raspberry Pi

Step 1. Install uArm python library

cd ~
git clone https://github.com/uArm-Developer/pyuarm.git
cd pyuarm
sudo python setup.py install

Step 2. Play with robot with python, Follow here to test the robot.

pi@raspberrypi:~/pyuarm/pyuarm $ python
Python 2.7.16 (default, Apr  6 2019, 01:42:57)
[GCC 8.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyuarm
>>> arm= pyuarm.UArm()
pyuarm - INFO - pyuarm version: 2.2.5.3
pyuarm - INFO - Connecting from port - /dev/ttyUSB0...
pyuarm - INFO - Connected...
pyuarm - INFO - Firmware Version: 2.2.1
>>> arm.set_servo_angle(0, 30)
True
>>> arm.set_servo_angle(0, 0)
True
>>> arm.set_servo_angle(0, 30)
True
>>> arm.set_servo_angle(0, 60)
True
>>> arm.set_servo_angle(0, 90)
True
>>> arm.set_servo_angle(1, 0)
True
>>> arm.set_servo_angle(1, 30)
True
>>> arm.set_servo_angle(1, 60)
True
>>> arm.set_servo_angle(1, 90)
True
>>> arm.set_servo_angle(2, 90)
True
>>> arm.set_servo_angle(2, 60)
True
>>> arm.set_servo_angle(2, 30)
True
>>> arm.set_servo_angle(2, 0)
True
>>> arm.set_servo_angle(2, -30)

#2. Play 3D Gesture & Tracking Shield for Raspberry Pi (MGC3130) with Raspberry Pi

#2.1.Enable I2C

Step 1. Run sudo raspi-config.

Step 2. Use the down arrow to select 5 Interfacing Options.

Step 3. Arrow down to P5 I2C.

Step 4. Select yes when it asks you to enable I2C.

Step 5. Also select yes if it asks about automatically loading the kernel module.

Step 6. Use the right arrow to select the button.

Step 7. Select yes when it asks to reboot.

#2.2. Play 3D gesture with Python code

Step 1. Run below command to verify if the shield is detected by raspberry pi through I2C.

i2cdetect -y 1

Here is the output and the I2C address is 0x42.

pi@raspberrypi:~ $ i2cdetect -y 1
    0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- UU -- -- -- --
40: -- -- 42 -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --
pi@raspberrypi:~ $

Step 2. Download the code and run the demo as below.

cd ~
git clone https://github.com/linux-downey/seeed_mgc3x30_python.git
cd seeed_mgc3x30_python
./flick-demo

Here is the output.

                                                        **** Flick Demo ****
  ┌─┐
  │ X Y Z     :                                                                                                                  │
  │ Flick     :                                                                                                                  │
  │ Airwheel  :                                                                                                                  │
  │ Touch     :                                                                                                                  │
  │ Tap       :                                                                                                                  │
  │ Doubletap :                                                                                                                  │
  └─┘
  ┌─┐
  │ Firmware valid: Yes                                                                                                          │
  │ Hardware Revison: 99.128                                                                                                     │
  │ Params Start Addr: 0x7300                                                                                                    │
  │ Library Loader Version: 19.100                                                                                               │
  │ Library Loader Platform: Hillstar                                                                                            │
  │ Firmware Start Addr: 0x1000                                                                                                  │
  │ Firmware Version: 2.1.0;p:HillstarV01;x:Hillstar                                                                             │
  │ DSP: ID9001r3686;i:B;f:22500;nMsg;s:trunkr2163:MO;                                                                           │
  └─┘
                                                         Control-C to exit

• X/Y/Z: Lower left corner is X=0, Y=0. Z is the height. Max is 65535.
• Flick: Wave your hands over the module, you can see the wave direction as West to East, East to West, South to North and North to South.
• Airwheel: The number will increase with clockwise direction and reduce with counterclockwise direction. It will have some delay on the display.
• Touch : Put your finger on the specific area, you will get Center, East, West, South and North.
• Tap: Tap your finger on the specific area, you will get Center, East, West, South and North.
• Double Tap: Double Tap your finger on the specific area, you will get Center, East, West, South and North.

#2.3. Play 3D gesture with robot arm

Step 1. Copy the flick-demo as flick-demo-arm

cd ~/seeed_mgc3x30_python
cp flick-demo flick-demo-arm

Step 2. Modify the flick-demo-arm to include the robot arm functions.

import pyuarmarm= pyuarm.UArm()

We add 3 functions:

First is the pump function(tap is on and doubletap is off).

@flicklib.double_tap()def doubletap(position): global doubletaptxt    doubletaptxt = position    arm.set_pump(False)@flicklib.tap()def tap(position): global taptxt    taptxt = position    arm.set_pump(True)

Second, robot horizontal rotation function with air wheel, the response is little bit slow.

@flicklib.airwheel()def spinny(delta): global some_value global airwheeltxt    some_value += delta if some_value < 0:        some_value = 0 if some_value > 10000:        some_value = 10000    airwheeltxt = str(some_value/100)    arm.set_servo_angle(0, some_value/100)

Third, the z height motor up and down for picking and placing.

def flick(start,finish): global flicktxt    flicktxt = start + ' - ' + finish if flicktxt == "east - west":        arm.set_servo_angle(2, 30)  elif flicktxt == "west - east":        arm.set_servo_angle(2, 35)