Hardware components | ||||||
 |
| × | 1 | |||
| × | 1 | ||||
Software apps and online services | ||||||
 |
| |||||
| ||||||
 |
| |||||
I have this Combat Creature Attacknid that I want to add face seeking capabilities. The idea is to find a familiar object, like a face. Center on the target and lock in and shoot darts! It would roam around the room until it finds a target. It would have a camera on top of it to get first person view. Add drone like capabilities to the Spider bot, remotely control it. (Of course, I want some safety mechanism)
This is the next generation Spider Bot. Let's see how I made this happen
DescriptionHad to close my eyes just in case the dart starts shooting my face. I aimed it to my body. It's a fun project, especially figuring out the opencv and hacking attacknid.
What you Need:1. Special Edition Combat Creatures Attacknid. I have the bluetooth version of Attacknid, not sold anymore. Rare find.
http://www.combatcreatures.com/attacknids
2. Creator Ci20
3. USB Camera
If this project got you interested in opencv in python or nodeJS, feel free to add respect and follow me. Thanks.
Step 1: Update Debian Image on Ci20Here are the instructions how to update the Debian Image in your Ci20
http://elinux.org/CI20_Distros
Step 2: Install Python and OpenCVhttps://github.com/alduxvm/rpi-opencv
Your rpi must be connected to internet, and updated...
sudo apt-get updatesudo apt-get upgrade
Install the essentials:
sudo apt-get install python-wxgtk2.8 python-matplotlib python-opencv python-pip python-numpy
I had trouble installing NodeJS using debian packages since Creator Ci20 is a MIPS architecture, you have to compile nodeJS from source.
https://www.vultr.com/docs/installing-node-js-from-source-on-ubuntu-14-04
Step 4: Use Wireshark to monitor bluetooth serial messages to AttacknidThe Attacknid I got is bluetooth-enabled. They don't have an API available and any documentation so I monitor the message coming from attacknid phone app to the spider bot.
http://stackoverflow.com/questions/23877761/sniffing-logging-your-own-android-bluetooth-traffic
https://www.nowsecure.com/blog/2014/02/07/bluetooth-packet-capture-on-android-4-4/
Here are the commands I was able to get
forward = "7100028080808001";
backward = "7100828080808002";
left = "7100808280808008";
right = "7100800280808004";
down = "7100808002808010";
up = "7100808082808020";
light = "7100808080800240";
fire = "7100808080808280";
The Creator ci20 has bluetooth module so I can connect to Attacknid via bluetooth.
I used this code to control the Attacknid. To test the attacknid I used the bluetooth-serial-port package and the cylon / cylon keyboard package. The bluetooth serial port package to send commands to the Attacknid. The cylon package to map keyboard. This is already a fun project controlling the attacknid, creating an api for it.
https://github.com/rlyle78/nodebot-opencvpython/blob/master/keyboard.js
//keyboard.js
"use strict";
var Cylon = require("cylon");
var commands = {};
commands["forward"] = "7100028080808001";
commands["backward"] = "7100828080808002";
commands["left"] = "7100808280808008";
commands["right"] = "7100800280808004";
commands["down"] = "7100808002808010";
commands["up"] = "7100808082808020";
commands["light"] = "7100808080800240";
commands["fire"] = "7100808080808280";
//var selectedCommand = commands["forward"];
var btSerial = new (require('bluetooth-serial-port')).BluetoothSerialPort();
btSerial.on('found', function(address, name) {
console.log(address);
btSerial.findSerialPortChannel(address, function(channel) {
btSerial.connect(address, channel, function() {
console.log(address);
console.log('connected');
btSerial.on('data', function(buffer) {
//console.log(buffer.toString('utf-8'));
});
}, function () {
console.log('cannot connect');
});
// close the connection when you're ready
btSerial.close();
}, function() {
console.log('found nothing');
});
});
btSerial.inquire();
function SendCommand(command) {
if (btSerial.isOpen()) {
btSerial.write(new Buffer(command, 'hex'), function (err, bytesWritten) {
if (err) console.log(err);
});
}
}
Cylon.robot({
connections: {
keyboard: { adaptor: "keyboard" }
},
devices: {
keyboard: { driver: "keyboard" }
},
work: function(my) {
my.keyboard.on('left', function (key) {
console.log('left key pressed');
SendCommand(commands['left']); });
my.keyboard.on('right', function (key) {
console.log('right key pressed');
SendCommand(commands['right']);
});
my.keyboard.on('up', function (key) {
console.log('up key pressed');
SendCommand(commands['up']);
});
my.keyboard.on('down', function (key) {
console.log('down key pressed');
SendCommand(commands['down']);
});
my.keyboard.on('f', function (key) {
SendCommand(commands['forward']);
});
my.keyboard.on('b', function (key) {
SendCommand(commands['backward']);
});
my.keyboard.on('n', function (key) {
SendCommand(commands['fire']);
});
my.keyboard.on('m', function (key) {
SendCommand(commands['light']);
});
}
}).start();
I forked this code
https://github.com/gr3gdev/node-python-opencv
Module NodeJS for python-opencv
This module allows to use the features of opencv via python for:
- motion detection detection
- facial recognition
- streaming a webcam
https://github.com/rlyle78/node-python-opencv
The webcam part is slow so I added some parts that I learned from this
https://github.com/alduxvm/rpi-opencv
The face detection demo runs faster on raspberry pi
https://github.com/alduxvm/rpi-opencv/blob/master/face-detection.py
so I created a python server that would communicate with my node js app
https://github.com/rlyle78/node-python-opencv/blob/master/python/webcam_detect.py
import cv, imutils, base64, socket, sys, detect
import numpy as np
from os.path import isfile
MOVE_DETECTION = 'MOVDET'
FACE_DETECTION = 'FACDET'
FACE_RECOGNIZE = 'FACREC'
if __name__ == '__main__':
if len(sys.argv) == 2:
TCP_IP = '127.0.0.1'
TCP_PORT = int(sys.argv[1])
video = cv.CreateCameraCapture(0) #cv2.VideoCapture(0)
width = 320 #leave None for auto-detection
height = 240 #leave None for auto-detection
cv.SetCaptureProperty(video,cv.CV_CAP_PROP_FRAME_WIDTH,width)
cv.SetCaptureProperty(video,cv.CV_CAP_PROP_FRAME_HEIGHT,height)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((TCP_IP, TCP_PORT))
s.listen(1)
while 1:
conn, addr = s.accept()
data = conn.recv(16)
try:
if data == 'STOP':
break
if data.startswith(FACE_DETECTION):
buffer = int(data[6:])
conn.send('BUFFER OK')
data = conn.recv(buffer)
print data
flux = data.split(' ')
while True:
image = cv.QueryFrame(video)
faces = detect.FaceDetection.findFrame(image, flux[0], float(flux[1]), int(flux[2]))
if (faces):
finds = '{"faces": ['
index = 0
for ((x, y, w, h), n) in faces:
index += 1
finds += '{"x": "%s", "y": "%s", "w": "%s", "h": "%s"}' % (x, y, w, h)
if index < len(faces):
finds += ','
finds += ']}'
conn.sendall('%s' % finds)
if cv.WaitKey(10) >= 0:
break
except:
message = '%s' % sys.exc_value
message = message.replace('\\', '/').replace('\n', '').replace('\r', '')
conn.send('{"error": "%s"}' % message)
conn.close()
s.close()
video.release()
cv.destroyAllWindows()
then use this code for face detection
https://github.com/rlyle78/node-python-opencv/blob/master/python/detect.py
Step 5: Send messages from Python OpenCV to NodeJS Attacknid app via socketsI created a Node JS app that would communicate with python OpenCV server. I started receiving coordinates and width of the detected face. I setup logic that would move Attacknid to left/right to focus on the subject. The closer the subject is, it starts shooting dart.
var opencv = require('node-python-opencv');
var commands = {};
commands["forward"] = "7100028080808001";
commands["backward"] = "7100828080808002";
commands["left"] = "7100808280808008";
commands["right"] = "7100800280808004";
commands["down"] = "7100808002808010";
commands["up"] = "7100808082808020";
commands["light"] = "7100808080800240";
commands["fire"] = "7100808080808280";
var btSerial = new (require('bluetooth-serial-port')).BluetoothSerialPort();
btSerial.on('found', function(address, name) {
console.log(address);
btSerial.findSerialPortChannel(address, function(channel) {
btSerial.connect(address, channel, function() {
console.log(address);
console.log('connected');
btSerial.on('data', function(buffer) {
//console.log(buffer);
});
}, function () {
console.log('cannot connect');
});
// close the connection when you're ready
btSerial.close();
}, function() {
console.log('found nothing');
});
});
btSerial.inquire();
var sendCommand = function(command) {
if (btSerial.isOpen()) {
btSerial.write(new Buffer(command, 'hex'),
function (err, bytesWritten) {
if (err) console.log(err);});
}
}
lastcommand = "";
setInterval(function () {
//console.log(lastcommand);
if (lastcommand != "") {
console.log("sending " + lastcommand);
sendCommand(lastcommand);
}
},450);
var webcamdetector = new opencv.webcamdetector({
port: 9010,
useFastDetect : true
});
setTimeout(function () {
console.log('ready');
webcamdetector.findFaces({
'haarcascade': 'face.xml',
'scaleFactor': 1.2,
'minNeighbors': 2
}, function (data, err) {
if (err)
console.log("Error:" + err);
else
{
// Return JSON object {faces: [{x: N, y: N, w: N, h: N}, ...]}
console.log(JSON.stringify(data));
if (data.faces[0])
{
var face = data.faces[0];
var x = Number(face.x);
var y = Number(face.y);
var w = Number(face.w);
var h = Number(face.h);
console.log(x);
if (w > 60 && h > 60) {
console.log('fire');
lastcommand = commands['fire'];
}
else if (x < 10)
lastcommand = commands['left'];
else if (x > 90)
lastcommand = commands['right'];
else
lastcommand = ""
}
}
});
}, 5000);
Open 2 terminal windows
ci20@ci20:~$ python node_modules/node-python-opencv/python/webcam_detect.py 9010
in another window
ci20@ci20:~$ node docterdoom.js
What if it can recognize the face, look up data information about the target? With the power of the cloud, that's possible. That would be so awesome! Only shoot the right target to terminate.
If this project got you interested in using wireshark to monitor bluetooth commands or even working on opencv in python or nodeJS, feel free to add respect and follow me. Thanks.
Comments