Published December 28, 2017 © GPL3+

Walabot Security Robot with Alexa Command and Control

Use Walabot and a vision system to identify new objects arriving in the roving space of the robot. Alexa will be used to command and control

AdvancedWork in progressOver 2 days2,739

Top 25 Projects

3D Imaging with Walabot & Amazon Alexa

Walabot Security Robot with Alexa Command and Control

Things used in this project

Hardware components

Walabot Developer Pack

Raspberry Pi 3 Model B

Adafruit 16 channel PWM controller

Software apps and online services

Amazon Alexa Alexa Skills Kit

Amazon Web Services AWS IoT

Amazon Web Services AWS Lambda

Story

This is the continuation and improvement of a project I had started earlier on Hackster.io.

Robotic Assistant

Improved code for smooth azimuth elevation motion

closeup video of Walabot radar device tracking

Walabot radar device tracking target

1 / 3 • Azimuth Elevation Mount for Walabot and Camera

Improvements over the previous project

The Walabot device is moved to an Azimuth Elevation servo controlled mount. This increases the ability of the robot to observe its environment.

Alexa voice skills will be used to control the robot's motions and direct the use of the Walabot device and a camera

Alexa voice skills used to control multiple robots. Two will be demonstrated, until I get more project funding.

A very IoT thing

My previous IoT project was a Medicine Reminder . Portions of the Medicine Reminder project will be reused to control the robots.

Voice User Interface

The voice user interface is like a voice operated joystick for operating the robot. Any command can issued in any order. There is a single level to the operation there are no sub-levels to the command structure. This makes it similar to operating a car, your car does not care when you open or close the windows on the car.

VUI diagram

Alexa, Ask Freedom Security Robot , this is the invocation for the Alexa skill to operate the robot

Direction Left Turn (number) , this sets the next motion to be a left turn with a radius of (number) centimeters. The command immediately sends a MQTT command to the robot with turn radius. The AMAZON.NUMBER slot type is used, because I want to pick up any improvements that the programmers at Amazon come up with without changing my code.

Direction Straight, this sets the next motion to be straight ahead.

Direction Right Turn (number) , this sets the next motion to be a left turn with a radius of (number) centimeters. The command immediately sends a MQTT command to the robot with turn radius.

Motion Forward (number), this sets the next motion to be a forward motion of (number) centimeters. The command immediately sends a MQTT command to the robot with the forward motion distance to the robot.

Motion Backward (number), this sets the next motion to be a backward motion of (number) centimeters. The command immediately sends a MQTT command to the robot with the backward motion distance to the robot.

Move, initiates a move with the turn radius and the forward or backward distance set by the previous commands. The robot will use the same turn radius and motion distance if several Move commands are given.The command immediately sends a MQTT command to the robot causing motion.

Set Automatic mode, gives the robot permission to look around on its own. It still accepts commands to move and commands to direct the camera and Walabot .

Clear Automatic mode , tells the robot to stop looking around on its own.

Look Left (number), pan the az-el camera mount left (number) degrees. The command immediately sends a MQTT command to the robot to pan left.

Look Right (number), pan the az-el camera mount right (number) degrees. The command immediately sends a MQTT command to the robot to pan right.

Look Up (number), tilt the az-el camera mount up (number) degrees. The command immediately sends a MQTT command to the robot to tilt up.

Look Down (number), tilt the az-el camera mount down (number) degrees. The command immediately sends a MQTT command to the robot to tilt down.

Step by Step Instructions

1) Download Walabot API into RPi

A link to the Walabot getting started page use this link on the Raspberry Pi chromium web browser to go to download the installer for Raspberry Pi.

Select the RASPBERRYPI INSTALLER

A download occurs and places the installation file in the downloads directory.

The downloaded Walabot installation program

Open a terminal window to run the dpkg installer program. Note that the current directory is changed to /home/pi/Dowloads before starting dpkg.

cd /home/pi/Downloads
sudo dpkg -i walabot_maker_1.0.34_raspberry_arm32.deb

the terminal screen

A EULA screen will appear when the dpkg program is started. The TAB key is used to select <Ok> and make it turn red.

TAB to select <Ok>

TAB to select that yes you agree with the agreement.

TAB to select <Yes>

Then dpkg will work its magic and all the library files and example source is copied to the correct places on Raspberry Pi! Now we can start writing code after looking at the example code a little.

A link to the Walabot API

2) Make Azimuth Elevation Servos Move

The process for getting the 16 channel PWM controller board is well documented on the Adafruit website. Here is the link to the web page.

adafruit-16-channel-servo-driver-with-raspberry-pi/using-the-adafruit-library

I have included slight changes to the demo example code since I have two servos to move and I want to set a much smaller range of motion when I test the servos for the first time.

1 / 4 • installing I2C tools

3) Merge Walabot sensor.py example code with tom_simpletest.py (servo test program).

The merged code has the Walabot measurements and PWM commands going on with each loop. I have also added the target tracking code. The idea is to track a Walabot radar sensor detected target with the azimuth and elevation servos.

4) Install camera on Raspberry Pi

A link for installing a Camera on Raspberry Pi

First Picture from the camera

5) Work on Alexa Skill

I have to get the intents filled in , here are some examples. Several different sample utterance approaches are taken, because I want the user interface to work in a natural fashion. AMAZON.NUMBER is used as the slot type to handle all of the number voice based data entry.

DirectionLeftTurn

LookUp Intent

6) Create the Lambda function

The first test of the Lambda function will be a private test that I don't intend to publish as a formal Alexa Skill. After I prove the connectivity with RPi I will change the Lambda function for Account Linking and make it like a real Alexa based product.

Creating the Lambda function

Code

from __future__ import print_function # WalabotAPI works on both Python 2 an 3.
from __future__ import division
from sys import platform
from os import system
from imp import load_source
from os.path import join
# merging stuff learned from Medicine Reminder project
#import paho.mqtt.client as mqtt
import json, time, sys, ssl
import logging, logging.handlers
#import RPi.GPIO as GPIO #GPIO is not exactly what we want for I2C
# Import the PCA9685 module.
import Adafruit_PCA9685
import math

if platform == 'win32':
	modulePath = join('C:/', 'Program Files', 'Walabot', 'WalabotSDK',
		'python', 'WalabotAPI.py')
elif platform.startswith('linux'):
    modulePath = join('/usr', 'share', 'walabot', 'python', 'WalabotAPI.py')     

wlbt = load_source('WalabotAPI', modulePath)
wlbt.Init()

def PrintSensorTargets(targets):
    system('cls' if platform == 'win32' else 'clear')
    if targets:
        for i, target in enumerate(targets):
            print('Target #{}:\nx: {}\ny: {}\nz: {}\namplitude: {}\n'.format(
                i + 1, target.xPosCm, target.yPosCm, target.zPosCm,
                target.amplitude))
    else:
        print('No Target Detected')

def RobotSensorApp():
    deadband = math.pi * (6/180)
    # Initialise the PCA9685 using the default address (0x40).
    pwm = Adafruit_PCA9685.PCA9685()
    # Configure min and max servo pulse lengths
    el_servo_min = 350 #320  #25 degrees inclined down
    el_servo_max = 420  #25 degrees inclined up
    el_servo_ctr = 370 #level
    el_servo_cmd = el_servo_ctr #start from center position
    az_servo_min = 270 #ahead
    az_servo_max = 455 #45 degrees to the right
    az_servo_ctr = 365 #45 degrees to the left
    az_servo_cmd = az_servo_ctr #start from center position
    
    # Set frequency to 60hz, good for servos.
    pwm.set_pwm_freq(60)
    pwm.set_pwm(0, 0, el_servo_ctr) #elevation
    pwm.set_pwm(1, 0, az_servo_ctr) #azimuth
       
    # wlbt.SetArenaR - input parameters
    minInCm, maxInCm, resInCm = 30, 200, 3
    # wlbt.SetArenaTheta - input parameters
    minIndegrees, maxIndegrees, resIndegrees = -15, 15, 5
    # wlbt.SetArenaPhi - input parameters
    minPhiInDegrees, maxPhiInDegrees, resPhiInDegrees = -60, 60, 5
    # Set MTI mode
    mtiMode = False
    # Configure Walabot database install location (for windows)
    wlbt.SetSettingsFolder()
    # 1) Connect : Establish communication with walabot.
    wlbt.ConnectAny()
    # 2) Configure: Set scan profile and arena
    # Set Profile - to Sensor.
    wlbt.SetProfile(wlbt.PROF_SENSOR)
    # Setup arena - specify it by Cartesian coordinates.
    wlbt.SetArenaR(minInCm, maxInCm, resInCm)
    # Sets polar range and resolution of arena (parameters in degrees).
    wlbt.SetArenaTheta(minIndegrees, maxIndegrees, resIndegrees)
    # Sets azimuth range and resolution of arena.(parameters in degrees).
    wlbt.SetArenaPhi(minPhiInDegrees, maxPhiInDegrees, resPhiInDegrees)
    # Moving Target Identification: standard dynamic-imaging filter
    filterType = wlbt.FILTER_TYPE_MTI if mtiMode else wlbt.FILTER_TYPE_NONE
    wlbt.SetDynamicImageFilter(filterType)
    # 3) Start: Start the system in preparation for scanning.
    wlbt.Start()
    if not mtiMode: # if MTI mode is not set - start calibrartion
        # calibrates scanning to ignore or reduce the signals
        wlbt.StartCalibration()
        while wlbt.GetStatus()[0] == wlbt.STATUS_CALIBRATING:
            wlbt.Trigger()
    while True:
        appStatus, calibrationProcess = wlbt.GetStatus()
        # 5) Trigger: Scan(sense) according to profile and record signals
        # to be available for processing and retrieval.
        wlbt.Trigger()
        # 6) Get action: retrieve the last completed triggered recording
        targets = wlbt.GetSensorTargets()
        rasterImage, _, _, sliceDepth, power = wlbt.GetRawImageSlice()
        # PrintSensorTargets(targets)
        #PrintSensorTargets(targets)
        # Move servos
        dist_sq = 10000000 #infinity
        x_ang = 0
        y_ang = 0
        if targets:
            for i, target in enumerate(targets):
                dist_calc_sq = target.xPosCm * target.xPosCm
                dist_calc_sq += target.yPosCm * target.yPosCm
                dist_calc_sq += target.zPosCm * target.zPosCm
                if dist_calc_sq <= dist_sq:
                    dist_sq = dist_calc_sq
                    x_ang = math.atan(target.xPosCm / target.zPosCm)
                    y_ang = math.atan(target.yPosCm / target.zPosCm)

        #move if not against a travel limit
        if x_ang > deadband:
           if az_servo_cmd < az_servo_max:
                   az_servo_cmd+=10
        if x_ang < -deadband:
           if az_servo_cmd > az_servo_min:
                   az_servo_cmd-=10
        if y_ang > deadband:
           if el_servo_cmd < el_servo_max:
                   el_servo_cmd+=10
        if y_ang < -deadband:
           if el_servo_cmd > el_servo_min:
                   el_servo_cmd-=10
        #print(az_servo_cmd)            
        pwm.set_pwm(0, 0, el_servo_cmd) #elevation
        pwm.set_pwm(1, 0, az_servo_cmd) #azimuth
    
    # 7) Stop and Disconnect.
    wlbt.Stop()
    wlbt.Disconnect()
    print('Terminate successfully')

if __name__ == '__main__':
    RobotSensorApp()

{
    "interactionModel": {
        "languageModel": {
            "invocationName": "freedom security robot",
            "intents": [
                {
                    "name": "AMAZON.CancelIntent",
                    "samples": []
                },
                {
                    "name": "AMAZON.HelpIntent",
                    "samples": []
                },
                {
                    "name": "AMAZON.StopIntent",
                    "samples": []
                },
                {
                    "name": "DirectionLeftTurn",
                    "slots": [
                        {
                            "name": "TurnRadius",
                            "type": "AMAZON.NUMBER"
                        }
                    ],
                    "samples": [
                        "Turn Left {TurnRadius}",
                        "Left Turn {TurnRadius}",
                        "Go Left {TurnRadius}",
                        "Make Left Turn {TurnRadius}",
                        "Direction Left Turn {TurnRadius}"
                    ]
                },
                {
                    "name": "DirectionRightTurn",
                    "slots": [
                        {
                            "name": "TurnRadius",
                            "type": "AMAZON.NUMBER"
                        }
                    ],
                    "samples": [
                        "Turn Right {TurnRadius}",
                        "Right Turn {TurnRadius}",
                        "Go Right {TurnRadius}",
                        "Make Right Turn {TurnRadius}",
                        "Direction Right Turn {TurnRadius}"
                    ]
                },
                {
                    "name": "DirectionStraight",
                    "slots": [],
                    "samples": [
                        "Go Straight",
                        "Straight Ahead",
                        "Direction Straight"
                    ]
                },
                {
                    "name": "MotionForward",
                    "slots": [
                        {
                            "name": "move",
                            "type": "AMAZON.NUMBER"
                        }
                    ],
                    "samples": [
                        "Forward {move}",
                        "Ahead {move}",
                        "Move Ahead {move}",
                        "Motion Forward {move}"
                    ]
                },
                {
                    "name": "MotionBackward",
                    "slots": [
                        {
                            "name": "move",
                            "type": "AMAZON.NUMBER"
                        }
                    ],
                    "samples": [
                        "Go Back {move}",
                        "Go Backward {move}",
                        "Move Back {move}",
                        "Move Backward {move}",
                        "Motion Backward {move}"
                    ]
                },
                {
                    "name": "MoveCommand",
                    "slots": [],
                    "samples": [
                        "Start Motion",
                        "Start Move",
                        "Move Robot",
                        "Robot Move",
                        "Do a move",
                        "Move"
                    ]
                },
                {
                    "name": "SetAuto",
                    "slots": [],
                    "samples": [
                        "Go Auto",
                        "Go Automatic",
                        "Set Auto",
                        "Set Automatic Mode"
                    ]
                },
                {
                    "name": "ClearAuto",
                    "slots": [],
                    "samples": [
                        "Stop Automatic Mode",
                        "Stop Auto",
                        "Clear Auto",
                        "Clear Automatic Mode"
                    ]
                },
                {
                    "name": "LookLeft",
                    "slots": [
                        {
                            "name": "angleincr",
                            "type": "AMAZON.NUMBER"
                        }
                    ],
                    "samples": [
                        "left {angleincr}",
                        "Look left {angleincr}",
                        "Look to the left {angleincr}"
                    ]
                },
                {
                    "name": "LookRight",
                    "slots": [
                        {
                            "name": "angleincr",
                            "type": "AMAZON.NUMBER"
                        }
                    ],
                    "samples": [
                        "right {angleincr}",
                        "look right {angleincr}",
                        "look to the right {angleincr}"
                    ]
                },
                {
                    "name": "LookUp",
                    "slots": [
                        {
                            "name": "angleincr",
                            "type": "AMAZON.NUMBER"
                        }
                    ],
                    "samples": [
                        "look up {angleincr}",
                        "tilt backward {angleincr}",
                        "tilt back {angleincr}",
                        "tilt up {angleincr}",
                        "up {angleincr}",
                        "upward {angleincr}",
                        "look upward {angleincr}"
                    ]
                },
                {
                    "name": "LookDown",
                    "slots": [
                        {
                            "name": "angleincr",
                            "type": "AMAZON.NUMBER"
                        }
                    ],
                    "samples": [
                        "Look Downward {angleincr}"
                    ]
                }
            ],
            "types": []
        }
    }
}

# Simple demo of of the PCA9685 PWM servo/LED controller library.
# This will move channel 0 from min to max position repeatedly.
# Author: Tony DiCola
# License: Public Domain
from __future__ import division
import time

# Import the PCA9685 module.
import Adafruit_PCA9685


# Uncomment to enable debug output.
#import logging
#logging.basicConfig(level=logging.DEBUG)

# Initialise the PCA9685 using the default address (0x40).
pwm = Adafruit_PCA9685.PCA9685()

# Alternatively specify a different address and/or bus:
#pwm = Adafruit_PCA9685.PCA9685(address=0x41, busnum=2)

# Configure min and max servo pulse lengths
servo_min = 375  #TWM was 150  Min pulse length out of 4096
servo_max = 400  #TWM was 600 Max pulse length out of 4096
az_servo_min = 350
az_servo_max = 450

# Helper function to make setting a servo pulse width simpler.
def set_servo_pulse(channel, pulse):
    pulse_length = 1000000    # 1,000,000 us per second
    pulse_length //= 60       # 60 Hz
    print('{0}us per period'.format(pulse_length))
    pulse_length //= 4096     # 12 bits of resolution
    print('{0}us per bit'.format(pulse_length))
    pulse *= 1000
    pulse //= pulse_length
    pwm.set_pwm(channel, 0, pulse)

# Set frequency to 60hz, good for servos.
pwm.set_pwm_freq(60)

print('Moving servo on channel 0, press Ctrl-C to quit...')
while True:
    # Move servo on channel O between extremes.
    pwm.set_pwm(0, 0, servo_min) #elevation
    pwm.set_pwm(1, 0, az_servo_min) #azimuth
    time.sleep(5)
    pwm.set_pwm(0, 0, servo_max)
    pwm.set_pwm(1, 0, az_servo_max)
    time.sleep(5)

from __future__ import print_function # WalabotAPI works on both Python 2 an 3.
from __future__ import division
from sys import platform
from os import system
from imp import load_source
from os.path import join
# merging stuff learned from Medicine Reminder project
#import paho.mqtt.client as mqtt
import json, time, sys, ssl
import logging, logging.handlers
#import RPi.GPIO as GPIO #GPIO is not exactly what we want for I2C
# Import the PCA9685 module.
import Adafruit_PCA9685
import math

if platform == 'win32':
	modulePath = join('C:/', 'Program Files', 'Walabot', 'WalabotSDK',
		'python', 'WalabotAPI.py')
elif platform.startswith('linux'):
    modulePath = join('/usr', 'share', 'walabot', 'python', 'WalabotAPI.py')     

wlbt = load_source('WalabotAPI', modulePath)
wlbt.Init()

def PrintSensorTargets(targets):
    system('cls' if platform == 'win32' else 'clear')
    if targets:
        for i, target in enumerate(targets):
            print('Target #{}:\nx: {}\ny: {}\nz: {}\namplitude: {}\n'.format(
                i + 1, target.xPosCm, target.yPosCm, target.zPosCm,
                target.amplitude))
    else:
        print('No Target Detected')

def RobotSensorApp():
    deadband = math.pi * (6/180)
    # Initialise the PCA9685 using the default address (0x40).
    pwm = Adafruit_PCA9685.PCA9685()
    # Configure min and max servo pulse lengths
    el_servo_min = 350 #320  #25 degrees inclined down
    el_servo_max = 420  #25 degrees inclined up
    el_servo_ctr = 370 #level
    el_servo_cmd = el_servo_ctr #start from center position
    az_servo_min = 270 #ahead
    az_servo_max = 455 #45 degrees to the right
    az_servo_ctr = 365 #45 degrees to the left
    az_servo_cmd = az_servo_ctr #start from center position
    
    # Set frequency to 60hz, good for servos.
    pwm.set_pwm_freq(60)
    pwm.set_pwm(0, 0, el_servo_ctr) #elevation
    pwm.set_pwm(1, 0, az_servo_ctr) #azimuth
       
    # wlbt.SetArenaR - input parameters
    minInCm, maxInCm, resInCm = 30, 200, 3
    # wlbt.SetArenaTheta - input parameters
    minIndegrees, maxIndegrees, resIndegrees = -15, 15, 5
    # wlbt.SetArenaPhi - input parameters
    minPhiInDegrees, maxPhiInDegrees, resPhiInDegrees = -60, 60, 5
    # Set MTI mode
    mtiMode = False
    # Configure Walabot database install location (for windows)
    wlbt.SetSettingsFolder()
    # 1) Connect : Establish communication with walabot.
    wlbt.ConnectAny()
    # 2) Configure: Set scan profile and arena
    # Set Profile - to Sensor.
    wlbt.SetProfile(wlbt.PROF_SENSOR)
    # Setup arena - specify it by Cartesian coordinates.
    wlbt.SetArenaR(minInCm, maxInCm, resInCm)
    # Sets polar range and resolution of arena (parameters in degrees).
    wlbt.SetArenaTheta(minIndegrees, maxIndegrees, resIndegrees)
    # Sets azimuth range and resolution of arena.(parameters in degrees).
    wlbt.SetArenaPhi(minPhiInDegrees, maxPhiInDegrees, resPhiInDegrees)
    # Moving Target Identification: standard dynamic-imaging filter
    filterType = wlbt.FILTER_TYPE_MTI if mtiMode else wlbt.FILTER_TYPE_NONE
    wlbt.SetDynamicImageFilter(filterType)
    # 3) Start: Start the system in preparation for scanning.
    wlbt.Start()
    if not mtiMode: # if MTI mode is not set - start calibrartion
        # calibrates scanning to ignore or reduce the signals
        wlbt.StartCalibration()
        while wlbt.GetStatus()[0] == wlbt.STATUS_CALIBRATING:
            wlbt.Trigger()
        x_ang = 0
        y_ang = 0
    while True:
        appStatus, calibrationProcess = wlbt.GetStatus()
        # 5) Trigger: Scan(sense) according to profile and record signals
        # to be available for processing and retrieval.
        wlbt.Trigger()

        for i in range(20):
                #move if not against a travel limit
                if x_ang > deadband:
                   if az_servo_cmd < az_servo_max:
                           az_servo_cmd+=1
                if x_ang < -deadband:
                   if az_servo_cmd > az_servo_min:
                           az_servo_cmd-=1
                if y_ang > deadband:
                   if el_servo_cmd < el_servo_max:
                           el_servo_cmd+=1
                if y_ang < -deadband:
                   if el_servo_cmd > el_servo_min:
                           el_servo_cmd-=1
                #print(az_servo_cmd)            
                pwm.set_pwm(0, 0, el_servo_cmd) #elevation
                pwm.set_pwm(1, 0, az_servo_cmd) #azimuth
                time.sleep(0.1)

        
        # 6) Get action: retrieve the last completed triggered recording
        targets = wlbt.GetSensorTargets()
        # rasterImage, _, _, sliceDepth, power = wlbt.GetRawImageSlice()
        # PrintSensorTargets(targets)
        #PrintSensorTargets(targets)
        # Move servos
        dist_sq = 10000000 #infinity
        x_ang = 0
        y_ang = 0
        if targets:
            for i, target in enumerate(targets):
                dist_calc_sq = target.xPosCm * target.xPosCm
                dist_calc_sq += target.yPosCm * target.yPosCm
                dist_calc_sq += target.zPosCm * target.zPosCm
                if dist_calc_sq <= dist_sq:
                    dist_sq = dist_calc_sq
                    x_ang = math.atan(target.xPosCm / target.zPosCm)
                    y_ang = math.atan(target.yPosCm / target.zPosCm)


    
    # 7) Stop and Disconnect.
    wlbt.Stop()
    wlbt.Disconnect()
    print('Terminate successfully')

if __name__ == '__main__':
    RobotSensorApp()