Hardware components | ||||||
| × | 2 | ||||
| × | 1 | ||||
| × | 1 | ||||
| × | 2 | ||||
Software apps and online services | ||||||
| ||||||
|
Handy is a concept of a hand on wheels. It is connected to an Amazon Echo device to enable service on voice command.
It can wave, pick up coloured objects, and you can even teleoperate it to reach you!
Handy is built using 2 Lego Mindstorms EV3 kits and several Lego Technic parts ( Building Instructions included ) and uses 6 motors and a colour sensor.
Handy is registered as an Alexa Gadget. Both the EV3 bricks are connected under the name of Handy. Then, an Alexa skill commands Handy to perform functions.
Mechanical ModelFollow the Lego Model Instructions given at the end of the project.
Setting up the EV3 brick environment- Follow the Setup guide given by the Lego Mindstorms Voice Challenge on both the EV3 bricks.
- Create a folder named
Handy
. Put the following files in the folder (code given at the end):
handy_arm.py
handy_arm.ini
handy_base.py
handy_base.ini
- Open VS Code and click Open Folder under Start and select the folder
Handy
that you previously created.
First EV3 brick:
The Finger Motor is connected to Output A.
The Thumb motor is connected to Output B.
The Wrist motor is connected to Output C.
The Elbow motor is connected to Output D.
Second EV3 brick:
The Left Tread Motor is connected to Output B.
The Right Tread Motor is connected to Output C.
The Colour Sensor is connected to Input C.
Making the Alexa Skill and Registering the Alexa GadgetRegistering the gadget- Sign in to developer.amazon.com and open the Developer Console.
- Hover over Alexa and select Alexa Voice Service option.
- Click on Products
- Click Create Product in the upper right section of the page.
- Fill out the following information:
Product Name: Handy
Product ID: Handy
Product Type: Alexa Gadget
Product Category: Animatronic or Figure
Product Description: Handy is basically a hand on wheels. It can wave, pick up coloured objects, and you can even teleoperate it to reach you.
How will end users interact with your product?: Hands Free
Skip Upload Image
Do you intend to distribute this product commercially?: No
Is this a children’s product or is it otherwise directed to children younger than 13 years old?: No
- Accept the services agreement and click Finish.
- You will be taken to a list of products. Click on Handy.
- Note the Amazon Id and Alexa Gadget Secret and enter the Amazon Id and Alexa Gadget Secret in the respective fields in
handy_arm.ini
andhandy_base.ini
- Now, you can download the workspace to the EV3 bricks:
- Connect the 1st EV3 brick to your computer with the usb cable given in the set.
- Select Click here to connect a device under the EV3DEV DEVICE BROWSER and select ev3dev.
- Download the workspace to the EV3 brick.
- Right click on ev3dev and click Disconnect.
- Follow the same steps for the 2nd brick.
- Open developer.amazon.com and and open the Developer Console.
- Hover over Alexa and select the Alexa Skills Kit option.
- Click Create Skill.
- Fill out the following information:
Skill Name: Handy
Default Language: (Select the language of your region. For eg. if you live in Australia, select English (AU) )
Choose a model to add to your skill: Custom
Choose a method to host your skill's backend resources: Alexa-Hosted (Node.js)
- Click Create Skill.
- Choose the Hello World Skill option in the Choose a template to add to your skill option and click Choose.
- The Build section of the skill appears.
- Click Interfaces from the sidebar on the left and enable Custom Interface Controller.
- Scroll to the top and click Save Interfaces.
- To define the skill interaction model, click on the JSON Editor under Interaction Model.
- Drag and drop the
model.json
file from the project code in the Drag and drop a.json file section. - Click Save Model and then Build Model. This will take some time.
- After building the model, head over to the Code section.
- Open the
skill-index.js
file. - Copy the code of the
skill-index.js
file in theindex.js
file in the Code Editor. - Copy the contents of
skill-util.js
andskill-package.json
files similarly. - Right click on lambda and select Create File.
- Enter the File Path as
lambda/common.js
- Copy the contents of
skill-common.js
in the Code Editor. - After copying all the files, click Deploy.
- Now, you can test the skill by heading over to the Test section of the skill.
- Switch the testing from Off to Development.
- In the left panel, type or press the microphone and say "open handy" and if the response is: "I couldn't find handy. Please check to make sure handy is connected, and try again." then it means that the skill is successfully created. Hooray!!
- On the EV3 bricks run the programs,
handy_base.py
on the first brick andhandy_arm.py
on the second brick:
- You should hear a tone on both the EV3 bricks which means that the program is successfully running.
- Both the EV3 bricks will show the gadget's friendly name which will look like
Gadget000
which means that the gadget is in pairing mode. - Note the gadget number of both the EV3 bricks.
- Open the Alexa App on your phone and Select Devices
- Select Echo & Alexa.
- Select the Echo device you want to pair with your Alexa Gadget.
- Select Pair Alexa Gadget, and then select the First EV3 brick's friendly name and pair.
- Repeat the steps for the second brick.
Now, you can say the following commands to make handy perform functions:
Alexa, say hi
Alexa, set speed 40 percent
Alexa, move forward 2 seconds
Alexa, move elbow down 90 degrees
Alexa, pick
Alexa, drop
Alexa, give me the blue basket
WARNING: The elbow motor can only move 90 degrees up and down. So, be careful while voice commanding Handy.Inventory System
Follow the instructions given at the end of the project to make the carrier basket of the boxes:
There has to be a one beam distance between two carriers.
The boxes can be made with the instructions given at the end of the project or you can create your own by checking the height that the coloured tile needs in order to be detected by the colour sensor:
Both the EV3 bricks are connected to the Echo device via bluetooth.
When the command which requires the use of both the EV3 bricks is given by the user to Alexa, Alexa sends the directive to the respective EV3 brick. Then, the EV3 brick sends an event to Alexa, who analyses it and then sends a directive to the base which performs the given task.
ConclusionThe project has taught us how to create and use Alexa Gadgets and Skills. It has led to a new learning of Python Coding on EV3 brick which unlocks even more possibilities with Lego Mindstorms EV3 kit. The voice based communication enhances the interaction between the robot and the user.
Overall, the project inspired us to create more Lego-Alexa Projects and is a product of our hardwork, patience, and creative thinking.
#!/usr/bin/env python3
import os
import sys
import time
import logging
import json
import random
import threading
from enum import Enum
from agt import AlexaGadget
from ev3dev2.led import Leds
from ev3dev2.sound import Sound
from ev3dev2.sensor.lego import TouchSensor
from ev3dev2.motor import OUTPUT_A, OUTPUT_B, OUTPUT_C, OUTPUT_D, MediumMotor, LargeMotor
from time import sleep
# Set the logging level to INFO to see messages from AlexaGadget
logging.basicConfig(level=logging.INFO, stream=sys.stdout, format='%(message)s')
logging.getLogger().addHandler(logging.StreamHandler(sys.stderr))
logger = logging.getLogger(__name__)
class MindstormsGadget(AlexaGadget):
"""
A Mindstorms gadget that performs movement based on voice commands.
Two types of commands are supported, directional movement and preset.
"""
def __init__(self):
"""
Performs Alexa Gadget initialization routines and ev3dev resource allocation.
"""
super().__init__()
# Gadget state
# Ev3dev initialization
self.leds = Leds()
self.sound = Sound()
self.fingers = MediumMotor(OUTPUT_A)
self.thumb = MediumMotor(OUTPUT_B)
self.wrist = LargeMotor(OUTPUT_C)
self.elbow = LargeMotor(OUTPUT_D)
def on_connected(self, device_addr):
"""
Gadget connected to the paired Echo device.
:param device_addr: the address of the device we connected to
"""
self.leds.set_color("LEFT", "GREEN")
self.leds.set_color("RIGHT", "GREEN")
logger.info("{} connected to Echo device".format(self.friendly_name))
def on_disconnected(self, device_addr):
"""
Gadget disconnected from the paired Echo device.
:param device_addr: the address of the device we disconnected from
"""
self.leds.set_color("LEFT", "RED")
self.leds.set_color("RIGHT", "RED")
logger.info("{} disconnected from Echo device".format(self.friendly_name))
def on_custom_mindstorms_gadget_control(self, directive):
"""
Handles the Custom.Mindstorms.Gadget control directive.
:param directive: the custom directive with the matching namespace and name
"""
try:
payload = json.loads(directive.payload.decode("utf-8"))
print("Control payload: {}".format(payload), file=sys.stderr)
control_type = payload["type"]
if control_type == "wave":
self._wave()
elif control_type == "pick":
self._pick()
elif control_type == "move_elbow":
self._move_elbow(payload['degrees'], payload['up_down'])
elif control_type == "drop":
self._drop()
elif control_type == "color_found":
self._pick_2()
except:
pass
def on_custom_mindstorms_gadget_communication_communication(self, directive):
"""
Handles the Custom.Mindstorms.Gadget control directive.
:param directive: the custom directive with the matching namespace and name
"""
try:
payload = json.loads(directive.payload.decode("utf-8"))
print("Comm payload: {}".format(payload), file=sys.stderr)
control_type = payload["type"]
if control_type == "ColorFound":
self._pick_2()
except KeyError:
print("Missing expected parameters: {}".format(directive), file=sys.stderr)
def _wave(self):
for i in range(1,2):
self.wrist.on_for_degrees(75, 90)
self.wrist.on_for_degrees(75, -180)
self.wrist.on_for_degrees(75, 90)
def _pick(self):
self.thumb.on_for_degrees(75, -150)
self.fingers.on_for_degrees(75, 315)
def _drop(self):
self.fingers.on_for_degrees(75, -315)
sleep(1)
self.thumb.on_for_degrees(75, 150)
def _move_elbow(self,degrees, up_down):
if up_down == "Up" or up_down == "up":
self.elbow.on_for_degrees(75, 12 * int(degrees))
if up_down == "Down" or up_down == "down":
self.elbow.on_for_degrees(-75, 12 * int(degrees))
def _pick_2(self):
self.elbow.on_for_degrees(-75,1080)
self.thumb.on_for_degrees(75, -150)
self.fingers.on_for_degrees(75, 315)
def _send_event(self, name, payload):
"""
Sends a custom event to trigger a sentry action.
:param name: the name of the custom event
:param payload: the sentry JSON payload
"""
self.send_custom_event('Custom.Mindstorms.Gadget',name, payload)
if __name__ == '__main__':
gadget = MindstormsGadget()
# Set LCD font and turn off blinking LEDs
os.system('setfont Lat7-Terminus12x6')
gadget.leds.set_color("LEFT", "BLACK")
gadget.leds.set_color("RIGHT", "BLACK")
# Startup sequence
gadget.sound.play_song((('C4', 'e'), ('D4', 'e'), ('E5', 'q')))
gadget.leds.set_color("LEFT", "GREEN")
gadget.leds.set_color("RIGHT", "GREEN")
# Gadget main entry point
gadget.main()
# Shutdown sequence
gadget.sound.play_song((('E5', 'e'), ('C4', 'e')))
gadget.leds.set_color("LEFT", "BLACK")
gadget.leds.set_color("RIGHT", "BLACK")
[GadgetSettings]
amazonId = YOUR_GADGET_AMAZON_ID
alexaGadgetSecret = YOUR_GADGET_SECRET
[GadgetCapabilities]
Custom.Mindstorms.Gadget = 1.0
#!/usr/bin/env python3
import os
import sys
import time
import logging
import json
import random
import threading
from enum import Enum
from agt import AlexaGadget
from ev3dev2.led import Leds
from ev3dev2.sound import Sound
from ev3dev2.sensor.lego import ColorSensor
from ev3dev2.motor import OUTPUT_A, OUTPUT_B, OUTPUT_C, MoveTank, SpeedPercent, MediumMotor
# Set the logging level to INFO to see messages from AlexaGadget
logging.basicConfig(level=logging.INFO, stream=sys.stdout, format='%(message)s')
logging.getLogger().addHandler(logging.StreamHandler(sys.stderr))
logger = logging.getLogger(__name__)
class Direction(Enum):
"""
The list of directional commands and their variations.
These variations correspond to the skill slot values.
"""
FORWARD = ['forward', 'forwards', 'go forward']
BACKWARD = ['back', 'backward', 'backwards', 'go backward']
LEFT = ['left', 'go left']
RIGHT = ['right', 'go right']
STOP = ['stop', 'brake']
class MindstormsGadget(AlexaGadget):
"""
A Mindstorms gadget that performs movement based on voice commands.
Two types of commands are supported, directional movement and preset.
"""
def __init__(self):
"""
Performs Alexa Gadget initialization routines and ev3dev resource allocation.
"""
super().__init__()
# Gadget state
# Ev3dev initialization
self.leds = Leds()
self.sound = Sound()
self.color_sensor = ColorSensor()
self.drive = MoveTank(OUTPUT_B, OUTPUT_C)
def on_connected(self, device_addr):
"""
Gadget connected to the paired Echo device.
:param device_addr: the address of the device we connected to
"""
self.leds.set_color("LEFT", "GREEN")
self.leds.set_color("RIGHT", "GREEN")
logger.info("{} connected to Echo device".format(self.friendly_name))
def on_disconnected(self, device_addr):
"""
Gadget disconnected from the paired Echo device.
:param device_addr: the address of the device we disconnected from
"""
self.leds.set_color("LEFT", "RED")
self.leds.set_color("RIGHT", "RED")
logger.info("{} disconnected from Echo device".format(self.friendly_name))
def on_custom_mindstorms_gadget_control(self, directive):
"""
Handles the Custom.Mindstorms.Gadget control directive.
:param directive: the custom directive with the matching namespace and name
"""
try:
payload = json.loads(directive.payload.decode("utf-8"))
print("Control payload: {}".format(payload), file=sys.stderr)
control_type = payload["type"]
if control_type == "move":
self._move(payload['direction'], int(payload['duration']), int(payload['speed']))
if control_type == "pick_color":
self._pick_color(payload['color'])
except KeyError:
print("Missing expected parameters: {}".format(directive), file=sys.stderr)
def _send_event(self, name, payload):
"""
Sends a custom event to trigger a sentry action.
:param name: the name of the custom event
:param payload: the sentry JSON payload
"""
self.send_custom_event('Custom.Mindstorms.Gadget', name, payload)
def _move(self, direction, duration: int, speed: int):
if direction in Direction.FORWARD.value:
self.drive.on_for_seconds(SpeedPercent(speed), SpeedPercent(speed), duration)
if direction in Direction.BACKWARD.value:
self.drive.on_for_seconds(SpeedPercent(-speed), SpeedPercent(-speed), duration)
if direction in (Direction.RIGHT.value + Direction.LEFT.value):
self._turn(direction, speed)
self.drive.on_for_seconds(SpeedPercent(speed), SpeedPercent(speed), duration)
if direction in Direction.STOP.value:
self.drive.off()
def _turn(self, direction, speed):
"""
Turns based on the specified direction and speed.
Calibrated for hard smooth surface.
:param direction: the turn direction
:param speed: the turn speed
"""
if direction in Direction.LEFT.value:
self.drive.on_for_rotations(-speed, speed, 1.45)
if direction in Direction.RIGHT.value:
self.drive.on_for_rotations(speed, -speed, 1.45)
def _pick_color(self, required_color_name):
color_name = self.color_sensor.color_name
while color_name.lower() != "NoColor".lower():
if color_name.lower() == required_color_name.lower():
self.drive.on_for_rotations(-75, 75, 1.45)
self.drive.on_for_rotations(-75, -75, 0.8)
self._send_event("color_found", {})
else:
self.drive.on_for_rotations(75, 75, 1)
color_name = self.color_sensor.color_name
if __name__ == '__main__':
gadget = MindstormsGadget()
# Set LCD font and turn off blinking LEDs
os.system('setfont Lat7-Terminus12x6')
gadget.leds.set_color("LEFT", "BLACK")
gadget.leds.set_color("RIGHT", "BLACK")
# Startup sequence
gadget.sound.play_song((('C4', 'e'), ('D4', 'e'), ('E5', 'q')))
gadget.leds.set_color("LEFT", "GREEN")
gadget.leds.set_color("RIGHT", "GREEN")
# Gadget main entry point
gadget.main()
# Shutdown sequence
gadget.sound.play_song((('E5', 'e'), ('C4', 'e')))
gadget.leds.set_color("LEFT", "BLACK")
gadget.leds.set_color("RIGHT", "BLACK")
[GadgetSettings]
amazonId = YOUR_GADGET_AMAZON_ID
alexaGadgetSecret = YOUR_GADGET_SECRET
[GadgetCapabilities]
Custom.Mindstorms.Gadget = 1.0
{
"interactionModel": {
"languageModel": {
"invocationName": "handy",
"intents": [
{
"name": "AMAZON.CancelIntent",
"samples": []
},
{
"name": "AMAZON.HelpIntent",
"samples": []
},
{
"name": "AMAZON.StopIntent",
"samples": []
},
{
"name": "AMAZON.NavigateHomeIntent",
"samples": []
},
{
"name": "MoveIntent",
"slots": [
{
"name": "Direction",
"type": "DirectionType"
},
{
"name": "Duration",
"type": "AMAZON.NUMBER"
}
],
"samples": [
"{Direction} now",
"{Direction} {Duration} seconds",
"move {Direction} for {Duration} seconds"
]
},
{
"name": "SetSpeedIntent",
"slots": [
{
"name": "Speed",
"type": "AMAZON.NUMBER"
}
],
"samples": [
"set speed {Speed} percent",
"set {Speed} percent speed",
"set speed to {Speed} percent"
]
},
{
"name": "PickIntent",
"slots": [],
"samples": [
"activate pick command",
"pick"
]
},
{
"name": "PickColorIntent",
"slots": [
{
"name": "Color",
"type": "ColorType"
}
],
"samples": [
"pick {Color} basket",
"bring me the {Color} basket",
"hand me that {Color} basket",
"give me the {Color} basket",
"pick up {Color} basket",
"fetch me the {Color} box",
"bring me the {Color} box",
"hand me that {Color} box",
"give me the {Color} box",
"pick up {Color} box ",
"pick {Color} box"
]
},
{
"name": "DropIntent",
"slots": [],
"samples": [
"drop that basket",
"put that down",
"drop that box",
"drop "
]
},
{
"name": "WaveIntent",
"slots": [
{
"name": "Greet",
"type": "GreetType"
}
],
"samples": [
"wish me luck",
"say {Greet}",
"wave"
]
},
{
"name": "MoveElbowIntent",
"slots": [
{
"name": "Degrees",
"type": "AMAZON.NUMBER"
},
{
"name": "UpDown",
"type": "UpDownType"
}
],
"samples": [
"move elbow {Degrees} degrees {UpDown} ",
"elbow {UpDown} {Degrees} degrees",
"rotate elbow {Degrees} degrees {UpDown}",
"move elbow {UpDown} {Degrees} degrees ",
"move elbow {UpDown} {Degrees}"
]
}
],
"types": [
{
"name": "DirectionType",
"values": [
{
"name": {
"value": "brake"
}
},
{
"name": {
"value": "go backward"
}
},
{
"name": {
"value": "go forward"
}
},
{
"name": {
"value": "go right"
}
},
{
"name": {
"value": "go left"
}
},
{
"name": {
"value": "right"
}
},
{
"name": {
"value": "left"
}
},
{
"name": {
"value": "backwards"
}
},
{
"name": {
"value": "backward"
}
},
{
"name": {
"value": "forwards"
}
},
{
"name": {
"value": "forward"
}
}
]
},
{
"name": "ColorType",
"values": [
{
"name": {
"value": "Brown"
}
},
{
"name": {
"value": "White"
}
},
{
"name": {
"value": "Red"
}
},
{
"name": {
"value": "Yellow"
}
},
{
"name": {
"value": "Green"
}
},
{
"name": {
"value": "Blue"
}
},
{
"name": {
"value": "Black"
}
}
]
},
{
"name": "GreetType",
"values": [
{
"name": {
"value": "goodbye"
}
},
{
"name": {
"value": "bye-bye"
}
},
{
"name": {
"value": "hello"
}
},
{
"name": {
"value": "bye"
}
},
{
"name": {
"value": "hi"
}
}
]
},
{
"name": "UpDownType",
"values": [
{
"name": {
"value": "down"
}
},
{
"name": {
"value": "up"
}
}
]
}
]
}
}
}
const Alexa = require('ask-sdk-core');
const Util = require('./util');
const Common = require('./common');
// The namespace of the custom directive to be sent by this skill
const NAMESPACE = 'Custom.Mindstorms.Gadget';
// The name of the custom directive to be sent this skill
const NAME_CONTROL = 'control';
const LaunchRequestHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'LaunchRequest';
},
handle: async function(handlerInput) {
let request = handlerInput.requestEnvelope;
let { apiEndpoint, apiAccessToken } = request.context.System;
let apiResponse = await Util.getConnectedEndpoints(apiEndpoint, apiAccessToken);
if ((apiResponse.endpoints || []).length === 0) {
return handlerInput.responseBuilder
.speak(`I couldn't find handy. Please check to make sure handy is connected, and try again.`)
.getResponse();
}
// Store the gadget endpointId to be used in this skill session
let endpointId = apiResponse.endpoints[0].endpointId || [];
let endpointId2 = apiResponse.endpoints[1].endpointId || [];
Util.putSessionAttribute(handlerInput, 'handy_arm', endpointId);
Util.putSessionAttribute(handlerInput, 'handy_base', endpointId2);
// Set skill duration to 5 minutes (ten 30-seconds interval)
Util.putSessionAttribute(handlerInput, 'duration', 10);
// Set the token to track the event handler
const token = handlerInput.requestEnvelope.request.requestId;
Util.putSessionAttribute(handlerInput, 'token', token);
return handlerInput.responseBuilder
.speak("This is handy. How may i help you?")
.reprompt("Awaiting commands")
.addDirective(Util.buildStartEventHandler(token,60000, {}))
.getResponse();
}
};
// Add the speed value to the session attribute.
// This allows other intent handler to use the specified speed value
// without asking the user for input.
const SetSpeedIntentHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
&& Alexa.getIntentName(handlerInput.requestEnvelope) === 'SetSpeedIntent';
},
handle: function (handlerInput) {
// Bound speed to (1-100)
let speed = Alexa.getSlotValue(handlerInput.requestEnvelope, 'Speed');
speed = Math.max(1, Math.min(100, parseInt(speed)));
Util.putSessionAttribute(handlerInput, 'speed', speed);
return handlerInput.responseBuilder
.speak(`speed set to ${speed} percent.`)
.getResponse();
}
};
// Construct and send a custom directive to the connected gadget with
// data from the MoveIntent request.
const MoveIntentHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
&& Alexa.getIntentName(handlerInput.requestEnvelope) === 'MoveIntent';
},
handle: function (handlerInput) {
const request = handlerInput.requestEnvelope;
const direction = Alexa.getSlotValue(request, 'Direction');
// Duration is optional, use default if not available
const duration = Alexa.getSlotValue(request, 'Duration') || "2";
// Get data from session attribute
const attributesManager = handlerInput.attributesManager;
const speed = attributesManager.getSessionAttributes().speed || "50";
const handy_base = attributesManager.getSessionAttributes().handy_base || [];
// Construct the directive with the payload containing the move parameters
const directive = Util.build(handy_base, NAMESPACE, NAME_CONTROL,
{
type: 'move',
direction: direction,
duration: duration,
speed: speed
});
const speechOutput = (direction === "brake")
? "Applying brake"
: `${direction} ${duration} seconds at ${speed} percent speed`;
return handlerInput.responseBuilder
.speak(speechOutput)
.addDirective(directive)
.getResponse();
}
};
const MoveElbowIntentHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
&& Alexa.getIntentName(handlerInput.requestEnvelope) === 'MoveElbowIntent';
},
handle: function (handlerInput) {
const request = handlerInput.requestEnvelope;
// Duration is optional, use default if not available
const degrees = Math.max(1, Math.min(90, parseInt(Alexa.getSlotValue(request, 'Degrees')))) || "2";
const up_down = Alexa.getSlotValue(request, 'UpDown');
// Get data from session attribute
const attributesManager = handlerInput.attributesManager;
const handy_arm = attributesManager.getSessionAttributes().handy_arm || [];
// Construct the directive with the payload containing the move parameters
const directive = Util.build(handy_arm, NAMESPACE, NAME_CONTROL,
{
type: 'move_elbow',
degrees: degrees,
up_down: up_down
});
const speechOutput = `Moving elbow ${up_down} ${degrees} degrees`;
return handlerInput.responseBuilder
.speak(speechOutput)
.addDirective(directive)
.getResponse();
}
};
const PickIntentHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
&& Alexa.getIntentName(handlerInput.requestEnvelope) === 'PickIntent';
},
handle: function (handlerInput) {
const request = handlerInput.requestEnvelope;
// Get data from session attribute
const attributesManager = handlerInput.attributesManager;
const handy_arm = attributesManager.getSessionAttributes().handy_arm || [];
// Construct the directive with the payload containing the move parameters
const directive = Util.build(handy_arm, NAMESPACE, NAME_CONTROL,
{
type: 'pick'
});
return handlerInput.responseBuilder
.speak("picking object")
.addDirective(directive)
.getResponse();
}
};
const WaveIntentHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
&& Alexa.getIntentName(handlerInput.requestEnvelope) === 'WaveIntent';
},
handle: function (handlerInput) {
const request = handlerInput.requestEnvelope;
const greet = Alexa.getSlotValue(request, 'Greet') || "hi";
// Get data from session attribute
const attributesManager = handlerInput.attributesManager;
const handy_arm = attributesManager.getSessionAttributes().handy_arm || [];
// Construct the directive with the payload containing the move parameters
const directive = Util.build(handy_arm, NAMESPACE, NAME_CONTROL,
{
type: 'wave'
});
return handlerInput.responseBuilder
.speak(`${greet}`)
.addDirective(directive)
.getResponse();
}
};
const DropIntentHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
&& Alexa.getIntentName(handlerInput.requestEnvelope) === 'DropIntent';
},
handle: function (handlerInput) {
const request = handlerInput.requestEnvelope;
// Get data from session attribute
const attributesManager = handlerInput.attributesManager;
const handy_arm = attributesManager.getSessionAttributes().handy_arm || [];
// Construct the directive with the payload containing the move parameters
const directive = Util.build(handy_arm, NAMESPACE, NAME_CONTROL,
{
type: 'drop'
});
return handlerInput.responseBuilder
.speak("dropping object")
.addDirective(directive)
.getResponse();
}
};
// Construct and send a custom directive to the connected gadget with data from
// the SetCommandIntent request.
const PickColorIntentHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
&& Alexa.getIntentName(handlerInput.requestEnvelope) === 'PickColorIntent';
},
handle: function (handlerInput) {
let color = Alexa.getSlotValue(handlerInput.requestEnvelope, 'Color');
const attributesManager = handlerInput.attributesManager;
let handy_base = attributesManager.getSessionAttributes().handy_base || [];
let handy_arm = attributesManager.getSessionAttributes().handy_arm || [];
// Construct the directive with the payload containing the move parameters
let directive = Util.build(handy_base, NAMESPACE, NAME_CONTROL,
{
type: 'pick_color',
color: color
});
return handlerInput.responseBuilder
.speak(`Finding the ${color} box`)
.addDirective(directive)
.getResponse();
}
};
const EventsReceivedRequestHandler = {
// Checks for a valid token and endpoint.
canHandle(handlerInput) {
let { request } = handlerInput.requestEnvelope;
console.log('Request type: ' + Alexa.getRequestType(handlerInput.requestEnvelope));
if (request.type !== 'CustomInterfaceController.EventsReceived') return false;
const attributesManager = handlerInput.attributesManager;
let sessionAttributes = attributesManager.getSessionAttributes();
let customEvent = request.events[0];
// Validate event token
if (sessionAttributes.token !== request.token) {
console.log("Event token doesn't match. Ignoring this event");
return false;
}
// Validate endpoint
let requestEndpoint = customEvent.endpoint.endpointId;
if (requestEndpoint !== sessionAttributes.handy_base) {
console.log("Event endpoint id doesn't match. Ignoring this event");
return false;
}
return true;
},
handle(handlerInput) {
const attributesManager = handlerInput.attributesManager;
let handy_arm = attributesManager.getSessionAttributes().handy_arm || [];
console.log("== Received Custom Event ==");
let customEvent = handlerInput.requestEnvelope.request.events[0];
let payload = customEvent.payload;
let name = customEvent.header.name;
let directive;
let speechOutput = ""
if (name === 'color_found') {
directive = Util.build(handy_arm, NAMESPACE, NAME_CONTROL,
{
type: 'color_found'
});
speechOutput = "Picking Box"
}
return handlerInput.responseBuilder
.speak(speechOutput)
.addDirective(directive)
.getResponse();
}
};
const ExpiredRequestHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'CustomInterfaceController.Expired'
},
handle(handlerInput) {
console.log("== Custom Event Expiration Input ==");
// Set the token to track the event handler
const token = handlerInput.requestEnvelope.request.requestId;
Util.putSessionAttribute(handlerInput, 'token', token);
const attributesManager = handlerInput.attributesManager;
let duration = attributesManager.getSessionAttributes().duration || 0;
if (duration > 0) {
Util.putSessionAttribute(handlerInput, 'duration', --duration);
// Extends skill session
const speechOutput = `${duration} minutes remaining.`;
return handlerInput.responseBuilder
.addDirective(Util.buildStartEventHandler(token, 60000, {}))
.speak(speechOutput)
.getResponse();
}
else {
// End skill session
return handlerInput.responseBuilder
.speak("Skill duration expired. Goodbye.")
.withShouldEndSession(true)
.getResponse();
}
}
};
// The SkillBuilder acts as the entry point for your skill, routing all request and response
// payloads to the handlers above. Make sure any new handlers or interceptors you've
// defined are included below. The order matters - they're processed top to bottom.
exports.handler = Alexa.SkillBuilders.custom()
.addRequestHandlers(
LaunchRequestHandler,
SetSpeedIntentHandler,
MoveIntentHandler,
MoveElbowIntentHandler,
PickIntentHandler,
WaveIntentHandler,
DropIntentHandler,
PickColorIntentHandler,
EventsReceivedRequestHandler,
ExpiredRequestHandler,
Common.HelpIntentHandler,
Common.CancelAndStopIntentHandler,
Common.SessionEndedRequestHandler,
Common.IntentReflectorHandler, // make sure IntentReflectorHandler is last so it doesn't override your custom intent handlers
)
.addRequestInterceptors(Common.RequestInterceptor)
.addErrorHandlers(
Common.ErrorHandler,
)
.lambda();
'use strict'
const Alexa = require('ask-sdk-core');
const HelpIntentHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
&& Alexa.getIntentName(handlerInput.requestEnvelope) === 'AMAZON.HelpIntent';
},
handle(handlerInput) {
const speakOutput = 'You can command handy to move forward, pick, drop and more.';
return handlerInput.responseBuilder
.speak(speakOutput)
.reprompt(speakOutput)
.getResponse();
}
};
const CancelAndStopIntentHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
&& (Alexa.getIntentName(handlerInput.requestEnvelope) === 'AMAZON.CancelIntent'
|| Alexa.getIntentName(handlerInput.requestEnvelope) === 'AMAZON.StopIntent');
},
handle(handlerInput) {
const speakOutput = 'Goodbye!';
return handlerInput.responseBuilder
.speak(speakOutput)
.getResponse();
}
};
const SessionEndedRequestHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'SessionEndedRequest';
},
handle(handlerInput) {
// Any cleanup logic goes here.
return handlerInput.responseBuilder
.getResponse();
}
};
// The intent reflector is used for interaction model testing and debugging.
// It will simply repeat the intent the user said. You can create custom handlers
// for your intents by defining them above, then also adding them to the request
// handler chain below.
const IntentReflectorHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest';
},
handle(handlerInput) {
const intentName = Alexa.getIntentName(handlerInput.requestEnvelope);
const speakOutput = `You just triggered ${intentName}`;
return handlerInput.responseBuilder
.speak(speakOutput)
.reprompt("I don't understand this command, try again")
.getResponse();
}
};
// Generic error handling to capture any syntax or routing errors. If you receive an error
// stating the request handler chain is not found, you have not implemented a handler for
// the intent being invoked or included it in the skill builder below.
const ErrorHandler = {
canHandle() {
return true;
},
handle(handlerInput, error) {
console.log(`~~~~ Error handled: ${error.stack}`);
const speakOutput = `Sorry, I cannot understand what you are saying. Please try again`;
return handlerInput.responseBuilder
.speak(speakOutput)
.reprompt(speakOutput)
.getResponse();
}
};
// The request interceptor is used for request handling testing and debugging.
// It will simply log the request in raw json format before any processing is performed.
const RequestInterceptor = {
process(handlerInput) {
let { attributesManager, requestEnvelope } = handlerInput;
let sessionAttributes = attributesManager.getSessionAttributes();
// Log the request for debug purposes.
console.log(`=====Request==${JSON.stringify(requestEnvelope)}`);
console.log(`=========SessionAttributes==${JSON.stringify(sessionAttributes, null, 2)}`);
}
};
module.exports = {
HelpIntentHandler,
CancelAndStopIntentHandler,
SessionEndedRequestHandler,
IntentReflectorHandler,
ErrorHandler,
RequestInterceptor
};
/*
* Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
* You may not use this file except in compliance with the terms and conditions
* set forth in the accompanying LICENSE.TXT file.
*
* THESE MATERIALS ARE PROVIDED ON AN "AS IS" BASIS. AMAZON SPECIFICALLY DISCLAIMS, WITH
* RESPECT TO THESE MATERIALS, ALL WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING
* THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.
*/
'use strict';
const Https = require('https');
const AWS = require('aws-sdk');
const Escape = require('lodash/escape');
const s3SigV4Client = new AWS.S3({
signatureVersion: 'v4'
});
/**
* Get the authenticated URL to access the S3 Object. This URL expires after 60 seconds.
* @param s3ObjectKey - the S3 object key
* @returns {string} the pre-signed S3 URL
*/
exports.getS3PreSignedUrl = function getS3PreSignedUrl(s3ObjectKey) {
const bucketName = process.env.S3_PERSISTENCE_BUCKET;
return Escape(s3SigV4Client.getSignedUrl('getObject', {
Bucket: bucketName,
Key: s3ObjectKey,
Expires: 60 // the Expires is capped for 1 minute
}));
};
/**
* Builds a directive to start the EventHandler.
* @param token - a unique identifier to track the event handler
* @param {number} timeout - the duration to wait before sending back the expiration
* payload to the skill.
* @param payload - the expiration json payload
* @see {@link https://developer.amazon.com/docs/alexa-gadgets-toolkit/receive-custom-event-from-gadget.html#start}
*/
exports.buildStartEventHandler = function (token, timeout = 30000, payload) {
return {
type: "CustomInterfaceController.StartEventHandler",
token: token,
expiration : {
durationInMilliseconds: timeout,
expirationPayload: payload
}
};
};
/**
*
* Builds a directive to stops the active event handler.
* The event handler is identified by the cached token in the session attribute.
* @param {string} handlerInput - the context from Alexa Service
* @see {@link https://developer.amazon.com/docs/alexa-gadgets-toolkit/receive-custom-event-from-gadget.html#stop}
*/
exports.buildStopEventHandlerDirective = function (handlerInput) {
let token = handlerInput.attributesManager.getSessionAttributes().token || '';
return {
"type": "CustomInterfaceController.StopEventHandler",
"token": token
}
};
/**
* Build a custom directive payload to the gadget with the specified endpointId
* @param {string} endpointId - the gadget endpoint Id
* @param {string} namespace - the namespace of the skill
* @param {string} name - the name of the skill within the scope of this namespace
* @param {object} payload - the payload data
* @see {@link https://developer.amazon.com/docs/alexa-gadgets-toolkit/send-gadget-custom-directive-from-skill.html#respond}
*/
exports.build = function (endpointId, namespace, name, payload) {
// Construct the custom directive that needs to be sent
// Gadget should declare the capabilities in the discovery response to
// receive the directives under the following namespace.
return {
type: 'CustomInterfaceController.SendDirective',
header: {
name: name,
namespace: namespace
},
endpoint: {
endpointId: endpointId
},
payload
};
};
/**
* A convenience routine to add the a key-value pair to the session attribute.
* @param handlerInput - the context from Alexa Service
* @param key - the key to be added
* @param value - the value be added
*/
exports.putSessionAttribute = function(handlerInput, key, value) {
const attributesManager = handlerInput.attributesManager;
let sessionAttributes = attributesManager.getSessionAttributes();
sessionAttributes[key] = value;
attributesManager.setSessionAttributes(sessionAttributes);
};
/**
* To get a list of all the gadgets that meet these conditions,
* Call the Endpoint Enumeration API with the apiEndpoint and apiAccessToken to
* retrieve the list of all connected gadgets.
*
* @param {string} apiEndpoint - the Endpoint API url
* @param {string} apiAccessToken - the token from the session object in the Alexa request
* @see {@link https://developer.amazon.com/docs/alexa-gadgets-toolkit/send-gadget-custom-directive-from-skill.html#call-endpoint-enumeration-api}
*/
exports.getConnectedEndpoints = function(apiEndpoint, apiAccessToken) {
// The preceding https:// need to be stripped off before making the call
apiEndpoint = (apiEndpoint || '').replace('https://', '');
return new Promise(((resolve, reject) => {
const options = {
host: apiEndpoint,
path: '/v1/endpoints',
method: 'GET',
headers: {
'Content-Type': 'application/json',
'Authorization': 'Bearer ' + apiAccessToken
}
};
const request = Https.request(options, (response) => {
response.setEncoding('utf8');
let returnData = '';
response.on('data', (chunk) => {
returnData += chunk;
});
response.on('end', () => {
resolve(JSON.parse(returnData));
});
response.on('error', (error) => {
reject(error);
});
});
request.end();
}));
};
{
"name": "handy",
"version": "1.1.0",
"description": "The skill which can interact with handy and perform functions",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "Shreyas Jha and Sukriti Jha",
"license": "ISC",
"dependencies": {
"ask-sdk-core": "^2.6.0",
"ask-sdk-model": "^1.18.0",
"aws-sdk": "^2.326.0",
"request": "^2.81.0",
"lodash": "^4.17.11"
}
}
Shreyas Jha and Sukriti Jha
Comments