Hardware components | ||||||
| × | 1 | ||||
Software apps and online services | ||||||
| ||||||
| ||||||
|
"Be at peace on your vacation knowing your pet Parakeet is being well taken care of by Alexa and the LEGO Mindstorms Parakeet Babysitter!"
Our neighbors had a pet parakeet that they asked us to look after while they were gone for a few days. We thought it would be fine to go over once a day and feed and play with the parakeet. One day we took a day trip and thought we would be back within 24 hours. But it started to snow and we decided to stay away for a day longer. But what would happen with the parakeet! We looked up on the internet and sure enough, pet parakeets get sick and can possibly die if not fed after 24 hours. We rushed back home in a terrible stress, but we wished we had some way to check in on our friend's bird without being there!
The SolutionIn comes the Parakeet Babysitter! We thought that if we could automate a robot to take care of the pet parakeet (his name is Tony, and yes he survived our rush to get back to him, but just barely). So we went ahead and got to work programming a Mindstorms robot to work with Alexa Voice Commands to babysit Tony, feed him, watch him and even entertain him!
1. The EntertainerFirst we built a "Entertainer" Bot which taps on the cage while parakeet sounds are playing in the background. We noticed that when we tapped on the cage the bird would start to dance as well, so we thought this would be fun for Tony rather than being all alone in silence :(.
2. The FeederNext we wanted to build a device that could provide extra food for the parakeet so they could stock on food when we're gone, enough for three days worth of food! We built a container that would release whenever we told Alexa to "feed the parakeet"!
3. The CameramanThirdly we thought it would be helpful to be able to see the bird in case there was something wrong and they started looking ill. We knew there were some popular apps like Alfred Camera or ManyThing App but we also knew we would need to have the camera move around so we can see where the bird was in the cage (it's a big cage!). So we built a device to move the smartphone camera around left and right as well as up and down, controlled by Alexa Voice Control.
The Final Product - VideoNow watch a video of our final product demonstrated by our expert LEGO Mindstorms Builder David Golland:
#Acknowledgement: We have used elements of the Amazon Alexa Challenge code for Mission 3 and modified the code to adapt to our project.
import os
import sys
import time
import logging
import json
import random
import threading
from enum import Enum
from agt import AlexaGadget
from ev3dev2.led import Leds
from ev3dev2.sound import Sound
from ev3dev2.motor import OUTPUT_A, OUTPUT_B, OUTPUT_C, OUTPUT_D, Motor, LargeMotor, MoveTank, SpeedPercent, MediumMotor
# Set the logging level to INFO to see messages from AlexaGadget
logging.basicConfig(level=logging.INFO, stream=sys.stdout, format='%(message)s')
logging.getLogger().addHandler(logging.StreamHandler(sys.stderr))
logger = logging.getLogger(__name__)
class Direction(Enum):
"""
The list of directional commands and their variations.
These variations correspond to the skill slot values.
"""
FORWARD = ['forward', 'forwards', 'go forward']
BACK = ['back', 'backward', 'backwards', 'go backward']
LEFT = ['left', 'turn left']
RIGHT = ['right', 'turn right']
STOP = ['stop', 'brake']
class Command(Enum):
"""
The list of preset commands and their invocation variation.
These variations correspond to the skill slot values.
"""
MOVE_CIRCLE = ['circle', 'spin']
MOVE_SQUARE = ['square']
PATROL = ['patrol', 'guard mode', 'sentry mode']
FIRE_ONE = ['cannon', '1 shot', 'one shot']
FIRE_ALL = ['all shot']
class MindstormsGadget(AlexaGadget):
"""
A Mindstorms gadget that performs movement based on voice commands.
Two types of commands are supported, directional movement and preset.
"""
def __init__(self):
"""
Performs Alexa Gadget initialization routines and ev3dev resource allocation.
"""
super().__init__()
# Gadget state
self.patrol_mode = False
# Ev3dev initialization
self.leds = Leds()
self.sound = Sound()
self.survey1 = LargeMotor(OUTPUT_B)
self.survey2 = LargeMotor(OUTPUT_C)
self.entertain = MediumMotor(OUTPUT_A)
self.feed = LargeMotor(OUTPUT_D)
# Start threads
threading.Thread(target=self._patrol_thread, daemon=True).start()
def on_connected(self, device_addr):
"""
Gadget connected to the paired Echo device.
:param device_addr: the address of the device we connected to
"""
self.leds.set_color("LEFT", "YELLOW")
self.leds.set_color("RIGHT", "AMBER")
logger.info("{} connected to Echo device".format(self.friendly_name))
def on_disconnected(self, device_addr):
"""
Gadget disconnected from the paired Echo device.
:param device_addr: the address of the device we disconnected from
"""
self.leds.set_color("LEFT", "BLACK")
self.leds.set_color("RIGHT", "BLACK")
logger.info("{} disconnected from Echo device".format(self.friendly_name))
def on_custom_mindstorms_gadget_control(self, directive):
"""
Handles the Custom.Mindstorms.Gadget control directive.
:param directive: the custom directive with the matching namespace and name
"""
try:
payload = json.loads(directive.payload.decode("utf-8"))
print("Control payload: {}".format(payload), file=sys.stderr)
control_type = payload["type"]
if control_type == "move":
# Expected params: [direction, duration, speed]
self._move(payload["direction"], int(payload["duration"]), int(payload["speed"]))
if control_type == "command":
# Expected params: [command]
self._activate(payload["command"])
except KeyError:
print("Missing expected parameters: {}".format(directive), file=sys.stderr)
def _move(self, direction, duration, speed: int, is_blocking=False):
"""
Handles move commands from the directive.
Right and left movement can under or over turn depending on the surface type.
:param direction: the move direction
:param duration: the duration in seconds
:param speed: the speed percentage as an integer
:param is_blocking: if set, motor run until duration expired before accepting another command
"""
print("Move command: ({}, {}, {}, {})".format(direction, speed, duration, is_blocking), file=sys.stderr)
if direction in Direction.FORWARD.value:
self.entertain.on_for_seconds(SpeedPercent(20), SpeedPercent(20), duration, block=is_blocking)
self.feed.on_for_seconds(SpeedPercent(10), SpeedPercent(10), duration, block=is_blocking)
if direction in Direction.BACKWARD.value:
self.entertain.on_for_seconds(SpeedPercent(-20), SpeedPercent(-20), duration, block=is_blocking)
self.feed.on_for_seconds(SpeedPercent(10), SpeedPercent(10), duration, block=is_blocking)
if direction in (Direction.RIGHT.value + Direction.LEFT.value):
self._turn(70, speed)
self.entertain.on_for_seconds(SpeedPercent(20), SpeedPercent(20), duration, block=is_blocking)
if direction in Direction.STOP.value:
self.entertain.off()
self.feed.off()
self.patrol_mode = False
def _activate(self, command, speed=20):
"""
Handles preset commands.
:param command: the preset command
:param speed: the speed if applicable
"""
print("Activate command: ({}, {})".format(command, speed), file=sys.stderr)
if command in Command.MOVE_CIRCLE.value:
self.drive.on_for_seconds(SpeedPercent(int(speed)), SpeedPercent(5), 12)
if command in Command.MOVE_SQUARE.value:
for i in range(4):
self._move("right", 2, speed, is_blocking=True)
if command in Command.PATROL.value:
# Set patrol mode to resume patrol thread processing
self.patrol_mode = True
if command in Command.FIRE_ONE.value:
self.weapon.on_for_rotations(SpeedPercent(20), 3)
if command in Command.FIRE_ALL.value:
self.weapon.on_for_rotations(SpeedPercent(20), 10)
def _turn(self, direction, speed):
"""
Turns based on the specified direction and speed.
Calibrated for hard smooth surface.
:param direction: the turn direction
:param speed: the turn speed
"""
if direction in Direction.LEFT.value:
self.drive.on_for_seconds(SpeedPercent(0), SpeedPercent(speed), 2)
if direction in Direction.RIGHT.value:
self.drive.on_for_seconds(SpeedPercent(speed), SpeedPercent(0), 2)
if __name__ == '__main__':
gadget = MindstormsGadget()
# Set LCD font and turn off blinking LEDs
os.system('setfont Lat7-Terminus12x6')
gadget.leds.set_color("LEFT", "BLACK")
gadget.leds.set_color("RIGHT", "BLACK")
# Startup sequence
gadget.sound.play_song((('C4', 'e'), ('D4', 'e'), ('E4', 'q'),('C4', 'e'), ('C4', 'e'), ('D4', 'e'), ('E4', 'q'), ('C4', 'e'),('E4', 'e'), ('F4', 'e'), ('G4', 'q'),('E4', 'e'), ('F4', 'e'), ('G4', 'q'),))
gadget.leds.set_color("LEFT", "GREEN")
gadget.leds.set_color("RIGHT", "GREEN")
# Gadget main entry point
gadget.main()
# Shutdown sequence
gadget.sound.play_song((('E5', 'e'), ('C4', 'e')))
gadget.leds.set_color("LEFT", "BLACK")
gadget.leds.set_color("RIGHT", "BLACK")
LICENSE.TXT
Plain textCopyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
These materials are licensed as "Restricted Program Materials" under the Program Materials License Agreement (the "Agreement") in connection with the Alexa Voice Service Program. The Agreement is available at https://developer.amazon.com/public/support/pml.html. See the Agreement for the specific terms and conditions of the Agreement. Capitalized terms not defined in this file have the meanings given to them in the Agreement.
{
"interactionModel": {
"languageModel": {
"invocationName": "bird babysitter",
"intents": [
{
"name": "AMAZON.CancelIntent",
"samples": []
},
{
"name": "AMAZON.HelpIntent",
"samples": []
},
{
"name": "AMAZON.StopIntent",
"samples": []
},
{
"name": "AMAZON.NavigateHomeIntent",
"samples": []
},
{
"name": "FeedIntent",
"samples": [
"Feed now",
"Feed the bird",
"Go feed"
]
},
{
"name": "EntertainIntent",
"samples": [
"Dance",
"Play with the bird",
"Its play time",
"Hangout with the bird",
"Have fun with the bird"
]
}
}
}
}
}
/*
* Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
* You may not use this file except in compliance with the terms and conditions
* set forth in the accompanying LICENSE.TXT file.
*
* THESE MATERIALS ARE PROVIDED ON AN "AS IS" BASIS. AMAZON SPECIFICALLY DISCLAIMS, WITH
* RESPECT TO THESE MATERIALS, ALL WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING
* THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.
*/
// This sample demonstrates sending directives to an Echo connected gadget from an Alexa skill
// using the Alexa Skills Kit SDK (v2). Please visit https://alexa.design/cookbook for additional
// examples on implementing slots, dialog management, session persistence, api calls, and more.
const Alexa = require('ask-sdk-core');
const Util = require('./util');
const Common = require('./common');
// The namespace of the custom directive to be sent by this skill
const NAMESPACE = 'Custom.Mindstorms.Gadget';
// The name of the custom directive to be sent this skill
const NAME_CONTROL = 'control';
const LaunchRequestHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'LaunchRequest';
},
handle: async function(handlerInput) {
let request = handlerInput.requestEnvelope;
let { apiEndpoint, apiAccessToken } = request.context.System;
let apiResponse = await Util.getConnectedEndpoints(apiEndpoint, apiAccessToken);
if ((apiResponse.endpoints || []).length === 0) {
return handlerInput.responseBuilder
.speak(`I couldn't find an EV3 Brick connected to this Echo device. Please check to make sure your EV3 Brick is connected, and try again.`)
.getResponse();
}
// Store the gadget endpointId to be used in this skill session
let endpointId = apiResponse.endpoints[0].endpointId || [];
Util.putSessionAttribute(handlerInput, 'endpointId', endpointId);
return handlerInput.responseBuilder
.speak("Hello, I'm your Parakeet Babysitter. Would you like to play with your bird, watch them or feed them?")
.reprompt("Awaiting commands")
.getResponse();
}
};
// Add the speed value to the session attribute.
// This allows other intent handler to use the specified speed value
// without asking the user for input.
const SetSpeedIntentHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
&& Alexa.getIntentName(handlerInput.requestEnvelope) === 'SetSpeedIntent';
},
handle: function (handlerInput) {
// Bound speed to (1-10)
let speed = Alexa.getSlotValue(handlerInput.requestEnvelope, 'Speed');
speed = Math.max(1, Math.min(10, parseInt(speed)));
Util.putSessionAttribute(handlerInput, 'speed', speed);
return handlerInput.responseBuilder
.speak(`speed set to ${speed} percent.`)
.reprompt("awaiting command")
.getResponse();
}
};
// Construct and send a custom directive to the connected gadget with
// data from the MoveIntent request.
const MoveIntentHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
&& Alexa.getIntentName(handlerInput.requestEnvelope) === 'MoveIntent';
},
handle: function (handlerInput) {
const request = handlerInput.requestEnvelope;
const direction = Alexa.getSlotValue(request, 'Direction');
// Duration is optional, use default if not available
const duration = Alexa.getSlotValue(request, 'Duration') || "2";
// Get data from session attribute
const attributesManager = handlerInput.attributesManager;
const speed = attributesManager.getSessionAttributes().speed || "10";
const endpointId = attributesManager.getSessionAttributes().endpointId || [];
// Construct the directive with the payload containing the move parameters
const directive = Util.build(endpointId, NAMESPACE, NAME_CONTROL,
{
type: 'move',
direction: direction,
duration: duration,
speed: speed
});
const speechOutput = (direction === "brake")
? "Applying brake"
: `${direction} ${duration} seconds at ${speed} percent speed`;
return handlerInput.responseBuilder
.speak(speechOutput)
.reprompt("awaiting command")
.addDirective(directive)
.getResponse();
}
};
// Construct and send a custom directive to the connected gadget with data from
// the SetCommandIntent request.
const SetCommandIntentHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
&& Alexa.getIntentName(handlerInput.requestEnvelope) === 'SetCommandIntent';
},
handle: function (handlerInput) {
let command = Alexa.getSlotValue(handlerInput.requestEnvelope, 'Command');
if (!command) {
return handlerInput.responseBuilder
.speak("Can you repeat that?")
.reprompt("What was that again?").getResponse();
}
const attributesManager = handlerInput.attributesManager;
let endpointId = attributesManager.getSessionAttributes().endpointId || [];
let speed = attributesManager.getSessionAttributes().speed || "10";
// Construct the directive with the payload containing the move parameters
let directive = Util.build(endpointId, NAMESPACE, NAME_CONTROL,
{
type: 'command',
command: command,
speed: speed
});
return handlerInput.responseBuilder
.speak(`command ${command} activated`)
.reprompt("awaiting command")
.addDirective(directive)
.getResponse();
}
};
// The SkillBuilder acts as the entry point for your skill, routing all request and response
// payloads to the handlers above. Make sure any new handlers or interceptors you've
// defined are included below. The order matters - they're processed top to bottom.
exports.handler = Alexa.SkillBuilders.custom()
.addRequestHandlers(
LaunchRequestHandler,
SetSpeedIntentHandler,
SetCommandIntentHandler,
MoveIntentHandler,
Common.HelpIntentHandler,
Common.CancelAndStopIntentHandler,
Common.SessionEndedRequestHandler,
Common.IntentReflectorHandler, // make sure IntentReflectorHandler is last so it doesn't override your custom intent handlers
)
.addRequestInterceptors(Common.RequestInterceptor)
.addErrorHandlers(
Common.ErrorHandler,
)
.lambda();
common.js
JSON/*
* Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
* You may not use this file except in compliance with the terms and conditions
* set forth in the accompanying LICENSE.TXT file.
*
* THESE MATERIALS ARE PROVIDED ON AN "AS IS" BASIS. AMAZON SPECIFICALLY DISCLAIMS, WITH
* RESPECT TO THESE MATERIALS, ALL WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING
* THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.
*/
'use strict'
const Alexa = require('ask-sdk-core');
const HelpIntentHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
&& Alexa.getIntentName(handlerInput.requestEnvelope) === 'AMAZON.HelpIntent';
},
handle(handlerInput) {
const speakOutput = 'Hello, I am your Parakeet Babysitter. I am here to help take care of your parakeet.';
return handlerInput.responseBuilder
.speak(speakOutput)
.reprompt(speakOutput)
.getResponse();
}
};
const CancelAndStopIntentHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
&& (Alexa.getIntentName(handlerInput.requestEnvelope) === 'AMAZON.CancelIntent'
|| Alexa.getIntentName(handlerInput.requestEnvelope) === 'AMAZON.StopIntent');
},
handle(handlerInput) {
const speakOutput = 'Goodbye!';
return handlerInput.responseBuilder
.speak(speakOutput)
.getResponse();
}
};
const SessionEndedRequestHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'SessionEndedRequest';
},
handle(handlerInput) {
// Any cleanup logic goes here.
return handlerInput.responseBuilder.getResponse();
}
};
// The intent reflector is used for interaction model testing and debugging.
// It will simply repeat the intent the user said. You can create custom handlers
// for your intents by defining them above, then also adding them to the request
// handler chain below.
const IntentReflectorHandler = {
canHandle(handlerInput) {
return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest';
},
handle(handlerInput) {
const intentName = Alexa.getIntentName(handlerInput.requestEnvelope);
const speakOutput = `You just triggered ${intentName}`;
return handlerInput.responseBuilder
.speak(speakOutput)
.reprompt("I don't understand this command, try again")
.getResponse();
}
};
// Generic error handling to capture any syntax or routing errors. If you receive an error
// stating the request handler chain is not found, you have not implemented a handler for
// the intent being invoked or included it in the skill builder below.
const ErrorHandler = {
canHandle() {
return true;
},
handle(handlerInput, error) {
console.log(`~~~~ Error handled: ${error.stack}`);
const speakOutput = `Sorry, I had trouble doing what you asked. Please try again.`;
return handlerInput.responseBuilder
.speak(speakOutput)
.reprompt(speakOutput)
.getResponse();
}
};
// The request interceptor is used for request handling testing and debugging.
// It will simply log the request in raw json format before any processing is performed.
const RequestInterceptor = {
process(handlerInput) {
let { attributesManager, requestEnvelope } = handlerInput;
let sessionAttributes = attributesManager.getSessionAttributes();
// Log the request for debug purposes.
console.log(`=====Request==${JSON.stringify(requestEnvelope)}`);
console.log(`=========SessionAttributes==${JSON.stringify(sessionAttributes, null, 2)}`);
}
};
module.exports = {
HelpIntentHandler,
CancelAndStopIntentHandler,
SessionEndedRequestHandler,
IntentReflectorHandler,
ErrorHandler,
RequestInterceptor
};
util.js
JSON/*
* Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
* You may not use this file except in compliance with the terms and conditions
* set forth in the accompanying LICENSE.TXT file.
*
* THESE MATERIALS ARE PROVIDED ON AN "AS IS" BASIS. AMAZON SPECIFICALLY DISCLAIMS, WITH
* RESPECT TO THESE MATERIALS, ALL WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING
* THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.
*/
'use strict';
const Https = require('https');
/**
* Build a custom directive payload to the gadget with the specified endpointId
* @param {string} endpointId - the gadget endpoint Id
* @param {string} namespace - the namespace of the skill
* @param {string} name - the name of the skill within the scope of this namespace
* @param {object} payload - the payload data
* @see {@link https://developer.amazon.com/docs/alexa-gadgets-toolkit/send-gadget-custom-directive-from-skill.html#respond}
*/
exports.build = function (endpointId, namespace, name, payload) {
// Construct the custom directive that needs to be sent
// Gadget should declare the capabilities in the discovery response to
// receive the directives under the following namespace.
return {
type: 'CustomInterfaceController.SendDirective',
header: {
name: name,
namespace: namespace
},
endpoint: {
endpointId: endpointId
},
payload
};
};
/**
* A convenience routine to add the a key-value pair to the session attribute.
* @param handlerInput - the context from Alexa Service
* @param key - the key to be added
* @param value - the value be added
*/
exports.putSessionAttribute = function(handlerInput, key, value) {
const attributesManager = handlerInput.attributesManager;
let sessionAttributes = attributesManager.getSessionAttributes();
sessionAttributes[key] = value;
attributesManager.setSessionAttributes(sessionAttributes);
};
/**
* To get a list of all the gadgets that meet these conditions,
* Call the Endpoint Enumeration API with the apiEndpoint and apiAccessToken to
* retrieve the list of all connected gadgets.
*
* @param {string} apiEndpoint - the Endpoint API url
* @param {string} apiAccessToken - the token from the session object in the Alexa request
* @see {@link https://developer.amazon.com/docs/alexa-gadgets-toolkit/send-gadget-custom-directive-from-skill.html#call-endpoint-enumeration-api}
*/
exports.getConnectedEndpoints = function(apiEndpoint, apiAccessToken) {
// The preceding https:// need to be stripped off before making the call
apiEndpoint = (apiEndpoint || '').replace('https://', '');
return new Promise(((resolve, reject) => {
const options = {
host: apiEndpoint,
path: '/v1/endpoints',
method: 'GET',
headers: {
'Content-Type': 'application/json',
'Authorization': 'Bearer ' + apiAccessToken
}
};
const request = Https.request(options, (response) => {
response.setEncoding('utf8');
let returnData = '';
response.on('data', (chunk) => {
returnData += chunk;
});
response.on('end', () => {
resolve(JSON.parse(returnData));
});
response.on('error', (error) => {
reject(error);
});
});
request.end();
}));
};
{
"name": "agt-mindstorms",
"version": "1.1.0",
"description": "A sample skill demonstrating how to use AGT with Lego Mindstorms",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "Amazon Alexa",
"license": "ISC",
"dependencies": {
"ask-sdk-core": "^2.6.0",
"ask-sdk-model": "^1.18.0",
"aws-sdk": "^2.326.0",
"request": "^2.81.0"
}
}
Comments