RoboTeam: Adrianna Zalewska, Krzysztof Sołtysik, Dominika Skrzypek, Michał Ossowski

Published January 2, 2020 © CC BY-NC-SA

Artist RoboHelper

Artist RoboHelper makes sure you get the right crayon, helps you put it back in place and may check the color of the pen you’re holding.

IntermediateFull instructions provided3 hours1,240

First 100 Submissions

LEGO MINDSTORMS Voice Challenge

Things used in this project

Hardware components

Amazon Alexa Echo Dot

Any other Alexa device may also be acceptable.

LEGO Mindstorms EV3 Programming Brick / Kit

Build with components from one LEGO Education® MINDSTORMS® EV3 set.

Micro SD Card

Any other SD Card with minimum 8 gb memory size is acceptable.

Software apps and online services

Amazon Alexa Alexa Skills Kit

Amazon Alexa Alexa Gadgets Toolkit

ev3dev - stretch for EV3

Microsoft Visual Studio Code

Python 3

Story

Project description:

The project presents an Artist RoboHelper build with components from one LEGO Education® MINDSTORMS® EV3 set connected to Alexa Echo device.

This drawing assistant model is built with two large motors, one EV3 brick, one color sensor and one touch sensor. The construction has a shape of a rotary toolbox with separate pockets for crayons and pencils and a convenient dispenser. Next to the dispenser is a special platform with sensor, which checks the color of the presented object.

Movie:

Artist RoboHelper working model in action

How does it work?

Once the user tells Alexa to open the MINDSTORMS application, he/she can ask to get a crayon of a specific color. As soon as Alexa receives the GIVE_’COLOR’ instruction, she turns on the motor responsible for rotating the toolbox for an amount of time programmed for the specified color. Next, she activates the motor connected to the release, which is responsible for removing the object from the dispenser. The object falls out of the dispenser to a special container made of tires, then the release closes again. Alexa turns on the first motor to rotate the container back to its starting position. The user can take the object from the container and use it. Then, he/she can ask for another crayon, or put the taken one back into the dispenser.

To put the crayon back, the user needs to use a ‘COLOR’ command. Alexa rotates the dispenser to make the place specified for the object of this color move to the front. She turns on only one motor, the release is not activated. Then, the user can put the crayon back in its place. The user may need some time for this, which is why Alexa is waiting for him/her to press a button. After putting the object back to the dispenser, the user presses the button and Alexa turns on the motor once again to reset the position.

Two voice instructions listed above may sometimes sound similar and the user can make a mistake while in pronunciation. The robot has a provision for such situations: manual instruction reversing. Before starting to execute the command, Alexa tells the user what she is about to do. In that moment, the user hears whether he/she has made a mistake; if that is the case, the user can press the button. If he/she wanted to take a crayon, but activated the ‘COLOR’ command instead, after pressing the button the command will be changed to GIVE_’COLOR’. Pressing the button will work opposite while activating GIVE_’COLOR’ accidentally.

Once the user finishes drawing, he/she may have more than one crayon taken out. The user can check what color the crayon is by putting it on a special platform and activating WHAT_COLOR command. Upon hearing the command, Alexa turns on the color reading via the colour sensor and activates the EV3 speaker to vocalize the color name three times. Knowing the name of the color is necessary to use other commands.

What’s more, there is also CALIBRATE command. It can be used when the user isn’t sure if the robot is in its starting position. The user should take all the crayons out, except the one (brown) next to the “calibration brick” (white bionicle 1x3 tooth with axlehole), and activate the instruction. In such case, Alexa opens the release and turns the motor on, so the toolbox starts to slowly rotate. The user must wait for the one crayon to fall out and, at this moment, press the button. The toolbox will stop rotating and the release will close at the end of the calibration process.

This project can be very useful for children while learning color names. They can use WHAT_COLOR command to know the crayon of what color they have in their hands. Alexa is also very good at answering questions about the color of things, so this function may be used along with the MINDSTORMS commands. This robot is also adapted to help kids that have problems with seeing colors or with sight in general. Thanks to the dispenser, they can easily get the color they want and keep everything organized. Not only the commands are adapted, but the whole construction is built in such a way that one can easily navigate it merely by using the sense of touch. The objects can be put back by relying on the release position. The robot also includes a “calibration brick” – a brick different than others in shape, which marks the starting point.

Key features (inclued also in a movie):

When designing this eye-catching and very helpful robot, we decided to combine some devices and concepts of great potential:

robotics,
programming,
Alexa Echo Device,
Lego Mindstorms,
RoboCamp teaching method,
innovative approach to learning

We were able to create a robot that looks cool, but is also very useful. It relies on voice commands given to Alexa.

Artist RoboHelper helps you choose a crayon, or any colorful pen or pencil
Makes sure you get the right color
Helps you put the pencil back in place
And in case you’re not sure, it can also check the color of the pen you’re holding and help you choose the one you’re looking for

The most important thing for us, as designers, is to create solutions that help others. Supporting people at risk of social exclusion is a high priority for us. So how exactly is our Artist RoboHelper innovative? Because it can help children and adults in 6 different ways:

Helps color-blind people to further develop their artistic skills
Gives opportunity to visually impaired children to participate in social life with other kids
Helps small children learn and recognize colors
Encourages the use of English language in practice, and loud
Helps children with low muscle tone and fine motor skills disorders to make manipulating small objects, such as pencils, easier
And of course, it will make you learn correct pronunciation

Sharing knowledge and inspiring others is part of our every-day life and work. We prepared detailed, step by step instructions on how to build and program Artist RoboHelper. In future we want to go even further and prepare digital, online teaching resources based on RoboCamp 5C teaching method. They will be easy to understand, to implement, and hopefully, encouraging for teachers and students of any age!

Included files:

Gadget source code file
Skill source code files
Video of working prototype and applications
Additional documentation in pdf:-Instruction for a user-LEGO building instructions-Coding instructions (for python file)-Code samples (for JSON Editor file)

Submission walk-through:

The idea of a robot that is voice-controlled and helps in drawing arose in October 2019. Originally it was thought to be a vehicle driving on the table that scans the colors of the crayons lying on it and gives them to the illustrator. It turned out to be quite inconvenient in construction and it was never built completely.

Artist RoboHelper – Version 1

First complete construction of Artist RoboHelper

The first fully accomplished version of a robot was created in November. The construction was redesigned to a form of a rotary toolbox. The design was very cool but we discovered some things to change. We have drawn the following conclusions while using the first construction:1. The color sensor has a simple to build housing but has trouble distinguishing colors in it.2. The crayons were inserted with the spike down, which unfortunately caused drawing the platform.3. The crayon feeding mechanism works well and smoothly, so it is worth keeping it unchanged.

Artist RoboHelper – Version 2

Second version of Artist RoboHelper with Alexa

Redesigned platform for color checking

Picture of certain mechansims

We have drawn conclusions from version 1. We have changed the housing of the color sensor - now it has a shape of a longitudinal platform that is mounted near to the sensor’s surface. The crayons are loaded from above and the robot is working very well! However, we decided to work on the appearance of the robot.

Artist RoboHelper – Version 3

Final version of Artist RoboHelper

Construction with ergonomic design and different EV3 brick mounting

We've rebuilt the robot to make it look better. The main change was a different position of an EV3 brick. The construction not only looks better but also takes up less space on the table. Now it looks and works just perfect!

Code

# Copyright 2019 Amazon.com, Inc. or its affiliates.  All Rights Reserved.
# 
# You may not use this file except in compliance with the terms and conditions 
# set forth in the accompanying LICENSE.TXT file.
#
# THESE MATERIALS ARE PROVIDED ON AN "AS IS" BASIS. AMAZON SPECIFICALLY DISCLAIMS, WITH 
# RESPECT TO THESE MATERIALS, ALL WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING 
# THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.

import time
import logging
import json

from enum import Enum    
from time import sleep   

# Importing AlexaGadget class
from agt import AlexaGadget    

# Importing EV3dev resources
from ev3dev2.led import Leds
from ev3dev2.sound import Sound
from ev3dev2.motor import OUTPUT_B, OUTPUT_C, LargeMotor
from ev3dev2.sensor.lego import TouchSensor, ColorSensor

# Set the logging level to INFO to see messages from AlexaGadget
logging.basicConfig(level=logging.INFO)

class Command(Enum):

    BLACK = ['black','color black','black color']
    BLUE = ['blue','color blue','blue color']
    GREEN = ['green','color green','green color']
    YELLOW = ['yellow','color yellow','yellow color']
    RED = ['red','color red','red color']
    BROWN = ['brown','color brown','brown color']
    WHAT_COLOR = ['what color','what','color']  
    GIVE_BLACK = ['give me black','give black','me black','get black']
    GIVE_BLUE = ['give me blue','give blue','me blue','get blue']
    GIVE_GREEN = ['give me green','give green','me green','get green']
    GIVE_RED = ['give me red','give red','me red','get red']
    GIVE_YELLOW = ['give me yellow','give yellow','me yellow','get yellow']
    GIVE_BROWN = ['give me brown','give brown','me brown','get brown']
    CALIBRATION = ['calibration','calibrate','bright']

class MindstormsGadget(AlexaGadget):
# Initializing AlexaGadget with MindstormsGadget extension
    def __init__(self):
        super().__init__()

        # Ev3dev initialization
        self.leds = Leds()
        self.sound = Sound()
        self.rotation = LargeMotor(OUTPUT_B)     
        self.trap = LargeMotor(OUTPUT_C)
        self.color = ColorSensor()
        self.touch = TouchSensor()

    # Turning lights on when EV3 brick is connected with parameter friendly_name for EV3 brick
    def on_connected(self, device_addr):
        self.leds.set_color("LEFT", "GREEN")
        self.leds.set_color("RIGHT", "GREEN")
        print("{} connected to Echo device".format(self.friendly_name))

    # Turning lights off when EV3 brick is disconnected with parameter friendly_name for EV3 brick
    def on_disconnected(self, device_addr):
        self.leds.set_color("LEFT", "BLACK")
        self.leds.set_color("RIGHT", "BLACK")
        print("{} disconnected from Echo device".format(self.friendly_name))        

    # Defining custom directive to handle EV3 commands
    def on_custom_mindstorms_gadget_control(self, directive):
        try:
            payload = json.loads(directive.payload.decode("utf-8"))
            print("Control payload: {}".format(payload))
            control_type = payload["type"]
            if control_type == "command":
                # Expected params: [command]
                self._activate(payload["command"])

        except KeyError:
            print("Missing expected parameters: {}".format(directive))
    
    # Defining command logic mapped to EV3 actions
    def _activate(self, command):
        print("Activate command: ({})".format(command))

        
        # Defining 'COLOR' commands 
            # Use self.rotation.on_for_degrees(speed=[speed value], degrees=[speed value]) to rotate B Motor with specified parameters
            # Use self.trap.on_for_degrees(speed=[speed value], degrees=[speed value]) to rotate C Motor with specified parameters
            # Use self.touch.is_pressed to check Touch Sensor readout
            # Use time.sleep([time]) to wait a specified amount of time
        if command in Command.BLACK.value:
            print("BLACK")
            self.rotation.on_for_degrees(speed=100, degrees=270)
            if self.touch.is_pressed:
                self.trap.on_for_degrees(speed=80, degrees=90)
                time.sleep(0.5)
                self.trap.on_for_degrees(speed=80, degrees=-90)
            else:
                time.sleep(1)
                self.touch.wait_for_bump ()
            self.rotation.on_for_degrees(speed=100, degrees=-270)
 
        if command in Command.BLUE.value:
            print("BLUE")
            self.rotation.on_for_degrees(speed=100, degrees=-1080)
            if self.touch.is_pressed:
                self.trap.on_for_degrees(speed=80, degrees=90)
                time.sleep(0.5)
                self.trap.on_for_degrees(speed=80, degrees=-90)
            else:
                time.sleep(1)
                self.touch.wait_for_bump ()
            self.rotation.on_for_degrees(speed=100, degrees=1080)

        if command in Command.GREEN.value:
            print("GREEN")
            self.rotation.on_for_degrees(speed=100, degrees=-1400)
            if self.touch.is_pressed:
                self.trap.on_for_degrees(speed=80, degrees=90)
                time.sleep(0.5)
                self.trap.on_for_degrees(speed=80, degrees=-90)
            else:
                time.sleep(1)
                self.touch.wait_for_bump ()
            self.rotation.on_for_degrees(speed=100, degrees=1400)

        if command in Command.YELLOW.value:
            print("YELLOW")
            self.rotation.on_for_degrees(speed=100, degrees=1200)
            if self.touch.is_pressed:
                self.trap.on_for_degrees(speed=80, degrees=90)
                time.sleep(0.5)
                self.trap.on_for_degrees(speed=80, degrees=-90)
            else:
                time.sleep(1)
                self.touch.wait_for_bump ()
            self.rotation.on_for_degrees(speed=100, degrees=-1200)

        if command in Command.RED.value:
            print("RED")
            self.rotation.on_for_degrees(speed=100, degrees=940)
            if self.touch.is_pressed:
                self.trap.on_for_degrees(speed=80, degrees=90)
                time.sleep(0.5)
                self.trap.on_for_degrees(speed=80, degrees=-90)
            else:
                time.sleep(1)
                self.touch.wait_for_bump ()
            self.rotation.on_for_degrees(speed=100, degrees=-940)

        if command in Command.BROWN.value:
            print("BROWN")
            self.rotation.on_for_degrees(speed=100, degrees=-180)
            if self.touch.is_pressed:
                self.trap.on_for_degrees(speed=80, degrees=90)
                time.sleep(0.5)
                self.trap.on_for_degrees(speed=80, degrees=-90)
            else:
                time.sleep(1)
                self.touch.wait_for_bump ()
            self.rotation.on_for_degrees(speed=100, degrees=180)

        # Defining WHAT_COLOR command
            # Use self.color.color_name to read form Color Sensor
            # Use self.sound.speak([name]) to generate speaker sound with specified name
        if command in Command.WHAT_COLOR.value:
            print("WHAT_COLOR")
            for i in range(3):
                print(self.color.color_name)
                self.sound.speak(self.color.color_name)

        # Defining CALIBRATION command
            # Use self.rotation.on_for_degrees(speed=[speed value], degrees=[speed value]) to rotate B Motor with specified parameters
            # Use self.trap.on_for_degrees(speed=[speed value], degrees=[speed value]) to rotate C Motor with specified parameters
            # Use self.touch.is_pressed to check Touch Sensor readout
            # Use time.sleep([time]) to wait a specified amount of time
        if command in Command.CALIBRATION.value:
            print("CALIBRATION")
            self.trap.on_for_degrees(speed=80, degrees=90)
            while not self.touch.is_pressed:
                self.rotation.on_for_degrees(speed=50, degrees=20)
                time.sleep(0.1)
            self.rotation.on_for_degrees(speed=50, degrees=100)
            time.sleep(0.5)
            self.trap.on_for_degrees(speed=80, degrees=-90) 

        # Defining GIVE_'COLOR' commands 
            # Use self.rotation.on_for_degrees(speed=[speed value], degrees=[speed value]) to rotate B Motor with specified parameters
            # Use self.trap.on_for_degrees(speed=[speed value], degrees=[speed value]) to rotate C Motor with specified parameters
            # Use self.touch.is_pressed to check Touch Sensor readout
            # Use time.sleep([time]) to wait a specified amount of time
        if command in Command.GIVE_BLACK.value:
            print("GIVE_BLACK")
            self.rotation.on_for_degrees(speed=100, degrees=270)
            if self.touch.is_pressed:
                time.sleep(1)
                self.touch.wait_for_bump ()
            else:
                self.trap.on_for_degrees(speed=80, degrees=90)
                time.sleep(0.5)
                self.trap.on_for_degrees(speed=80, degrees=-90)
            self.rotation.on_for_degrees(speed=100, degrees=-270)

        if command in Command.GIVE_BLUE.value:
            print("GIVE_BLUE")
            self.rotation.on_for_degrees(speed=100, degrees=-1080)
            if self.touch.is_pressed:
                time.sleep(1)
                self.touch.wait_for_bump ()
            else:
                self.trap.on_for_degrees(speed=80, degrees=90)
                time.sleep(0.5)
                self.trap.on_for_degrees(speed=80, degrees=-90)
            self.rotation.on_for_degrees(speed=100, degrees=1080)

        if command in Command.GIVE_GREEN.value:
            print("GIVE_GREEN")
            self.rotation.on_for_degrees(speed=100, degrees=-1400)
            if self.touch.is_pressed:
                time.sleep(1)
                self.touch.wait_for_bump ()
            else:
                self.trap.on_for_degrees(speed=80, degrees=90)
                time.sleep(0.5)
                self.trap.on_for_degrees(speed=80, degrees=-90)
            self.rotation.on_for_degrees(speed=100, degrees=1400)

        if command in Command.GIVE_YELLOW.value:
            print("GIVE_YELLOW")
            self.rotation.on_for_degrees(speed=100, degrees=1200)
            if self.touch.is_pressed:
                time.sleep(1)
                self.touch.wait_for_bump ()
            else:
                self.trap.on_for_degrees(speed=80, degrees=90)
                time.sleep(0.5)
                self.trap.on_for_degrees(speed=80, degrees=-90)
            self.rotation.on_for_degrees(speed=100, degrees=-1200)

        if command in Command.GIVE_RED.value:
            print("GIVE_RED")
            self.rotation.on_for_degrees(speed=100, degrees=940)
            if self.touch.is_pressed:
                time.sleep(1)
                self.touch.wait_for_bump ()
            else:
                self.trap.on_for_degrees(speed=80, degrees=90)
                time.sleep(0.5)
                self.trap.on_for_degrees(speed=80, degrees=-90)
            self.rotation.on_for_degrees(speed=100, degrees=-940)

        if command in Command.GIVE_BROWN.value:
            print("GIVE_BROWN")
            self.rotation.on_for_degrees(speed=100, degrees=-180)
            if self.touch.is_pressed:
                time.sleep(1)
                self.touch.wait_for_bump ()
            else:
                self.trap.on_for_degrees(speed=80, degrees=90)
                time.sleep(0.5)
                self.trap.on_for_degrees(speed=80, degrees=-90)
            self.rotation.on_for_degrees(speed=100, degrees=180)

if __name__ == '__main__':

    # Startup sequence
    gadget = MindstormsGadget()
    gadget.sound.play_song((('C4', 'e'), ('D4', 'e'), ('E5', 'q')))
    gadget.leds.set_color("LEFT", "GREEN")
    gadget.leds.set_color("RIGHT", "GREEN")

    # Gadget main entry point
    gadget.main()

    # Shutdown sequence
    gadget.sound.play_song((('E5', 'e'), ('C4', 'e')))
    gadget.leds.set_color("LEFT", "BLACK")
    gadget.leds.set_color("RIGHT", "BLACK")

{
  "interactionModel": {
      "languageModel": {
          "invocationName": "mindstorms",
          "intents": [
              {
                  "name": "AMAZON.CancelIntent",
                  "samples": []
              },
              {
                  "name": "AMAZON.HelpIntent",
                  "samples": []
              },
              {
                  "name": "AMAZON.StopIntent",
                  "samples": []
              },
              {
                  "name": "AMAZON.NavigateHomeIntent",
                  "samples": []
              },
              {
                  "name": "MoveIntent",
                  "slots": [
                      {
                          "name": "Direction",
                          "type": "DirectionType"
                      },
                      {
                          "name": "Duration",
                          "type": "AMAZON.NUMBER"
                      }
                  ],
                  "samples": [
                      "{Direction} now",
                      "{Direction} {Duration} seconds",
                      "move {Direction} for {Duration} seconds"
                  ]
              },
              {
                  "name": "SetSpeedIntent",
                  "slots": [
                      {
                          "name": "Speed",
                          "type": "AMAZON.NUMBER"
                      }
                  ],
                  "samples": [
                      "set speed {Speed} percent",
                      "set {Speed} percent speed",
                      "set speed to {Speed} percent"
                  ]
              },
              {
                  "name": "SetCommandIntent",
                  "slots": [
                      {
                          "name": "Command",
                          "type": "CommandType"
                      }
                  ],
                  "samples": [
                      "activate {Command} mode",
                      "move in a {Command}",
                      "fire {Command}",
                      "activate {Command}"
                  ]
              }
          ],
          "types": [
              {
                  "name": "DirectionType",
                  "values": [
                      {
                          "name": {
                              "value": "brake"
                          }
                      },
                      {
                          "name": {
                              "value": "go backward"
                          }
                      },
                      {
                          "name": {
                              "value": "go forward"
                          }
                      },
                      {
                          "name": {
                              "value": "go right"
                          }
                      },
                      {
                          "name": {
                              "value": "go left"
                          }
                      },
                      {
                          "name": {
                              "value": "right"
                          }
                      },
                      {
                          "name": {
                              "value": "left"
                          }
                      },
                      {
                          "name": {
                              "value": "backwards"
                          }
                      },
                      {
                          "name": {
                              "value": "backward"
                          }
                      },
                      {
                          "name": {
                              "value": "forwards"
                          }
                      },
                      {
                          "name": {
                              "value": "forward"
                          }
                      }
                  ]
              },
              {
                  "name": "CommandType",
                  "values": [
                      {
                          "name": {
                              "value": "give black",
                              "synonyms": [
                                  "get black",
                                  "me black",
                                  "give me black"
                              ]
                          }
                      },
                      {
                          "name": {
                              "value": "give blue",
                              "synonyms": [
                                  "get blue",
                                  "me blue",
                                  "give me blue"
                              ]
                          }
                      },
                      {
                          "name": {
                              "value": "give green",
                              "synonyms": [
                                  "get green",
                                  "me green",
                                  "give me green"
                              ]
                          }
                      },
                      {
                          "name": {
                              "value": "give yellow",
                              "synonyms": [
                                  "get yellow",
                                  "me yellow",
                                  "give me yellow"
                              ]
                          }
                      },
                      {
                          "name": {
                              "value": "give red",
                              "synonyms": [
                                  "get red",
                                  "me red",
                                  "give me red"
                              ]
                          }
                      },
                      {
                          "name": {
                              "value": "give brown",
                              "synonyms": [
                                  "get brown",
                                  "me brown",
                                  "give me brown"
                              ]
                          }
                      },
                      {
                          "name": {
                              "value": "calibration",
                              "synonyms": [
                                  "bright",
                                  "calibrate"
                              ]
                          }
                      },
                      {
                          "name": {
                              "value": "what color",
                              "synonyms": [
                                  "color",
                                  "what"
                              ]
                          }
                      },
                      {
                          "name": {
                              "value": "brown",
                              "synonyms": [
                                  "color brown",
                                  "brown color"
                              ]
                          }
                      },
                      {
                          "name": {
                              "value": "red",
                              "synonyms": [
                                  "red color",
                                  "color red"
                              ]
                          }
                      },
                      {
                          "name": {
                              "value": "yellow",
                              "synonyms": [
                                  "color yellow",
                                  "yellow color"
                              ]
                          }
                      },
                      {
                          "name": {
                              "value": "green",
                              "synonyms": [
                                  "color green",
                                  "green color"
                              ]
                          }
                      },
                      {
                          "name": {
                              "value": "blue",
                              "synonyms": [
                                  "color blue",
                                  "blue color"
                              ]
                          }
                      },
                      {
                          "name": {
                              "value": "black",
                              "synonyms": [
                                  "color black",
                                  "black color"
                              ]
                          }
                      }
                  ]
              }
          ]
      }
  }
}

/*
 * Copyright 2019 Amazon.com, Inc. or its affiliates.  All Rights Reserved.
 *
 * You may not use this file except in compliance with the terms and conditions 
 * set forth in the accompanying LICENSE.TXT file.
 *
 * THESE MATERIALS ARE PROVIDED ON AN "AS IS" BASIS. AMAZON SPECIFICALLY DISCLAIMS, WITH 
 * RESPECT TO THESE MATERIALS, ALL WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING 
 * THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.
*/

'use strict'

const Alexa = require('ask-sdk-core');

const HelpIntentHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
            && Alexa.getIntentName(handlerInput.requestEnvelope) === 'AMAZON.HelpIntent';
    },
    handle(handlerInput) {
        const speakOutput = 'You can say hello to me! How can I help?';

        return handlerInput.responseBuilder
            .speak(speakOutput)
            .reprompt(speakOutput)
            .getResponse();
    }
};
const CancelAndStopIntentHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
            && (Alexa.getIntentName(handlerInput.requestEnvelope) === 'AMAZON.CancelIntent'
                || Alexa.getIntentName(handlerInput.requestEnvelope) === 'AMAZON.StopIntent');
    },
    handle(handlerInput) {
        const speakOutput = 'Goodbye!';
        return handlerInput.responseBuilder
            .speak(speakOutput)
            .getResponse();
    }
};
const SessionEndedRequestHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'SessionEndedRequest';
    },
    handle(handlerInput) {
        // Any cleanup logic goes here.
        return handlerInput.responseBuilder.getResponse();
    }
};

// The intent reflector is used for interaction model testing and debugging.
// It will simply repeat the intent the user said. You can create custom handlers
// for your intents by defining them above, then also adding them to the request
// handler chain below.
const IntentReflectorHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest';
    },
    handle(handlerInput) {
        const intentName = Alexa.getIntentName(handlerInput.requestEnvelope);
        const speakOutput = `You just triggered ${intentName}`;

        return handlerInput.responseBuilder
            .speak(speakOutput)
            .reprompt("I don't understand this command, try again")
            .getResponse();
    }
};

// Generic error handling to capture any syntax or routing errors. If you receive an error
// stating the request handler chain is not found, you have not implemented a handler for
// the intent being invoked or included it in the skill builder below.
const ErrorHandler = {
    canHandle() {
        return true;
    },
    handle(handlerInput, error) {
        console.log(`~~~~ Error handled: ${error.stack}`);
        const speakOutput = `Sorry, I had trouble doing what you asked. Please try again.`;

        return handlerInput.responseBuilder
            .speak(speakOutput)
            .reprompt(speakOutput)
            .getResponse();
    }
};

// The request interceptor is used for request handling testing and debugging.
// It will simply log the request in raw json format before any processing is performed.
const RequestInterceptor = {
    process(handlerInput) {
        let { attributesManager, requestEnvelope } = handlerInput;
        let sessionAttributes = attributesManager.getSessionAttributes();

        // Log the request for debug purposes.
        console.log(`=====Request==${JSON.stringify(requestEnvelope)}`);
        console.log(`=========SessionAttributes==${JSON.stringify(sessionAttributes, null, 2)}`);
    }
};

module.exports = {
    HelpIntentHandler,
    CancelAndStopIntentHandler,
    SessionEndedRequestHandler,
    IntentReflectorHandler,
    ErrorHandler,
    RequestInterceptor
    };

/*
 * Copyright 2019 Amazon.com, Inc. or its affiliates.  All Rights Reserved.
 *
 * You may not use this file except in compliance with the terms and conditions 
 * set forth in the accompanying LICENSE.TXT file.
 *
 * THESE MATERIALS ARE PROVIDED ON AN "AS IS" BASIS. AMAZON SPECIFICALLY DISCLAIMS, WITH 
 * RESPECT TO THESE MATERIALS, ALL WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING 
 * THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.
*/

// This sample demonstrates sending directives to an Echo connected gadget from an Alexa skill
// using the Alexa Skills Kit SDK (v2). Please visit https://alexa.design/cookbook for additional
// examples on implementing slots, dialog management, session persistence, api calls, and more.

const Alexa = require('ask-sdk-core');
const Util = require('./util');
const Common = require('./common');

// The namespace of the custom directive to be sent by this skill
const NAMESPACE = 'Custom.Mindstorms.Gadget';

// The name of the custom directive to be sent this skill
const NAME_CONTROL = 'control';

const LaunchRequestHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'LaunchRequest';
    },
    handle: async function(handlerInput) {

        let request = handlerInput.requestEnvelope;
        let { apiEndpoint, apiAccessToken } = request.context.System;
        let apiResponse = await Util.getConnectedEndpoints(apiEndpoint, apiAccessToken);
        if ((apiResponse.endpoints || []).length === 0) {
            return handlerInput.responseBuilder
            .speak(`I couldn't find an EV3 Brick connected to this Echo device. Please check to make sure your EV3 Brick is connected, and try again.`)
            .getResponse();
        }

        // Store the gadget endpointId to be used in this skill session
        let endpointId = apiResponse.endpoints[0].endpointId || [];
        Util.putSessionAttribute(handlerInput, 'endpointId', endpointId);

        return handlerInput.responseBuilder
            .speak("Welcome, you can start issuing move commands")
            .reprompt("Awaiting commands")
            .getResponse();
    }
};

// Add the speed value to the session attribute.
// This allows other intent handler to use the specified speed value
// without asking the user for input.
const SetSpeedIntentHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
            && Alexa.getIntentName(handlerInput.requestEnvelope) === 'SetSpeedIntent';
    },
    handle: function (handlerInput) {

        // Bound speed to (1-100)
        let speed = Alexa.getSlotValue(handlerInput.requestEnvelope, 'Speed');
        speed = Math.max(1, Math.min(100, parseInt(speed)));
        Util.putSessionAttribute(handlerInput, 'speed', speed);

        return handlerInput.responseBuilder
            .speak(`speed set to ${speed} percent.`)
            .reprompt("awaiting command")
            .getResponse();
    }
};

// Construct and send a custom directive to the connected gadget with
// data from the MoveIntent request.
const MoveIntentHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
            && Alexa.getIntentName(handlerInput.requestEnvelope) === 'MoveIntent';
    },
    handle: function (handlerInput) {
        const request = handlerInput.requestEnvelope;
        const direction = Alexa.getSlotValue(request, 'Direction');

        // Duration is optional, use default if not available
        const duration = Alexa.getSlotValue(request, 'Duration') || "2";

        // Get data from session attribute
        const attributesManager = handlerInput.attributesManager;
        const speed = attributesManager.getSessionAttributes().speed || "50";
        const endpointId = attributesManager.getSessionAttributes().endpointId || [];

        // Construct the directive with the payload containing the move parameters
        const directive = Util.build(endpointId, NAMESPACE, NAME_CONTROL,
            {
                type: 'move',
                direction: direction,
                duration: duration,
                speed: speed
            });

        const speechOutput = (direction === "brake")
            ?  "Applying brake"
            : `${direction} ${duration} seconds at ${speed} percent speed`;

        return handlerInput.responseBuilder
            .speak(speechOutput)
            .reprompt("awaiting command")
            .addDirective(directive)
            .getResponse();
    }
};

// Construct and send a custom directive to the connected gadget with data from
// the SetCommandIntent request.
const SetCommandIntentHandler = {
    canHandle(handlerInput) {
        return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
            && Alexa.getIntentName(handlerInput.requestEnvelope) === 'SetCommandIntent';
    },
    handle: function (handlerInput) {

        let command = Alexa.getSlotValue(handlerInput.requestEnvelope, 'Command');
        if (!command) {
            return handlerInput.responseBuilder
                .speak("Can you repeat that?")
                .reprompt("What was that again?").getResponse();
        }

        const attributesManager = handlerInput.attributesManager;
        let endpointId = attributesManager.getSessionAttributes().endpointId || [];
        let speed = attributesManager.getSessionAttributes().speed || "50";

        // Construct the directive with the payload containing the move parameters
        let directive = Util.build(endpointId, NAMESPACE, NAME_CONTROL,
            {
                type: 'command',
                command: command,
                speed: speed
            });

        return handlerInput.responseBuilder
            .speak(`command ${command} activated`)
            .reprompt("awaiting command")
            .addDirective(directive)
            .getResponse();
    }
};

// The SkillBuilder acts as the entry point for your skill, routing all request and response
// payloads to the handlers above. Make sure any new handlers or interceptors you've
// defined are included below. The order matters - they're processed top to bottom.
exports.handler = Alexa.SkillBuilders.custom()
    .addRequestHandlers(
        LaunchRequestHandler,
        SetSpeedIntentHandler,
        SetCommandIntentHandler,
        MoveIntentHandler,
        Common.HelpIntentHandler,
        Common.CancelAndStopIntentHandler,
        Common.SessionEndedRequestHandler,
        Common.IntentReflectorHandler, // make sure IntentReflectorHandler is last so it doesn't override your custom intent handlers
    )
    .addRequestInterceptors(Common.RequestInterceptor)
    .addErrorHandlers(
        Common.ErrorHandler,
    )
    .lambda();

{
  "name": "agt-mindstorms",
  "version": "1.1.0",
  "description": "A sample skill demonstrating how to use AGT with Lego Mindstorms",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Amazon Alexa",
  "license": "ISC",
  "dependencies": {
    "ask-sdk-core": "^2.6.0",
    "ask-sdk-model": "^1.18.0",
    "aws-sdk": "^2.326.0",
    "request": "^2.81.0",
    "lodash": "^4.17.11"
  }
}

/*
 * Copyright 2019 Amazon.com, Inc. or its affiliates.  All Rights Reserved.
 *
 * You may not use this file except in compliance with the terms and conditions 
 * set forth in the accompanying LICENSE.TXT file.
 *
 * THESE MATERIALS ARE PROVIDED ON AN "AS IS" BASIS. AMAZON SPECIFICALLY DISCLAIMS, WITH 
 * RESPECT TO THESE MATERIALS, ALL WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING 
 * THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.
*/

'use strict';

const Https = require('https');
const AWS = require('aws-sdk');
const Escape = require('lodash/escape');

const s3SigV4Client = new AWS.S3({
    signatureVersion: 'v4'
});

/**
 * Get the authenticated URL to access the S3 Object. This URL expires after 60 seconds.
 * @param s3ObjectKey - the S3 object key
 * @returns {string} the pre-signed S3 URL
 */
exports.getS3PreSignedUrl = function getS3PreSignedUrl(s3ObjectKey) {

    const bucketName = process.env.S3_PERSISTENCE_BUCKET;
    return Escape(s3SigV4Client.getSignedUrl('getObject', {
        Bucket: bucketName,
        Key: s3ObjectKey,
        Expires: 60 // the Expires is capped for 1 minute
    }));
};

/**
 * Builds a directive to start the EventHandler.
 * @param token - a unique identifier to track the event handler
 * @param {number} timeout - the duration to wait before sending back the expiration
 * payload to the skill.
 * @param payload - the expiration json payload
 * @see {@link https://developer.amazon.com/docs/alexa-gadgets-toolkit/receive-custom-event-from-gadget.html#start}
 */
exports.buildStartEventHandler = function (token, timeout = 30000, payload)  {
    return {
        type: "CustomInterfaceController.StartEventHandler",
        token: token,
        expiration : {
            durationInMilliseconds: timeout,
            expirationPayload: payload
        }
    };
};

/**
 *
 * Builds a directive to stops the active event handler.
 * The event handler is identified by the cached token in the session attribute.
 * @param {string} handlerInput - the JSON payload from Alexa Service
 * @see {@link https://developer.amazon.com/docs/alexa-gadgets-toolkit/receive-custom-event-from-gadget.html#stop}
 */
exports.buildStopEventHandlerDirective = function (handlerInput) {

    let token = handlerInput.attributesManager.getSessionAttributes().token || '';
    return {
        "type": "CustomInterfaceController.StopEventHandler",
        "token": token
    }
};

/**
 * Build a custom directive payload to the gadget with the specified endpointId
 * @param {string} endpointId - the gadget endpoint Id
 * @param {string} namespace - the namespace of the skill
 * @param {string} name - the name of the skill within the scope of this namespace
 * @param {object} payload - the payload data
 * @see {@link https://developer.amazon.com/docs/alexa-gadgets-toolkit/send-gadget-custom-directive-from-skill.html#respond}
 */
exports.build = function (endpointId, namespace, name, payload) {
    // Construct the custom directive that needs to be sent
    // Gadget should declare the capabilities in the discovery response to
    // receive the directives under the following namespace.
    return {
        type: 'CustomInterfaceController.SendDirective',
        header: {
            name: name,
            namespace: namespace
        },
        endpoint: {
            endpointId: endpointId
        },
        payload
    };
};

/**
 * A convenience routine to add the a key-value pair to the session attribute.
 * @param handlerInput - the handlerInput from Alexa Service
 * @param key - the key to be added
 * @param value - the value be added
 */
exports.putSessionAttribute = function(handlerInput, key, value) {
    const attributesManager = handlerInput.attributesManager;
    let sessionAttributes = attributesManager.getSessionAttributes();
    sessionAttributes[key] = value;
    attributesManager.setSessionAttributes(sessionAttributes);
};

/**
 * To get a list of all the gadgets that meet these conditions,
 * Call the Endpoint Enumeration API with the apiEndpoint and apiAccessToken to
 * retrieve the list of all connected gadgets.
 *
 * @param {string} apiEndpoint - the Endpoint API url
 * @param {string} apiAccessToken  - the token from the session object in the Alexa request
 * @see {@link https://developer.amazon.com/docs/alexa-gadgets-toolkit/send-gadget-custom-directive-from-skill.html#call-endpoint-enumeration-api}
 */
exports.getConnectedEndpoints = function(apiEndpoint, apiAccessToken) {

    // The preceding https:// need to be stripped off before making the call
    apiEndpoint = (apiEndpoint || '').replace('https://', '');

    return new Promise(((resolve, reject) => {

        const options = {
            host: apiEndpoint,
            path: '/v1/endpoints',
            method: 'GET',
            headers: {
                'Content-Type': 'application/json',
                'Authorization': 'Bearer ' + apiAccessToken
            }
        };

        const request = Https.request(options, (response) => {
            response.setEncoding('utf8');
            let returnData = '';
            response.on('data', (chunk) => {
                returnData += chunk;
            });

            response.on('end', () => {
                resolve(JSON.parse(returnData));
            });

            response.on('error', (error) => {
                reject(error);
            });
        });
        request.end();
    }));
};

Credits

Comments

Awards