LEGO MINDSTORMS Voice Challenge: Mission 1

Welcome to the world of LEGO MINDSTORMS EV3 and voice with Alexa! These instructions are part of a series created to show you how to connect your EV3 Brick to Alexa, and build custom voice-based interactions that incorporate EV3 motors and sensors. With this knowledge, you can build and submit your own creation to the LEGO MINDSTORMS Voice Challenge – Powered by Alexa. Don’t be shy bringing in other LEGO elements — let your imagination run wild!

Here is what is included in this series:

• Setup: Get your EV3 development environment setup.
• Mission 1: Get your EV3 Brick connected to a compatible Echo device, and reacting to the wake word. (you are here)
• Mission 2: Build EV3RSTORM's legs, and react to music played from your Echo device.
• Mission 3: Add arms and a cannon to EV3RSTORM, and react to voice commands using an Alexa skill you create.
• Mission 4: Give EV3RSTORM eyes (IR Sensor), and make your Alexa skill react anytime an intruder is detected.

Helpful resources

Throughout these missions, several resources will be referenced along the way. These references will come in handy as you learn more about the different ways you can create your own LEGO MINDSTORMS EV3 creation that works with Alexa:

• Alexa Gadgets Toolkit Overview
• Alexa Skills Kit Documentation
• EV3 Python API Documentation
• EV3Dev Python Robot Examples

Mission 1: React to the Alexa wake word

At this point, you should have completed the Setup Guide, where you got your EV3 Development environment set up. Nice work!

For this mission, you will get your EV3 Brick to connect to your Echo device and make it react to the wake word being detected when you speak. Whenever you say, “Alexa”, your EV3 Brick’s LEDs will flash different colors, and make different tones.

In this mission, you will:

• Create an Amazon developer account (if you don’t have one already)
• Register an Alexa Gadget
• Get your EV3 Brick connected to your Echo device
• Explore Python 3 code
• See your EV3 Brick react to Alexa’s wake word

In the end, your EV3 Brick should react like this:

Let’s get started!

Register an Alexa Gadget in the Alexa Developer Console

In order for your EV3 Brick to work with Alexa and your Echo device as an Alexa Gadget, you will need to register your gadget in the Alexa Developer Console. Registering your gadget ties your unique gadget to your account, and is required in order for your creation to work properly with Alexa.

1. Sign in to https://developer.amazon.com. Sign up for a developer account if you don't already have one.

2. Click on Developer Console in the upper-right of the page header.

3. In the top header, hover over Alexa, and click on Alexa Voice Service.

4. Click on Products. You may need to click AVS Program Requirements or Getting Started before you’re able to access the Products option.

5. Click on Create Product in the upper-right of the page.

6. Fill out the requested information:

• Name: MINDSTORMS EV3
• Product ID: EV3_01
• Product type: Alexa Gadget
• Product category: Animatronic or Figure
• Product description: Whatever you like
• Skip upload image
• Do you intend to distribute this product commercially?: No
• Is this a children’s product or is it otherwise directed to children younger than 13 years old?: No

7. Accept the services agreement and click Finish.

8. You will see a confirmation popup. Click OK.

9. You will be taken to a list of your products. Click on the gadget you just created.

10. At the top, you'll see an Amazon ID and Alexa Gadget Secret you'll need for the following steps to create your Alexa Gadget. Keep this ID and Secret private as it uniquely identifies your gadget.

You've successfully registered your Alexa Gadget!

Prep your EV3 Brick

Your EV3 Brick will be connecting to your Echo device using Bluetooth, which you'll need to enable on your Brick in order for the sample code to work correctly:

1. Using the buttons on your EV3 Brick, navigate to Wireless and Networks.

2. Select Bluetooth.

3. Toggle Powered to ON.

4. You should see a Bluetooth icon appear in the header of the interface right next to the Wifi icon.

5. From here, you can back out to the top level of your EV3 Brick using the back button on your EV3 Brick.

Note: If Bluetooth is “Not Available”, POWER OFF your Brick, power it back on, and try turning on Bluetooth again.

Once Bluetooth is enabled, you’re ready to work with some code!

At this point, you can switch over to the Visual Code Workspace you created during the setup instructions. The Explorer panel should display the mission folders that are on your computer.

You’re now ready to explore and modify files you’ll need to upload to your EV3 Brick to get the code to run. Expand the mission-01 folder and let’s take a look!

Registration and capabilities

Within the mission-01 folder, you will see an INI file, and a Python file. Open up the mission-01.ini file, which includes the following configuration details:

[GadgetSettings]
amazonId = YOUR_GADGET_AMAZON_ID
alexaGadgetSecret = YOUR_GADGET_SECRET

[GadgetCapabilities]
Alexa.Gadget.StateListener = 1.0 - wakeword

The INI (or initialization) file defines parameters for how your EV3 Brick should work as a gadget:

Gadget Settings: Specifies the Amazon ID and Alexa Gadget Secret that you received when you created your gadget in the Amazon Developer Console. It authenticates your EV3 Brick, and allows it to connect to your Echo device and Alexa.

Gadget Capabilities: Specifies what Alexa Gadget functionality your EV3 Brick supports. For this mission, your EV3 Brick is responding to the wake word, which is a part of the StateListener capability defined within the Alexa Gadgets Toolkit. There are more capabilities you can explore in the documentation.

You need to update this file with information tied to the Alexa Gadget you registered in the Amazon Developer Console:

Replace YOUR_GADGET_AMAZON_ID and YOUR_GADGET_SECRET with the ID and Secret tied to the Alexa Gadget you registered in the Amazon Developer Console.

With those changes, your EV3 Brick will be able to connect to your Echo device as an Alexa Gadget. Now, let’s take a look at the Python 3 code that will cause your EV3 Brick to connect to your Echo device and react to the wake word.

Establishing a Bluetooth connection with the Echo device

In order to detect the Alexa wake word, the EV3 Brick needs to establish a Bluetooth connection with the Echo device. Fortunately, we can leverage the AlexaGadget class provided by the Alexa Gadgets Toolkit API to accomplish this easily.

Within the mission-01.py file, you will see a MindstormsGadget class that extends the AlexaGadget class. The AlexaGadget class is invoked using the initialization method:

from agt import AlexaGadget

class MindstormsGadget(AlexaGadget):
    """
    Enabled Alexa device communication by extending from AlexaGadget
    """

    def __init__(self):
        """
        Performs Alexa Gadget initialization
        """
        super().__init__()

To indicate that we have Bluetooth connectivity, we can override two callback methods, on_connected and on_disconnected. This example prints out of the current connection state, but you could also tie the state to a physical outlet, like the LEDs on your EV3 Brick:

def on_connected(self, device_addr):
        """
        Gadget connected to the paired Echo device.
        :param friendly_name: the friendly name of the gadget that has connected to the Echo device
        """
        self.leds.set_color("LEFT", "GREEN")
        self.leds.set_color("RIGHT", "GREEN")
        print("{} connected to Echo device".format(self.friendly_name))

    def on_disconnected(self, device_addr):
        """
        Gadget disconnected from the paired Echo device.
        :param friendly_name: the friendly name of the gadget that has disconnected from the Echo device
        """
        self.leds.set_color("LEFT", "BLACK")
        self.leds.set_color("RIGHT", "BLACK")
        print("{} disconnected from Echo device".format(self.friendly_name))

Detecting the wake word

Once you establish a Bluetooth connection, you need to detect the wake word event. Since we have declared the Alexa.Gadget.StateListener capability, the callback method on_alexa_gadget_statelistener_stateupdate will give us the state data. By overriding this method, you can check for the wake word state:

def on_alexa_gadget_statelistener_stateupdate(self, directive):
        """
        A state listener callback where the directive contains the updated state information.
        :param directive: contains a payload with the updated state information from Alexa
        """
        for state in directive.payload.states:
            if state.name == 'wakeword':
                """
                Perform wakeword detection actions
                """

Reacting to the wake word

Once you detect the wake word, you can react to it with sound and lights using the ev3dev API. First, you need to initialize the resources – in this case, Sound and Leds. Then, specify that the LEDs should turn on or off based on the wake word states:

from ev3dev2.sound import Sound
from ev3dev2.led import Leds

class MindstormsGadget(AlexaGadget):

    def __init__(self):
    """
    Performs Alexa Gadget initialization routines and ev3dev resource allocation.
    """
        super().__init__()
        self.leds = Leds()
        self.sound = Sound()

    def on_alexa_gadget_statelistener_stateupdate(self, directive):
        """
        Listens for the wakeword state change and react by turning on the LED.
        :param directive: contains a payload with the updated state information from Alexa
        """
        color_list = ['BLACK', 'AMBER', 'YELLOW', 'GREEN']
        for state in directive.payload.states:
            if state.name == 'wakeword':
            if state.value == 'active':
                print("Wake word active - turn on LED")
                self.sound.play_song((('A3', 'e'), ('C5', 'e')))
                for i in range(0, 4, 1):
                    self.leds.set_color("LEFT", color_list[i], (i * 0.25))
                    self.leds.set_color("RIGHT", color_list[i], (i * 0.25))
                    time.sleep(0.25)
            elif state.value == 'cleared':
                print("Wake word cleared - turn off LED")
                self.sound.play_song((('C5', 'e'), ('A3', 'e')))
                for i in range(3, -1, -1):
                    self.leds.set_color("LEFT", color_list[i], (i * 0.25))
                    self.leds.set_color("RIGHT", color_list[i], (i * 0.25))
                    time.sleep(0.25)

Executing the code

The Python 3 code runs by invoking the MindstormsGadget main method at the bottom of the Python file:

if __name__ == 'main':
    MindstormsGadget().main()

You will find that the Python 3 code for the EV3 Brick in other missions follows a very similar pattern. Let’s run the code to see what it does!

Run the Python code

After reviewing the code, you can run it and see how your EV3 Brick reacts. As noted above, the code you will run will enable your EV3 Brick react to your Echo device detecting the Alexa wake word by illuminating the LEDs on the EV3 Brick and playing tones.

Note: The wake word is the word you say to get Alexa's attention. The default is Alexa, but you may have changed it to another option, such as Computer, Echo, or Amazon.

To see this in action, follow these steps:

1. Make sure VS Code is connected to your EV3 Brick. You should see a green dot next to your EV3 Brick’s name in the EV3 DEVICE BROWSER.

2. Sync your alexa-gadgets-mindstorms folder in your computer workspace to your EV3 Brick. Click on the Send workspace to device button next to the EV3DEV DEVICE BROWSER text that shows up when you hover over the text. When you click this button, you will see the status of the files being copied in the bottom-right of VS Code.

When you click this button, you will see the status of the files being copied in the bottom-right of VS Code.

3. Once the files have copied over to your EV3 Brick, you can run the Python code by navigating to the Python file in the EV3 DEVICE BROWSER, right-click on the Python file, and select Run.

Note: Once the Mission 1 code has been copied to your EV3 Brick, and run once from your computer, you can also run it using the Brick’s interface. This a great way to run the code without having to use your computer! In the Brick’s interface select File Browser > alexa-gadgets-mindstorms > mission-01 > mission-01.py*. You should see an asterisk next to the file name, which means you can run it directly from the EV3 Brick interface.

4. You will see a debug console display from the bottom of the VS Code window, and it will take a couple minutes for the program to boot up.

5. Once the program starts, you should hear your EV3 Brick play some tones, and see a prompt in the debug console telling you the friendly name of your gadget, which will look like Gadget000. This means your gadget is in pairing mode.

You will also see a message on the screen of your EV3 Brick.

6. You're now ready to pair your EV3 Brick to your Echo device. This pairing process will depend on whether or not your Echo device has a screen. See these instructions for how to pair.

Note: If you run into connectivity issues, you can try unpairing your EV3 Brick from your Echo device by following the Unpairing your EV3 Brick instructions below. You may also want to try forgetting your EV3 Brick from the Bluetooth settings of your Echo device before pairing again.

7. Once your device is paired, you should see an acknowledgement that your gadget has paired in the debug console.

You will also see a message on your EV3 Brick's screen.

Now comes the fun part! Try the following to see how your EV3 Brick reacts:

Say the wake word for your device

Try saying the wake word to your Echo device, e.g. "Alexa". When the Echo device hears the wake word, the LEDs on your EV3 Brick should turn on and cycle through colors, while two tones play from low to high.

When the wake word detection clears, the LEDs should cycle colors in reveres order to an off state, while two tones play from high to low.

In the debug console, and on your EV3 Brick’s screen, you should also see statements printed telling you the state of the LED:

You did it! Your EV3 Brick is now connected to your Echo device and reacting to Alexa. You can always stop the program by clicking the Stop button at the top of the screen in Debug mode, or pressing the back button on your EV3 Brick.

Note: For the Bluetooth connection, if at any time you want to disassociate your EV3 Brick from your Echo device, you can follow the Unpairing your EV3 Brick instructions below.

Unpairing your EV3 Brick

There may be situations where you’d like to unpair your EV3 Brick from your Echo device, including:

• Pairing to another Echo device
• Troubleshooting connectivity issues

There is an unpair.sh script included within the alexa-gadgets-mindstorms folder. When you run this script, it will clear the stored Bluetooth address of the Echo device, and prompt you to pair to an Echo device the next time you run any of the missions.

1. To run the script, right-click on the unpair.sh shown in the EV3 DEVICE BROWSER, and select Run:

2. When the script starts, the Debug Console will pop up. Your EV3 Brick should play some tones once the script runs.

3. Once the script completes, you will see a message in the Debug Console:

After running this script, you will be asked to pair your EV3 Brick the next time you run any of the missions, or your own, code. Keep in mind that you will also need to forget your EV3 Brick’s connection on the Echo device, otherwise the Echo device will still think it’s paired and you will not be able to repair to it.

Other things to explore

Reacting to the wake word is one of the things that you can do with your creation by leveraging the Alexa Gadgets Toolkit. You can also do things like react to notifications, timers, music, and more. Learn more by reviewing the documentation, or by referencing the INI and Python file within the Alexa Gadgets Kitchen Sink example for Raspberry Pi.

You can also explore other capabilities of ev3dev by referring to the documentation.

What to do next

In Mission 2, you’ll build part of EV3RSTORM and react to music played from your Echo device!

Go to Mission 2