Overview
Things
Story

Putting It All Together
On RPI3 open terminal and start Mosquitto
192.168.1.113:1880

Published February 6, 2020 © MIT

Lego EV3, Amazon ALEXA, RPi3 - 3 gadgets talk to each other

Lego EV3, Amazon Alexa Dot 3 and Raspberry Pi3 talking together, voice commands from Alexa cloud to RPI3, MQTT communications to Lego robot

IntermediateFull instructions provided10 hours1,435

Lego EV3, Amazon ALEXA, RPi3 - 3 gadgets talk to each other

Things used in this project

Hardware components

LEGO Mindstorms EV3 Programming Brick / Kit

Amazon Alexa Echo Dot

Itead Raspberry Pi 3 Model B Starter Pack RPI 3 Starter Kit

Software apps and online services

Amazon Alexa Alexa Skills Kit

Microsoft Visual Studio Code Extension for Arduino

Story

My project video on youtube.

I have tried to make this project as simple as possible, to enable children to reproduce my idea and make their own robots. Intents is: visual programming when it is possible.

1. Build your robot, I have built simple EXPLOR3R, from this instructions:

http://robotsquare.com/2015/10/06/explor3r-building-instructions/

2. prepare software for robot, read carefully excellent instructions for setup

https://www.hackster.io/alexagadgets/lego-mindstorms-voice-challenge-setup-17300f

Install EV3Dev: ev3dev is open source software that allows you to program your EV3 Brick using Python 3.

Install and Set up Visual Studio Code on Your PC or laptop that you can program Lego robot using Python 3.

Install nice tools on your computer to help you to communicate with robot and rpi3, and transfer files between them.

putty - https://www.putty.org/

WinSCP - https://winscp.net/eng/download.php

After you prepare SD card with EV3Dev and first start your Lego robot ( must wait a while, boot process is little long), you can try to connect with putty to robot.

You must know IP address of robot ( my is 192.168.1.84), activate wi-fi network on you robot using wi-fi dongle, you must have internet to download software for robot :

https://www.ev3dev.org/docs/networking/

https://www.ev3dev.org/docs/tutorials/connecting-to-ev3dev-with-ssh/

Connecting to Ev3dev Using SSH

You have now full working linux system on Lego Ev3, you can prepare it with some commands:

robot@ev3dev:~ sudo pip install paho-mqtt

I will use mqtt library in my python program, I can communicate over wifi to rpi3 running mosquitto broker, and Node red.

3. prepare software for raspberrypi3

Nice tutorial You can find here, you must install Rasbian as OS, and then Mosquitto.

From terminal you must install node-red-contrib-chatbot :

pi@raspberrypi:~ $ cd ~/.node-red
pi@raspberrypi:~/.node-red $ npm install node-red-contrib-chatbot

Wait few moment to the end..

+ node-red-contrib-chatbot@0.17.8
added 1305 packages from 1437 contributors and audited 16686 packages in 1581.321s
pi@raspberrypi:~/.node-red $

Most important tools on PC for raspberry: putty, winscp, VNC viewer

You can start mosquitto broker and monitor traffic with command:

mosquitto -v

You must install also Ngrok ( secure introspectable tunnels to localhost) with:

sudo npm install --unsafe-perm -g ngrok

Later you will start ngrok with:

ngrok http 127.0.0.1:1880

4.Create your Alexa skill. Sign in with your Amazon developer account at developer.amazon.com. If you don't have an account yet, you can create new one.

Then from the top menu select Alexa -> Alexa Skills Kit.

Press Create Skill button and provide a name and default language for your new Alexa skill, enter skill name and chose Custom model and Provision your own.

Give skill invocation name: lego robot

After that make some custom Intents: movef, moveb, movel, mover, stop

After that you can edit intents to add aditional words ( commands ), for me:

movef: move forward, forward, go

moveb: move back, back, go back

movel: move left, left, turn left

mover: move right, right, turn right

stop: lego stop ( you can not use only stop because this is command to close conversation with robot )

After editing your skill save model, and build.

And last step and very important: ENDPOINT

1. here put address which you can see when you start ngrok in rpi3 terminal

in this example something like this:

https://5b581529.ngrok.io/redbot/alexa/test

this numbers and letters you must read from your instance of ngrok and will be valid for 8 hours, after that you must start ngrok again and put here new value.

After that your skill is complete, and you can test it in alexa simulator:

You can speak commands if you have microphone on your pc or type it. You will see something like this:

All is OK !!! Now You can test it on your alexa dot3 vith voice commands.

Putting It All Together:

1. Turn on lego robot, after booting switch on wifi connection. With putty start

phyton program mqtt-11.py :

robot@ev3dev:~/mqtt$ sudo python3 mqtt-11.py

Then connect with vnc to rpi.

On RPI3 open terminal and start Mosquitto:

mosquitto -v

In another terminal start ngrok:

ngrok http 127.0.0.1:1880

Open alexa developer console and edit ENDPOINT, save and build model.

On RPI3 start NODE RED CONSOLE which will start NODE RED FLOW:

Function node is main part with definicion of: speed, time and direction

This is only part of flow which send commands from RPI to LEGO robot over MQTT. You can open it on PC, mobile phone or tablet in Internet browser with adress:

192.168.1.113:1880

You also have nice dashboard with command buttons to test robot moving.

You can open it on PC, mobile phone or tablet in Internet browser with adress:

192.168.1.113:1880/ui

Another part is connection with alexa cloud:

Thanks to this man this is working wonderful:

https://github.com/guidone/node-red-contrib-chatbot/wiki

Alexa receiver node is very simple:

Most important is here RULES NODE, with multiple outputs, every output for single command from alexa skill.

Output 6 is message on start

Every output is connected to Speech node where you define answer to command. It is also connected as input to function node on MQTT part to give MQTT command to robot.

Very useful tool for PC is MQTT.fx where you see all comunications

That is all for now, I will try to integrate lego sensors in part 2.

Code

# Copyright 2019 Amazon.com, Inc. or its affiliates.  All Rights Reserved.
# 
# You may not use this file except in compliance with the terms and conditions 
# set forth in the accompanying LICENSE.TXT file.
#
# THESE MATERIALS ARE PROVIDED ON AN "AS IS" BASIS. AMAZON SPECIFICALLY DISCLAIMS, WITH 
# RESPECT TO THESE MATERIALS, ALL WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING 
# THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.

import paho.mqtt.client as mqtt
import threading
import time
import ev3dev.ev3 as ev3
import json

# This is the Subscriber

def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))
    client.subscribe("ev3/command")

def on_message(client, userdata, msg):
    print("def on message")
    print("msg =",msg)
    print("msg.payload =",msg.payload)
    decoded_msg = msg.payload.decode("utf-8")
    print("decoded_msg =",decoded_msg)
    payload = json.loads(decoded_msg)
    print("payload =",payload)
    if 'voice' in payload:
        ev3.Sound.speak(payload['voice']).wait()
    if 'robotControl' in payload:
        arm_control(payload)
    if 'moveRobot' in payload:
        move_robot(payload)

# Move robot (direction and speed is defined in payload)
def move_robot(payload):
  print("def move robot")
  if 'direction' and 'speed' and 'time' in payload['moveRobot']:
        speed = int(payload['moveRobot']['speed'])
        duration = int(payload['moveRobot']['time'])
        direction = payload['moveRobot']['direction']
        print("speed =",speed)
        print("duration =",duration)
        print("direction =",direction)
        left = ev3.LargeMotor('outB')
        right = ev3.LargeMotor('outC')
        speed_l, speed_r = 0, 0
        if direction == 'fwd':
            speed_l = speed
            speed_r = speed
        elif direction == 'left':
            speed_l = -speed
            speed_r = speed
        elif direction == 'right':
            speed_l = speed
            speed_r = -speed
        elif direction == 'back':
            speed_l = -speed
            speed_r = -speed
        elif direction == 'stop':
            speed_l = 0
            speed_r = 0
        else:
            ev3.Sound.speak('Please tell me direction').wait()

        if left.connected and right.connected:
            left.run_timed(time_sp=duration, speed_sp=speed_l)
            right.run_timed(time_sp=duration, speed_sp=speed_r)

# When motor is stopped, its `state` attribute returns empty list.
# Wait until both motors are stopped:
        while any(left.state) or any(right.state):
            time.sleep(0.1)

# Move robot arm (Speed and duration is defined in payload)
def arm_control(payload):
    m = ev3.MediumMotor('outA')
    speed = int(payload['robotControl']['speed'])
    duration = int(payload['robotControl']['time'])
    if m.connected:
        m.run_timed(time_sp=duration, speed_sp=speed)
        while any(m.state):
            time.sleep(0.1)
        print("done")
    else:
        print("Motor not connected")

# On button press, send payload (MQTT)
def publish_data(run_event):
    while run_event.is_set():
        ts = ev3.TouchSensor()
#       if ts.connected and ts.value() == 1:
#           try:
#               ev3.Leds.set_color(ev3.Leds.LEFT, (ev3.Leds.GREEN, ev3.Leds.RED)[ts.value()])
#               payload = ("{\"touchSensorValue\": \"%d\"}" % ts.value
#              client.publish(ev3.publisher_topic, payload)
#               print("Data sent")
#           except:
#               print("Err")
#       else:
#           ev3.Leds.set_color(ev3.Leds.LEFT, (ev3.Leds.GREEN, ev3.Leds.RED)[ts.value()])



# Make new MQTT client
client = mqtt.Client()
client.connect("192.168.1.113",1883,60)

client.on_connect = on_connect
client.on_message = on_message

# Start 
client.loop_forever()

{
    "interactionModel": {
        "languageModel": {
            "invocationName": "lego robot",
            "intents": [
                {
                    "name": "AMAZON.FallbackIntent",
                    "samples": []
                },
                {
                    "name": "AMAZON.CancelIntent",
                    "samples": []
                },
                {
                    "name": "AMAZON.HelpIntent",
                    "samples": []
                },
                {
                    "name": "AMAZON.StopIntent",
                    "samples": []
                },
                {
                    "name": "AMAZON.NavigateHomeIntent",
                    "samples": []
                },
                {
                    "name": "movef",
                    "slots": [],
                    "samples": [
                        "go",
                        "forward",
                        "move forward"
                    ]
                },
                {
                    "name": "moveb",
                    "slots": [],
                    "samples": [
                        "go back",
                        "back",
                        "move back"
                    ]
                },
                {
                    "name": "movel",
                    "slots": [],
                    "samples": [
                        "turn left",
                        "left",
                        "move left"
                    ]
                },
                {
                    "name": "mover",
                    "slots": [],
                    "samples": [
                        "turn right",
                        "right",
                        "move right"
                    ]
                },
                {
                    "name": "stop",
                    "slots": [],
                    "samples": [
                        "stop lego robot"
                    ]
                }
            ],
            "types": []
        }
    }
}

[{"id":"40239d02.fe0584","type":"tab","label":"LEGO EV3 ALEXA","disabled":false,"info":""},{"id":"ff6b476d.5e6fb8","type":"debug","z":"40239d02.fe0584","name":"","active":true,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload","targetType":"msg","x":300,"y":620,"wires":[]},{"id":"265713a6.a88efc","type":"chatbot-telegram-receive","z":"40239d02.fe0584","bot":"b069f500.581368","botProduction":"","x":100,"y":540,"wires":[["ff6b476d.5e6fb8","69847dbf.a64744"]]},{"id":"74639f41.16df3","type":"chatbot-telegram-send","z":"40239d02.fe0584","bot":"b069f500.581368","botProduction":"","track":false,"passThrough":false,"outputs":0,"x":510,"y":540,"wires":[]},{"id":"69847dbf.a64744","type":"chatbot-message","z":"40239d02.fe0584","name":"message","message":[{"message":"Hello from rpi3"}],"answer":false,"silent":false,"x":310,"y":560,"wires":[["74639f41.16df3"]]},{"id":"7b338e0a.74c17","type":"chatbot-alexa-receive","z":"40239d02.fe0584","bot":"ff4f569b.436da8","botProduction":"","x":120,"y":40,"wires":[["cb0bf8e8.a4d158","fa477ea.e4ece8"]]},{"id":"51f7da44.bed6b4","type":"chatbot-alexa-send","z":"40239d02.fe0584","bot":"ff4f569b.436da8","botProduction":"","track":false,"passThrough":false,"outputs":0,"x":800,"y":200,"wires":[]},{"id":"165d22f9.78a1cd","type":"chatbot-message","z":"40239d02.fe0584","name":"turn right","message":[{"message":"Lego robot turned right"}],"answer":false,"silent":false,"x":520,"y":220,"wires":[["51f7da44.bed6b4"]]},{"id":"cb0bf8e8.a4d158","type":"debug","z":"40239d02.fe0584","name":"","active":true,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload","targetType":"msg","x":360,"y":40,"wires":[]},{"id":"fa477ea.e4ece8","type":"chatbot-rules","z":"40239d02.fe0584","name":"","rules":[{"type":"isIntentName","intent":"movef"},{"type":"isIntentName","intent":"moveb"},{"type":"isIntentName","intent":"movel"},{"type":"isIntentName","intent":"mover"},{"type":"isIntentName","intent":"stop"},{"type":"messageEvent","event":"start"},{"type":"messageEvent","event":"endSession"},{"type":"catchAll"}],"outputs":8,"x":130,"y":160,"wires":[["42e1278f.661018","15b1f915.e24647"],["db3fecce.9c808","eeaef051.c0d7"],["3fa8fcf8.e34864","5c92a003.130f2"],["165d22f9.78a1cd","4f6c99fb.51b4a8"],["9d1b8650.7e87c8","7965c969.f8cfb8"],["652d05f.a72d1fc"],["be0e70b5.89419"],["829c659.7347118"]]},{"id":"42e1278f.661018","type":"chatbot-message","z":"40239d02.fe0584","name":"forward","message":[{"message":"Lego robot moved forward"}],"answer":false,"silent":false,"x":520,"y":100,"wires":[["51f7da44.bed6b4"]]},{"id":"db3fecce.9c808","type":"chatbot-message","z":"40239d02.fe0584","name":"back","message":[{"message":"Lego robot move back"}],"answer":false,"silent":false,"x":510,"y":140,"wires":[["51f7da44.bed6b4"]]},{"id":"3fa8fcf8.e34864","type":"chatbot-message","z":"40239d02.fe0584","name":"turn left","message":[{"message":"Lego robot turned left"}],"answer":false,"silent":false,"x":520,"y":180,"wires":[["51f7da44.bed6b4"]]},{"id":"9d1b8650.7e87c8","type":"chatbot-message","z":"40239d02.fe0584","name":"stop","message":[{"message":"lego robot stopped"}],"answer":false,"silent":false,"x":510,"y":260,"wires":[["51f7da44.bed6b4"]]},{"id":"652d05f.a72d1fc","type":"chatbot-message","z":"40239d02.fe0584","name":"first message","message":[{"message":"Hello from lego robot "}],"answer":false,"silent":false,"x":500,"y":300,"wires":[["51f7da44.bed6b4"]]},{"id":"be0e70b5.89419","type":"chatbot-message","z":"40239d02.fe0584","name":"last message","message":[{"message":"Good bue  from lego robot "}],"answer":false,"silent":false,"x":490,"y":340,"wires":[["51f7da44.bed6b4"]]},{"id":"15b1f915.e24647","type":"link out","z":"40239d02.fe0584","name":"","links":["74fd12d1.ef638c"],"x":295,"y":340,"wires":[]},{"id":"eeaef051.c0d7","type":"link out","z":"40239d02.fe0584","name":"","links":["6ecf7e64.e422e"],"x":295,"y":380,"wires":[]},{"id":"5c92a003.130f2","type":"link out","z":"40239d02.fe0584","name":"","links":["ea2c6a1e.36f778"],"x":295,"y":420,"wires":[]},{"id":"4f6c99fb.51b4a8","type":"link out","z":"40239d02.fe0584","name":"","links":["7572ffe3.ade0a"],"x":295,"y":460,"wires":[]},{"id":"7965c969.f8cfb8","type":"link out","z":"40239d02.fe0584","name":"","links":["268f3d78.6b1bb2"],"x":175,"y":380,"wires":[]},{"id":"829c659.7347118","type":"chatbot-message","z":"40239d02.fe0584","name":"dont understand ","message":[{"message":"I do not understand your command !\nPlease say valid robot commands:\n go, back,left,right,stop lego robot"}],"answer":false,"silent":false,"x":510,"y":380,"wires":[["51f7da44.bed6b4"]]},{"id":"9568dc5e.6ea7b","type":"inject","z":"40239d02.fe0584","name":"","topic":"sensor","payload":"true","payloadType":"bool","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":470,"y":460,"wires":[["8ac0f9f2.bf772"]]},{"id":"8ac0f9f2.bf772","type":"chatbot-message","z":"40239d02.fe0584","name":"sensor message","message":[{"message":"message from lego robot:\nto close to barrier !"}],"answer":false,"silent":false,"x":650,"y":460,"wires":[["51f7da44.bed6b4"]]},{"id":"b069f500.581368","type":"chatbot-telegram-node","z":"","botname":"malamlaka_bot","usernames":"","providerToken":"","polling":"1000","store":"","log":"","parseMode":"","debug":false,"webHook":"","connectMode":"polling"},{"id":"ff4f569b.436da8","type":"chatbot-alexa-node","z":"","botname":"alexa","usernames":"","store":"","log":"/logs/chatbot-alexa.log","debug":true}]

Credits

Darko Vranesa

5 projects • 6 followers

Lego EV3, Amazon ALEXA, RPi3 - 3 gadgets talk to each other

Things used in this project

Hardware components

Software apps and online services

Story

Putting It All Together:

On RPI3 open terminal and start Mosquitto:

192.168.1.113:1880

Schematics

Networked gadget schematic diagram

Code

mqqt-12.py

model.json

Nodered flow Lego EV3 Alexa

Credits

Darko Vranesa

Comments

Embed the widget on your own site

Lego EV3, Amazon ALEXA, RPi3 - 3 gadgets talk to each other

Lego EV3, Amazon ALEXA, RPi3 - 3 gadgets talk to each other

Things used in this project

Hardware components

Software apps and online services

Story

Putting It All Together:

On RPI3 open terminal and start Mosquitto:

192.168.1.113:1880

Schematics

Networked gadget schematic diagram

Code

mqqt-12.py

model.json

Nodered flow Lego EV3 Alexa

Credits

Darko Vranesa

Comments

Related channels and tags