MajorTom: Alexa Voice Controlled ARDrone 2.0

See what's new at Remyx.ai

Motivation

Our previous project with the ARDrone 2.0 helped us learn about a new project. Our current Alexa and Arduino Smart Home Challenge entry introduced us to the skills needed to create a working demo for a voice-controlled drone.

Here is what we came up with:

Below, we outline our process.

Pi Software Setup

To set up the Pi with the necessary software, install flask-ask:

$ sudo pip install flask-ask

You'll also need the library that controls the ARDrone via WiFi. Clone this repo and run the install.sh script provided.

Setting Up the Alexa Skill and App

We created a skill that was linked to a server running off the raspberry pi. Flask is an easy to use python library to serve an application. Using flask-ask, we can configure the server to communicate with our Alexa skill. We use Ngrok to serve the application externally and gives us an https link we'll need for our Alexa skill. Here's a snippet of code for the application. The full app.py script will be attached below.

#!/usr/bin/env python
from flask import Flask
from flask_ask import Ask, statement
import ardrone
import time
app = Flask(__name__)
ask = Ask(app, '/')
drone = ardrone.ARDrone()
time.sleep(5)
drone.reset()
@app.route('/')
def home():
    return 'Ground Control to Major Tom'
@ask.intent('reset')
def reset():
    drone.reset()
    speech_text = 'resetting'
    return statement(speech_text)

To run this application, just type:

python app.py

For this project, we used ngrok to serve the application over https. We used a reserved subdomain name so the link can be consistent with the Amazon Alexa skill configuration. To serve the application, run:

./ngrok http -subdomain=<your-subdomain-here> 5000

Because the Raspberry Pi joins the ARDrone's WiFi access point, we use the Hologram Nova USB modem as a cellular backdoor enabling Alexa to communicate with the flask-ask server.

We found that the application worked smoothly using hologram nova with a monthly data plan of 250mb. You can set this up using the hologram dashboard and activating a sim card.

Setting up the Alexa skill, navigate to the Amazon developer dashboard and log in. Click on Alexa and get started with Alexa Skill kit. Then create a new skill by clicking Add a new skill. Start completing the tabs on the left following their instructions.

Follow the instructions given by the gui.

• Under the Interaction Model, fill out the Intent Schema box with the following:

{
 "intents": [
   {
     "intent": "reset"
   },
   {
     "intent": "takeoff"
   },
   {
     "intent": "land"
   },
   {
     "intent": "hover"
   },
   {
     "intent": "goForward"
   },
   {
     "intent": "goBackward"
   },
   {
     "intent": "goLeft"
   },
   {
     "intent": "goRight"
   },
   {
     "intent": "AMAZON.PreviousIntent"
   },
   {
     "intent": "AMAZON.NextIntent"
   },
   {
     "intent": "AMAZON.HelpIntent"
   },
   {
     "intent": "AMAZON.ScrollUpIntent"
   },
   {
     "intent": "AMAZON.ScrollLeftIntent"
   },
   {
     "intent": "AMAZON.ScrollDownIntent"
   },
   {
     "intent": "AMAZON.ScrollRightIntent"
   },
   {
     "intent": "AMAZON.PageUpIntent"
   },
   {
     "intent": "AMAZON.PageDownIntent"
   },
   {
     "intent": "AMAZON.MoreIntent"
   },
   {
     "intent": "AMAZON.NavigateSettingsIntent"
   },
   {
     "intent": "AMAZON.StopIntent"
   }
 ]
}

• In the Sample Utterances box, write:

reset reset
takeoff takeoff
takeoff liftoff
takeoff rise up
takeoff blast off
takeoff rise
land land
land come down to earth
land come down
land get on the ground
land ground control
hover hover
hover stay put
hover stay where you are
goForward move forward
goForward go forward
goForward come
goBackward go back
goBackward go backwards
goBackward move backwards
goBackward get back
goLeft move left
goLeft go left
goLeft turn left
goRight go right
goRight move right
goRight turn right

• In the Configuration tab, make sure to choose the service end point to HTTPS. Copy your https link and paste it in the Default box underneath. Account linking can be left to No.
• In the SSL Certificate pick the middle option, "My development endpoint is a sub-domain of a domain that has a wildcard certificate from a certificate authority".
• The Test tab will allow you to test the new skill by typing in one of your sample commands.

Finish filling out the last two tabs until all the check marks are green. Then launch your skill with the beta testing feature. This allows you to host your skill on any echo device before publishing it. Follow the instructions on the email link to install the skill on your Echo device.

Run

To run the entire application, first make sure your drone is on and its wifi connecting is up. On the Raspberry Pi server, connect to your ARDrone's network. In the terminal, connect to Hologram's network running the following command:

$ sudo hologram network connect

Once the blue LED on the Nova is solid and this process has finished, you are on the cellular network. On a separate tab, run the ngrok command we mentioned earlier:

$ ./ngrok http -subdomain=<your-subdomain-here> 5000

Finally, in another tab, run the application

$ python app.py

If the drone's lights are green, you're good to fly. If not, ask Alexa to reset it by saying, "Alexa, ask MajorTom to reset."

Now try to command your drone to fly! Say, "Alexa, ask MajorTom to blast off."

🎶 I think my spaceship knows which way to go 🎶

kindbot.io