Integrating Alexa & Ultra96

Introduction

Many IoT developments these days, integrate and operate with Amazon's Alexa to provide voice interaction.

In this project we are going to look at how we can integrate Alexa with our Ultra96 design. This means we can develop applications which leverage the programmable logic and Arm cores of the Ultra96 and then integrate the solution with Alexa.

To integrate our application with Alexa we are going to use a number of Amazon Web Services (AWS) along with running Python applications on the Ultra96.

As such to implement this project you will need a AWS account.

Architecture

This project will use a number of AWS functions both at the edge and in the cloud. The architecture of the solution is outlined in the diagram below:

Once Alexa receives the invocation and intent word, we will configure the Alexa skill to send a JSON request to a AWS Lambda service. AWS Lambda will then access a AWS S3 bucket and read a file which has been uploaded by the Ultra96 with the current measured temperature.

AWS Lambda will then process the file and generate a JSON response which Alexa interprets and voices the temperature.

Of course while this example shows the communication path in one direction, we could use the same approach to control actions in the Ultra96.

Creating the Alexa Skill

The first thing we need to do is create a Alexa skill we do this by going to the Alexa developer console here:

https://developer.amazon.com/alexa

Once logged into the console we need to create a new skill - click on the Create Skill button.

Click on the created skill and we can step through the skill creation process, the first step is to create the invocation word. This is what we say to Alexa to start the interaction with our Skill - I used "Ultra Ninety Six" for this application.

The next step is to configure the intents, we can have multiple intents but for this example we will have one which is the intent for requesting the temperature.

Once we have set up the invocation and intents, the next step is to set up the end point.

This is where we set up up the interface with AWS Lambda here will, use the skill ID and the AWS Lambda settings.

Setting up the S3 Bucket

Before we set up the AWS Lambda we need to be able to store files on the S3 Bucket. If we do not have a S3 bucket we need to configure one, sign into the AWS console and from storage select S3.

When you open the S3 instance you will see the following screen enabling you to create a bucket.

For this example I called my bucket the MicroZed Chronicles:

The final step is to generate the keys to enable us to access bucket. Click on your user name on the top right and select "My Security Credentials."

Under here you can create a Key ID and Secret Access Key.

With the S3 Bucket and access keys created the next step is to create the AWS Lambda solution.

AWS Lambda

Within AWS Lambda, we need to create a function, click on the create function button.

Once the function is created we need to edit it - I called my function Lambda temperature.

The first thing we need to do in the Lambda function is to add in the triggers. In this case these are Alexa and S3 Bucket.

The next step is to add in the Node.Js code for the Lambda temperature, I used the code below - Remember to replace the KEY and SECRET KEY with your credentials.

exports.handler =(event, context, callback) => {
  
    var prompt = ''; 
    var myParams = { Bucket:'microzedchronicles', Key: 'data.txt' };
    varAWS = require('aws-sdk'); var s3 = new AWS.S3();
    AWS.config.update({
    accessKeyId:"KEY",
    secretAccessKey:"SECRET KEY"
    });
  
s3.getObject(myParams,function(err, data) { 
    if(err) { 
        console.log(err, err.stack); }
    else { 
        var fileText =data.Body.toString(); 
        prompt += fileText;
        callback(null,buildResponse(fileText));
        }
    });
  
};
  
functionbuildResponse(response) {
return {
    version: '1.0',
    response: {
        outputSpeech: {
            type: 'PlainText',
            text: response,
        },
        shouldEndSession: true,
        },
    sessionAttributes: {},
    };
}

Within Lambda we can set up a test invocation to test our responses, and remove any errors without the need to go via Alexa.

The final stage of the application is to create the Ultra96 Application - For this I based the application of the original image which comes with the Ultra96.

Creating the Ultra96 Environment

To measure the temperature the Ultra96 will be connected to the grove starter kit for 96 boards and the temperature sensor as shown below.

Connect the Ultra96 to a network which enables internet access and SSH into it, we need to download the python packages which allow us to work with the AWS and the S3 Bucket we created.

To interface with the Grove StarterKit temperature sensor and the AWS S3 bucket we need to create four files

• Make File
• Bash File to run the application
• Python file to create the file and push it to the S3 Instance
• Ardunio file to upload to the Grove starter kit and read the temperature sensor

I created these files on my lap top and uploaded them using WinSCP to

/home/root/alexa

The contents of the files are

Makefile

MONITOR_PORT=/dev/ttyS2
include /usr/share/arduino/Arduino.mk
run: upload 
	python ada.py

run_me.sh

#!/bin/bash
export PYTHONPATH=$PYTHONPATH:/usr/lib/python3.5/site-packages
cd /home/root/alexa
make run

ada.py

#from Adafruit_IO import Client
import serial
import boto3
#aio = Client('USER', 'CHANGE') 
ard = serial.Serial('/dev/ttyS2', 9600)
#s3 = boto3.resource('s3')
s3 = boto3.client(
   's3',
   # Hard coded strings as credentials, not recommended.
   aws_access_key_id='KEY ID ',
   aws_secret_access_key='SECRETKEY'
)
#s3 = session.resource('s3')
#s3.create_bucket(Bucket= 'microzedchronicles')
if __name__ == '__main__':
	print("Welcome to the Humidity & Temperature reader!!!")
	try:
		while True:
			ardOut = ard.readline()
			ardHumid = ardOut.split('Temperature')[0]
			ardTemp = ardOut.split('Temperature:')[1]
			#aio.send('test', ardTemp)
			print "Temperature" + str(ardTemp)
			f = open('data.txt','w')
			f.write(str(ardTemp))
			f.close()
			s3.upload_file('data.txt','microzedchronicles','data.txt')
	except KeyboardInterrupt:
		print("CTRL-C!! Exiting...")

read_dht.ino

#include "DHT.h"
DHT dht(A0, DHT11);
void setup()
{
       Serial.begin(9600);
       dht.begin();
}
void loop()
{
       float h = dht.readHumidity();
       float t = dht.readTemperature();
       // check if valid, if NaN (not a number) then something went wrong!
       if (isnan(t) || isnan(h)) {
               Serial.println("Failed to read from DHT");
               return;
       }
       Serial.print("Humidity: ");
       Serial.print(h);
       Serial.print(" %\t");
       Serial.print("Temperature: ");
       Serial.print(t);
       Serial.println(" *C");
       delay(2000);
}

To use the file above I also included the dht.h and dht.cpp files, the complete upload directory is available on my github

We can run the application by running run_me.sh on our Ultra96

To test the application we can go back to our Alexa skill and select the test application on the test applications tab

You will be able to see both the output and response JSON files

When I put this all together I was able to talk to Alexa using the invocation word and obtain the temperature as monitored by my Ultra96.

If you desire to release the Alexa Skill to the library you can then push forward with the Distribution and Certification tasks.

See previous projects here.

Additional Information on Xilinx FPGA / SoC Development can be found weekly on MicroZed Chronicles.