Nick's House - Echo Pi Weather

Completed project video with demonstration

Background

After recently receiving my Amazon Echo I've been itching to build something for it, so this competition came at just the right time! I did manage to get a skill published which assists in getting your AWS DevOps Professional Certification by asking you sample questions (available at http://alexa.amazon.com/spa/index.html#skills/dp/B01HCJSXKK/?ref=skill_dsk_skb_sr_0 if you have an echo), but I wanted to get my hands dirty with the AWS IoT SDK, and having never really done anything with electronics before I thought this would be a great opportunity.

I went to my local electronics store (http://www.jaycar.com.au) and the only sensor they really had which I could use with my spare Raspberry Pi (who doesn't have two or three lying around..?) was the Freetronics temperature/humidity sensor which is already attached to a board ready for some cables to be soldered, and no messing with resistors is required:

I thought a great way to get familiar with AWS Lambda, AWS IoT and the Alexa Skill Kit would be to combine them all and create an indoor weather station where I could ask Alexa what the temperature and humidity was inside. So, I got to work!

Hardware

I began by installing Raspbian via NOOBS. As a Linux Sysadmin in a past life, this is not something I wanted to spend too much time on, and it was fairly straight forward.

Then I got my brother to solder some wires onto the sensor so I could connect it to the Pi (thanks Simon!). Then I had to figure out where I could plug it in. I eventually found this helpful diagram over at myelectronicslab:

"What on earth is a GPIO?" I ask myself.. well.. lets look at the back of the DHT22:

So.. we have a "GND".. this obviously matches the "Ground" port, so I connect this to pin 9. Then we have a "VIN".. which says 5V, so I assume this is the power port. The GPIO diagram has a "DC Power 5V" at pin 2, so I connect this one. We're left with DATA, so I choose a random GPIO port as I assume the IO in GPIO means input/output.. data.. yeah.. pin 15 you're the winner!

I then discover Joan over at the raspberry pi forums has written a DHT22 library in python, which is perfect! I grab that from her post here. It requires pigpiod, so I installed and started that:

$ apt-get update
$ sudo apt-get install pigpio python-pigpio python3-pigpio
$ service pigpiod start

Poking through the code in DHT22.py I want to see which GPIO port I should be connected to. I see that the gpio variable is passed when the class is defined here:

s = DHT22.sensor(pi, 22, LED=16, power=8)

GPIO is the second field, which is '22'. Checking back on the GPIO graph, I'm plugged in to pin 15 which is... GPIO22! Wow, could this possibly work first go? that never happens. Lets try running the script:

$ python DHT22.py
1 38.7 25.0 0.20 0 0 0 0
2 37.9 24.9 0.19 0 0 0 0
3 38.1 24.9 0.19 0 0 0 0
4 38.1 24.9 0.19 0 0 0 0
5 38.2 24.9 0.19 0 0 0 0

Well that's a rare sight.. it worked first go! It looks like we're receiving humidity, the temperature and a field called "staleness" which is the time since the measurement was made. Awesome! That's the OS and the sensor working, now we need to push the data somewhere.

Development

It took some time for me to figure out the best way of storing the data in a way that Alexa could retrieve it. I knew I would be using the AWS IoT service as it integrates with Lambda very easily, but I started out by trying to store the data payload from the MQTT stream in DynamoDB and then I was going to try and pull the payload out with Lambda and attempt to parse it.. and then I was pointed to the "Device Shadow" functionality of an AWS IoT thing. Quoted from the IoT Developer Guide:

A thing shadow (sometimes referred to as a device shadow) is a JSON document that is used to store and retrieve current state information for a thing (device, app, and so on). The Thing Shadows service maintains a thing shadow for each thing you connect to AWS IoT. You can use thing shadows to get and set the state of a thing over MQTT or HTTP, regardless of whether the thing is connected to the Internet. Each thing shadow is uniquely identified by its name.

This sounds perfect. If all I want is the current temperature or humidity from my sensor, I don't need a DynamoDB database to store a full history of my temperatures (unless I want to graph it I guess). I just need to send the current temperature and humidity to the Device Shadow for my raspberry pi. Awesome!

Setting up IoT

I've created a git repository with a few of the files required in the next section.

$ git clone git@github.com:xelfer/nickshouse.git
$ cd nickshouse

I now need to configure the pi to be able to communicate and push data to the AWS IoT service.

I start by creating a user in my AWS IAM console, I gave mine the AdministratorAccess policy for testing, but extra security is always good and something like IAMFullAccess, AWSIoTFullAccess and AWSLambdaFullAcess should suffice. Record the Access Key ID and Secret Access Key as you'll need it in the configure command.

Then install and configure the AWS CLI (use pip, apt-get has an old version) and run configure:

$ sudo pip install awscli
$ aws configure

..and enter those keys. You can choose the region you require, I used Sydney (ap-southeast-2). Keep in mind that you will have to use us-east-1 for your Lambda function, as it's the only region that can interact with the Alexa Skill Kit.

Once the AWS CLI is configured I need to create the certificates for our AWS IoT device. These instructions were taken from https://github.com/mariocannistra/python-paho-mqtt-for-aws-iot and they work perfectly.

1. Create thing

$ aws iot create-thing --thing-name "pi"

2. Check it worked

$ aws iot list-things
{
"things": [
  {
    "attributes": {}, 
    "thingName": "pi"
  }
]
}

3. Create certificates and keys

$ aws iot create-keys-and-certificate --set-as-active \ 
--certificate-pem-outfile cert.pem \ 
--public-key-outfile publicKey.pem \ 
--private-key-outfile privkey.pem

4. Get the certificateARN

$ aws iot list-certificates
{
"certificates": [
  {
      "certificateArn": "arn:aws:iot:ap-southeast-2:926342229229:cert/58debbfe90fa58cad4df8426bdf3f20a71df7644437203e231b503b994dfb8f3", 
      "status": "ACTIVE", 
      "creationDate": 1467893789.688,  
      "certificateId": "58debbfe90fa58cad4df8426bdf3f20a71cf7644437203e251b503b994dfb8f3"
  }
]
}

5. Create a policy from the iotpolicy.json in the git repository (it should be in your current directory) and download the root certificate from symantec.

$ wget https://www.symantec.com/content/en/us/enterprise/verisign/roots/VeriSign-Class%203-Public-Primary-Certification-Authority-G5.pem -O aws-iot-rootCA.crt
$ aws iot create-policy --policy-name "PubSubToAnyTopic" --policy-document file://iotpolicy.json

6. Attach the principal, using the arn from earlier in step 4

$ aws iot attach-principal-policy --principal "arn:aws:iot:ap-southeast-2:926342229229:cert/58debbfe90fa58cad4df8426bdf3f20a71df7644437203e231b503b994dfb8f3" --policy-name "PubSubToAnyTopic"

7. Get the endpoint

$ aws iot describe-endpoint
{
"endpointAddress": "a1v9yu6xvaswb1.iot.ap-southeast-2.amazonaws.com"
}

8. Attach the thing principal, using the arn from earlier in step 4

aws iot attach-thing-principal --thing-name "pi" --principal "arn:aws:iot:ap-southeast-2:926342229229:cert/58debbfe90fa58cad4df8426bdf3f20a71df7644437203e231b503b994dfb8f3"

and we're done!

Scripts

I began with the script Joan provided and begun hacking it together with a sample script provided in this git repository.. and literally the next morning AWS announced their IoT Python SDK. So I switched to that immediately. I start off by installing it:

pip install AWSIoTPythonSDK

Now for my main script. It imports a few libraries, sets some variables and then uses the new IoT Python SDK to push the temperature to the device shadow for my raspberry pi. It's simple enough, and hopefully the comments in my script explain it clearly enough. In my git repository this is awstemp.py

#!/usr/bin/python
import time
import pigpio
import DHT22
from AWSIoTPythonSDK.MQTTLib import AWSIoTMQTTShadowClient
# Edit this to be the awshost you got from `aws iot describe-endpoint`
awshost = "a1v9yu6xvaswb1.iot.ap-southeast-2.amazonaws.com"
# Edit this to be your device name in the AWS IoT console
thing = "pi"
if __name__ == "__main__":
# Connect to the sensor
pi = pigpio.pi()
s = DHT22.sensor(pi, 22, LED=16, power=8)
# define your variables
awsport = 8883
caPath = "aws-iot-rootCA.crt"
certPath = "cert.pem"
keyPath = "privkey.pem"
# Set up the shadow client
myShadowClient = AWSIoTMQTTShadowClient(thing)
myShadowClient.configureEndpoint(awshost, awsport)
myShadowClient.configureCredentials(caPath, keyPath, certPath)
myShadowClient.configureConnectDisconnectTimeout(60)
myShadowClient.configureMQTTOperationTimeout(10)
myShadowClient.connect()
myDeviceShadow = myShadowClient.createShadowHandlerWithName("pi", True)
# You can implement a custom callback function if you like, but once working I didn't require one. We still need to define it though.
customCallback = "" 
# The main loop
while 1==1:
# Get the sensor to take a reading
s.trigger()
# Create our payload in JSON format
tempreading = "{ \"state\" : { \"reported\": { \"temp\": \"%s\", \"humid\": \"%s\" } } }" % (str(s.temperature()), str(s.humidity()))
# Print the result just for some debugging
print("payload: %s" % tempreading)
# Only report the temperature if a valid reading has been taken
if s.temperature() != -999:
  myDeviceShadow.shadowUpdate(tempreading, customCallback, 5)
# Wait a minute before updating again, you can lower this for debug purposes
time.sleep(60)

and that's it! the only thing left to do is make the script run on boot. Firstly I have a simple script which changes to the correct directory (so our certificates work). It also has a "sleep 5" so during boot it waits for the pigpiod service to load first. In my git repository this is awstemp.sh.

#!/bin/bash
sleep 10
cd /home/pi/nickshouse/
./awstemp.py

Now we can add the following to the /etc/rc.local of our raspberry pi. This is the only way I could figure out how to get the script to load on boot inside a a screen session (so we can re-attach as the root user to see if it's working and debug if there's any issues).

/usr/bin/pigpiod
screen -dmS "awstemp"
screen -S "awstemp" -p 0 -X stuff "/home/pi/nickshouse/awstemp.sh$printf \\r" 

The second screen command is a bit odd, but it works. You can read about it here on stackoverflow. A quick reboot and a confirmation that screen is running our script, and it's time to check the AWS IoT console.

Testing IoT

Time to head on over to the IoT console in AWS and we should see your policy, thing, and certificate:

Click on your thing (in this case "pi") and on the right hand side we should see our device shadow status and state payload. It looks good!

This is all I need to do with the IoT console. I have confirmed that the script is pushing data to the IoT service, now I can start working on the Alexa skill to pull the data out and read it out loud.

Lambda setup

A reminder, you must be in the us-east-1 region for the lambda function to be able to work with the Alexa skill kit.

First I go to the Lambda console, click the big blue "Create a Lambda Function" button, click "next" and skip the blueprint. On the next "Configure triggers" page, select the "Alexa Skills Kit".

I give it the name, description, my runtime is Nodejs 4.3, and I'll be editing my code inline. I leave handler as default, and let the UI create the role by selecting "create a custom role" in the "Role" pull down menu. This launches the IAM management console and creates a role called "lambda_basic_execution". We'll need our role to have access to the IoT services as well, so click "view policy document", then click "edit" and add "iot:*". The policy should look like:

{
"Version": "2012-10-17",
"Statement": [
  {
    "Sid": "",
    "Resource": "*",
    "Action": [
      "iot:*",
      "logs:CreateLogGroup",
      "logs:CreateLogStream",
      "logs:PutLogEvents"
    ],
    "Effect": "Allow"
  }
]
}

It has access to CloudWatch logs, as that's where your error messages will be going to help you debug your lambda code, and all of IoT.

Leave everything else as default, hit next, review your settings and click "create function". Now you can edit the function inline.

Lambda code

I wont go through all 230 lines of my lambda function (available in the github repo as NicksHouse.js), but I will pull out the bits I think are important. It's relatively straight forward.

I start off by configuring the IoT broker

var config = {};
config.IOT_BROKER_ENDPOINT      = "a1v9yu6xvaswb1.iot.ap-southeast-2.amazonaws.com";
config.IOT_BROKER_REGION        = "ap-southeast-2";
config.IOT_THING_NAME           = "pi";
config.params                   = { thingName: 'pi' };

The endpoint we found earlier, the thing name is pi, and we need a special JSON snippet called params to be able to access the device shadow data.

I then load the AWS SDK library so we can use the IoT functions

AWS.config.region = config.IOT_BROKER_REGION;
var iotData = new AWS.IotData({endpoint: config.IOT_BROKER_ENDPOINT});

We should always only allow our Alexa app to call our lambda function (you'll get this string in the next section when creating the Alexa skill):

if (event.session.application.applicationId !== "amzn1.echo-sdk-ams.app.c05192e2-4525-429c-8235-3095b4d8fec8") {
   context.fail("Invalid Application ID");
}

The function onIntent is used to determine which function will be called when a user says certain keywords. These are explained more in the next section, but as you can see here I have different functions being called for the intent "temperature", "humidity", "weather", "help" and "stop".

if ("GetTemperature" === intentName) {
   getTemperature(intent, session, callback);
} else if ("GetHumidity" === intentName) {
   getHumidity(intent, session, callback);
} else if ("GetWeather" == intentName) {
   getWeather(intent, session, callback);
} else if ("AMAZON.HelpIntent" === intentName) {
   getHelp(callback);
} else if ("AMAZON.StopIntent" === intentName || "AMAZON.CancelIntent" === intentName) {
   handleSessionEndRequest(callback);
} else {
   throw "Invalid intent";
} 

I'll mention briefly here that the getHelp intent is a requirement if you want to publish your skill to the Alexa store.

getTemperature, getHumidity and getWeather are almost identical, so i'll show just one of them here.

function getTemperature(intent, session, callback) {
   // set up some variables
   var cardTitle = "Temperature";
   var repromptText = "";
   var sessionAttributes = {};
   var shouldEndSession = true;
   var temp = 0;
   // Use the IoT SDK to get the current shadow data
   iotData.getThingShadow(config.params, function(err, data) {
       if (err)  {
          console.log(err, err.stack); // an error occurred
       } else {
          // Get the payload and parse it into a variable
          payload = JSON.parse(data.payload);
          // Retrieve the temperature from the payload
          temp = payload.state.reported.temp;
       }
       // Create the speech text we want Alexa to say
       speechOutput = "The temperature is " + temp + " degrees celcius";
       // Return the speech response
       callback(sessionAttributes, buildSpeechletResponse(cardTitle, speechOutput, repromptText, shouldEndSession));
   });
}

And that's all for the lambda code! there's 2 helper functions that exist as well which you can see at the bottom of the file which assist in building the responses.

The Alexa Skill

The final task is to create the Alexa Skill itself. Login at http://developer.amazon.com/ and select the Alexa Tab, and then "Get Started" under Alexa Skills Kit. Then click "Add a new skill".

Skill type will be "Custom Interaction model", the name and invocation name are both "Nick's house". Click Next.

Under "Intent schema" we will list the possible things a user may ask the skill. This was what the lambda function onIntent mentioned earlier uses to decide which lambda functions to call based on what the user says.

{
"intents": [
  {
    "intent": "GetTemperature"
  },
  {
    "intent": "GetHumidity"
  },
  {
    "intent": "GetWeather"
  },
  {
    "intent": "AMAZON.HelpIntent"
  },
  {
    "intent": "AMAZON.StopIntent"
  }
]
}

We have no Custom Slot Types so we'll leave that and proceed to Sample Utterances. These are all the different ways someone may request something from your application. You specify which intent you're creating an utterance for, and then provide all the different options you can think of. The more of these you have, the more accurately your application will respond to the user.

GetTemperature whats the temperature
GetTemperature whats the current temperature
GetTemperature what is the temperature
GetTemperature what is the current temperature
GetTemperature tell me the temperature
GetTemperature tell me the current temperature
GetTemperature temperature
GetTemperature current temperature
GetTemperature how cold is it
GetTemperature how hot is it
GetHumidity whats the humidity
GetHumidity whats the current humidity
GetHumidity what is the humidity
GetHumidity what is the current humidity
GetHumidity tell me the humidity
GetHumidity tell me the current humidity
GetHumidity humidity
GetHumidity current humidity
GetWeather what's the weather
GetWeather tell me the weather
GetWeather how's the weather

In the next "configuration" section, we need the ARN of your lambda function. This can be found in the top right corner of the AWS console when you click on your lambda function name

Paste that into your "Endpoint" box and select the Lambda option

Leave "account linking" set to no.

In the "Test" section it has some instructions on how to test it on your echo at home, and you can use text to test out your skill if you don't have an Amazon Echo.

The "Publishing information" is straight forward, enter a description, ensure the three sample utterances are tested and working and any testing instructions you may have for the Alexa skill testing team.

Finally there's some Privacy and Compliance questions which are very straight forward too. At this point it's ready to submit for certification!

Conclusion

That's really all there is to it. The skill should be working and I can now ask Alexa what the temperature, humidity or weather is at nick's house, and it will use the lambda function to query the IoT device shadow for my pi, parse the payload and read out the result!

Alexa Skill Certification

My Alexa skill was in review for over a month, and was eventually rejected on the basis of not describing how a user should configure their own raspberry pi.. however my skill interacts with my own raspberry pi, I don't really understand how you would be able to provide an alexa skill user configurable to pull data from your own pi (maybe you can? i'm not sure). So unfortunately I'll have to go without the bonus points on this one :(

You can view my other published skill here.

Bonus content

What better way to round off a fun project than with some 3D Printing? I fired up my FlashForge Creator Pro and printed a nice Rasperry Pi 3 case and a housing the sensor. Both were found on thingiverse:

• https://www.thingiverse.com/thing:559858 - pi case

• https://www.thingiverse.com/thing:866900 - sensor housing

I think they turned out great!

Be sure to check out my video to see the project in action, it has been great fun to work on and I hope to create more soon!

- Nick