Crystal Plate - A Wireless Memo Maker

Note: This skill is published in the Alexa Skill catalog. Try navigating the voice UI on the Echo without any setup here!

Hi there!

This project not only shows you how to create a wireless notepad, but also details my experience getting communication working between Alexa and a Raspberry Pi from a first-time user's perspective. I'll detail the cool things and pitfalls I found when making this application, and show you how to get data both to and from the Raspberry Pi using a custom skill so you can get your Pi to speak and act to your heart's content!

Opening Credits

• Chris Synan's guide to setting up an AWS IoT drone was really helpful to me for getting the Raspberry Pi hooked up to the AWS ecosystem

• The Alexa Quick Start in Node.js was great for becoming familiar with hooking up AWS Lambda with an Alexa skill and became the starting base for my code

• Also, if you have any suggestions or improvements, feel free to drop a comment so I can improve this guide, both for myself and anyone who might be reading. Thanks!

Introducing CrystalPlate

Since the invention of the sticky note, we've always been leaving memos; for others and ourselves alike. When I was pondering what to create for this challenge (and thus my first dive into AWS IoT, Lambda, and Node.js), I thought, why waste the paper when I have CPU resources idly wasting away on my Pi? Here's a few more reasons I chose to make this as my first project:

• It's simple. A few wires, a few installs, a bit of code and bam - we're done!

• It's fun. Not only do I immediately get to see the fruits of my work, but now I won't have to deal with the Post-it litter all over my desk.

• It's useful. Being back-lit and multi-color, I can leave notices for both me and my roomate wherever I am.

• It's low-risk. Better safe than sorry when starting up - if I mess up, I don't want my room catching on fire!

What can CrystalPlate do? Well, it currently offers the following functionalities:

• Set a message (with an optional color): "Ask CrystalPlate to write hello world to the display in green"

• Change the display color: "Ask CrystalPlate to make the display blue"

• Get the display message/color: "Ask CrystalPlate, what's on the display?"

• Clear the display: "Ask CrystalPlate to clear the display"

Alright, enough small talk, let's build this thing!

Setting up AWS IoT

Again, Chris Synan has a great guide on setting this up, so I'll just note a few thoughts that went through my head (as a new AWS IOT user) when following his guide:

• I was mildly surprised that I could just "create" a device on IOT without hooking it up to anything; we can do this because a "device" on AWS IoT is used to receive messages all the time; since this is independent of your Pi, the application will continue working even when your Pi is offline!

• There's no application to install here; we'll be using a messaging system known as MQTT to communicate with the Pi, the details for which AWS generates (we'll get to this in a bit)

• I went ahead and created to certificates and attached them both to the Pi - one for the Pi to talk to AWS, and one for Alexa to talk to the Pi through Lambda. There's really no harm in creating two.

• Don't forget to create a policy for permissions regarding the Pi! I just went ahead and used Chris's suggestion of allowing everything.

Connecting the Pi to AWS IoT

Note: This could cost you (a small amount of) money! The AWS free tier includes the ability to deliver 250,000 messages a month for free for 12 months, but it will cost $5 per million messages after that. We're not getting anywhere near that, but you can check out the pricing info here.

First, we'll need to set up our Pi. If you don't have a Raspberry Pi Zero, don't worry! The code we'll be running should work on any Pi platform. I started with the Raspbian Lite image on the Raspberry Pi website, and we'll work from there.

Since AWS IoT can use MQTT to actually communicate between devices, we'll need to install a client we can use within our own Python script. First, make sure you have pip installed:

sudo apt-get install python-pip

and then install the Paho MQTT client:

sudo pip install paho-mqtt

While we're installing things, we'll also need the Adafruit Character LCD Library for our Pi, installation instructions for which can be found here. Essentially, the commands boil down as follows:

sudo apt-get install build-essential python-dev python-smbus python-pip git
sudo pip install RPi.GPIO
git clone https://github.com/adafruit/Adafruit_Python_CharLCD.git
cd Adafruit_Python_CharLCD
sudo python setup.py install 

Shweta Nerake has a good starting base for hooking up up the Pi to AWS IoT with her file, mqtt-iot-subscriber. This script provides a skeleton for calling whatever code you write when a new message comes in, or when the Pi gets connected/disconnected. I've built the rest of my code upon this skeleton, and have turned it into CrystalPlate.py. For this specific project, all you need to do is download the file and change the following lines:

Here's an explanation of what each variable is:

• rootCAPath: Basically a certificate that let's your computer trust that it's talking to AWS. One's included in the GitHub

• certFilePath: The .crt you got and attached to your device in AWS IoT

• keyFilePath: the .key you got alongside your .crt

• iotThing: The name of your Thing in IoT

• clientID: This is the identifier it uses when connecting to MQTT. This can actually be anything, as long as it's unique in your project

So, with this setup, you'll want to create a folder on your Pi called certs in the same directory as the Python script (CrystalPlate.py), and place those files inside of that folder.

Now, when you run the script, your Pi should be ready to talk!

DIY Notes:

• If you're rolling your own code, I'd start with Nerake's mqtt-iot-subscriber. It subscribes to /update instead of /update/delta, so you'll be able to see all of the messages flowing through the pipeline in your application

• Figuring out what to subscribe to took me the longest time. Try making sure that you can see your messages by using the MQTT client on the top-right of the IoT page - publish to $aws/things/*YourThingName*/shadow/update and see if your Pi picks it up when the script is running!

Connecting the LCD to the Pi

This part's pretty simple - you can find assembly instructions for the Pi Plate by Adafruit here! Make sure you've installed male pins on your Pi Zero for the plate to fit on correctly.

DIY Notes:

• You may have noticed that my cover picture doesn't have the plate sitting on the Pi Zero's header - if you're like me and don't want to cover the other GPIO pins, you can simply just use jumper cables to connect the first three pairs of GPIO's, starting from the top.

• For a good connection, you'll want your solder "joints" (where the two things you're soldering come together) to look like concave spires and not simply a blob to ensure good contact. Sparkfun has a good tutorial on soldering if you're looking for more info.

Creating the Brains (AWS Lambda)

Amazon likes to integrate with its own tools, so it's no wonder that trying to create an Alexa app will push us towards AWS Lambda for hosting and running our code (setting up code elsewhere is possible, but harder). AWS Lambda is basically a tool to create a piece of code that runs in response to a request - basically, the Alexa Skill we'll create in a bit will take in a user's voice input, convert it to a standardized request format, and then send it over to our Lambda function, which will be responsible for all of the processing and sending a text response back to Alexa, which it will then, in turn, turn back into voice for the user to hear.

Note: Since AWS Lambda uses some amount of computing resources on Amazon's end, this service isn't totally free. However, Amazon is generous enough to host 1 million requests for free each month, which is more than plenty for our purposes. Find out more about the pricing here.

First, let's prepare our project files. Start with a nice, fresh folder we can start throwing things into:

Next, we'll want to install the Node.js module aws-iot-device-sdk . If you have npm installed (and on Linux, I'd recommend just going ahead an installing it, via apt-get , yum , or otherwise), you can just run:

npm install aws-iot-device-sdk

For the lazy out there, i've also included a zip file here that contains the same files npm would have created. Make sure to unzip it!

Now our folder should contain one folder inside of it, called node_modules. Next, copy over the project files package.json, index.js, rootCA.pem, and your certificate files over (if you created and attached two sets of certificates to your Thing in AWS IoT, use the other one here). Your folder should now look something like this:

Now we're almost done - we just need to update index.js using the text editor of your choice:

Make sure keyPath and certPath are the names of your certificate and key in your folder! If you downloaded my rootCA.pem, you won't need to change that; clientId just needs to be different from the client ID we set up on the Pi so MQTT doesn't get confused, and region should be us-east-1. You'll also want to comment out some code that's there for security for now, we can add it again later:

Once you've done that, our files just need to be zipped up to send to Amazon - select all of your files (NOT the parent folder!) and zip 'em up:

On Linux, with the zip command installed (which you can get via apt-get, yum, etc.), the command would be:

zip -r blah.zip *

Now we're ready to ship this off to Amazon Lambda!

Amazon made a great tutorial on how to do this, which can be found here. Here's what you'll need to keep in mind when moving through it:

• In step E, you'll want to choose the blueprint called "alexa-skills-kit-color-expert" (NOT the one that ends in "-python" !)

• The tutorial forgets to mention this, but you'll want to skip setting up an optional trigger after step E

• You can name the function whatever you want

• Make sure the runtime is set to some kind of Node.js - this application is written in it, so it won't work otherwise (you shouldn't have to change anything)

• Now, in the function code section, you'll want to upload the zip file we created earlier:

• For choosing a role, you'll want to pick "Create a Custom Role" - make sure popups are enabled, or else you'll miss the dialog! Then, make sure "lambda_basic_execution" is chosen, and choose "Allow"

• I'd recommend changing the timeout to 10 seconds, just in case of slow network/etc. on Amazon's end

• Once you reach step 2 on the tutorial, read on below!

Configuring the Alexa Skill

Before we start, I think it's a good idea to conceptualize what an Alexa "Skill" even is. Going into it, I thought that the Alexa Skill would be handling all of the logic, processing, and communication with the Pi, but in reality, it performs only one deceptively simple task: Turning the human voice into triggers that get sent to an application in the background for processing.

Again, Amazon has a great tutorial on setting this up here - we're looking at step 2. Here's my notes on the process:

• I called my app "CrystalPlate" with the invocation name "crystal plate" so that Alexa would understand it. The invocation name is what really counts here - This way, when you say, "talk to crystal plate" to the Echo, it knows to start interacting with this app

• When setting up the interaction model, you'll want to use my files on GitHub in the alexa folder here instead of the ones they suggest, since we're not setting up the Colorful app :P

• You can copy the contents of the intent schema and sample utterances text files directly, but for slot types, you'll need to set them up like so:

And we're done here - All we need to do is start the Python script and start receiving messages!

DIY Notes:

• Alexa will try to parse whatever voice input it receives and sort data into necessary buckets so your application can get variables out of it. The Intent Schema shows how this happens

• I spent forever trying to find the slot type for free-form input, which is what I needed for people to write arbitrary messages to the display. I thought the only way I could achieve this was Amazon's deprecated AMAZON.LITERAL slot, but I got poor recognition results. The solution? Just use a custom slot type and put as many examples as you can! Don't worry, Amazon will still pick up on other kinds of messages that aren't listed; you're just training Alexa on what length and type of message it should expect

• Paradigm shift (again, relating to the above bullet)! Slot values are not an exhaustive list! So if Alexa determines that the user was requesting a color and that color isn't on your slot value list, you still need to handle an "unknown color" in the Lambda code!

• You can add and change around the slot types to your liking for this project - try training Alexa in different ways!

• I highly recommend using the Service Simulator in the test tab to make sure everything's working - for this project, you can try tying in "write hello world to the display" in the utterance entry

Testing

You can get the script running simply by typing this in the folder with CrystalPlate.py:

python CrystalPlate.py

You should see "Connected! Ready for input" on the display, and from there, you're set to use the Echo connected to your account or EchoSim to try it out - have fun!

DIY Notes:

• Make sure testing is enabled on your Skill - you can find this in the test tab:

• If you receive an error, try typing your voice input into the utterances section. If that fails, you can see more detailed output by copying the Lambda request, going back to the AWS Lambda page, and pasting it into the "Test" section

• Because of this, debugging code by adding console.log to places can be very useful

Now that we're done...

So everything's working; wouldn't it be nice if this python script started itself when the Pi did? Well, it's possible (and easy)! RaspberryPi-Spy has a great tutorial on this - just change around the name of the python script and name of the service to match.

Remember those lines we commented out from index.js earlier? We can add an extra layer of security on our AWS Lambda function by copying the application ID from the Alexa Skill and pasting it back into the index.js script we uploaded to Lambda. Just put your ID there, and uncomment those lines so that other people's skills can't talk to your Lambda function!