Ping Pong Showdown

Introduction

This project is just a fun learning vehicle to develop skills around AWS, IoT and Alexa from Amazon. It uses both ASK and AVS to keep track of the score of a game of ping pong.

You begin a game by speaking to Alexa through an Echo, Tap or Dot. Then, when a player wins a point, a tap or double tap to an IoT button will trigger Alexa to announce the new score. She may also throw in some sass for good measure.

Breakdown of the project

There are 2 parts, with clear delineation. The full project contains both components, but each can be built in isolation of the other. They are:

• Part 1: Development of the Alexa Skill using ASK. The project starts by building the Alexa Skill. However, 'PingPong Showdown' is a published skill, you can find it here and skip all of the ASK development if you like. By using the published Skill, you can move directly to the 2nd part.

• Part 2: Setting up the Raspberry Pi (RPI) with AVS and IoT connectivity. In this part of the project, we create a means by which we can push a physical button attached to a ping pong table and have a voice response experience to control the flow of match play. There are 3 phases of this effort: 1) Initial RPI setup, 2) AVS setup and 3) IoT button integration. In phase 3, I used two different IoT buttons to learn a bit about each. The first is the Amazon AWS IoT Button, and the second is the Flic button. Both are described below, you can choose which way to go. Either way Alexa will respond to triggered events for a single or a double click depending on who won the point.

Part 1: Building the Alexa Skill

You can follow this tutorial if you are getting started from scratch with ASK, it will help with the basics in case you get stuck somewhere.

First, download this zip file and unzip it to a directory called PingPong.

1. Create an AWS account by going here if you don't already have one.

2. In Lambda, create a new function.

• On the 'Select Blueprint' screen, skip it by hitting Next.

• Under 'Configure Triggers', click in the dashed box and pick 'Alexa Skills Kit', then hit Next.

• Give your function a name like PingPong and leave the Runtime at the default node.js version.

• In the 'code entry type' pull-down, pick 'upload a ZIP file'. Now in your PingPong directory, go into the 'lambda' directory and ZIP up all of the files, and then select that ZIP file for upload back in the Lambda browser session.

• Now you need to give your function permission to access other AWS services. This skill makes use of DynamoDB and S3, so you need to define access for those in a Policy attached to a Role. Under 'Role', pull down 'Create a custom role'. This will launch a new IAM window. Under 'IAM Role', pick 'Create a new IAM role'. Name the role 'PingPong'. View the policy document, and hit 'edit'. Note I am by no means an IAM authority, you may want to use this as a starting point and then per best practice restrict access to a more granular level. Below is a policy that provides full access to logs, dynamodb and S3 to get started. Edit the policy and then hit 'Allow'.

{
"Version": "2012-10-17",
"Statement": [
{
    "Sid": "Stmt1455492578000",
    "Effect": "Allow",
    "Action": [
        "logs:*"
    ],
    "Resource": [
        "*"
    ]
},
{
    "Sid": "Stmt1455492603000",
    "Effect": "Allow",
    "Action": [
        "dynamodb:*"
    ],
    "Resource": [
        "*"
    ]
},
{
    "Sid": "Stmt1455492627001",
    "Effect": "Allow",
    "Action": [
        "s3:*"
    ],
    "Resource": [
        "*"
    ]
}        
]
}

• Leave everything else as default and hit 'Next'. Then hit 'Create Function'.

• Take note of the Lambda ARN number in the upper-right corner of the screen, you will need it later.

3. In DynamoDB, we will now create the 3 tables needed to support the skill.

• Now in your PingPong/dynamodb directory:

• Ensure you have node.js installed (if not, get it here)

• Ensure you have the AWS CLI installed and configured. If not, go here)

Create the Devices table with:

node CreatePingpongDevices.js

• Create the Matches table with:

node CreatePingpongMatches.js

• Create the Players table with:

node CreatePingpongPlayers.js

If you are new to DynamoDB and would like to know more, you can get a jump start with this tutorial.

4. Now we'll build create the Alexa Skill that will use these resources.

• Go to the Amazon Developer Console

• Under Alexa Skills Kit, select 'Get Started'

• Click 'Add a New Skill'

• Complete as below and hit Next:

• Now in your PingPong/speechAssets folder, copy the contents of IntentSchema.json into the intent schema section under 'Interaction Model'.

• Next, add custom slot types as follows (the full Type name of the 4th one down is 'SINGLES_OR_DOUBLES') :

• Then in your PingPong/speechAssets folder, copy the contents of SampleUtterances.txt into the Sample Utterances section under 'Interaction Model', hit 'Save', then 'Next'.

• Under 'Configuration', pick 'Lambda ARN', and then specify your Lambda ARN from above in the 'Endpoint' field.

• Select 'No' for Account Linking, hit 'Save', then 'Next'.

• There is no need to complete the last 2 sections (Publishing Information and Privacy & Compliance).

• Next, copy the Application ID shown in the red box below under 'Skill Information'.

• Now paste it in to PingPong/lambda/matchKeeper.js as shown here and save the file:

• You will then need to repeat the step above in your Lambda function - in the 'code entry type' pull-down, pick 'upload a ZIP file'. Now in your PingPong directory, go into the 'lambda' directory and ZIP up all of the files, and then select that ZIP file for upload back in the Lambda browser session. Finally, you can now go to the Test section of your Alexa Skill and try out your skill. In the Service Simulator, type 'start a match'. You should see the Lambda Request and Response similar to below.

• The skill randomly inserts quips, some of which contain song clips. Your skill will work with the code as is, pulling the .mp3 files from the object bucket I set up in S3. If you choose, you can replace any/all of them with your own items. In the Lambda function policy definition above, you gave your function access to the S3 service in your account, so just set up your own bucket and objects and point to them instead of the existing ones.

Overview of the Node.js Lambda Code

The lambda directory contains the various part of the node.js code supporting the skill. Here is a brief description of what each is doing:

• AlexaSkill.js: The base prototype and helper functions

• matchKeeper.js: A child of AlexaSkill.js

• index.js: The entry point for the lambda function execution

• eventHandlers.js: Has the logic from onLaunch, when the user specifies the skill without an intent

• intentHandlers.js: Contains all of the functions that support intents. This is the bulk of the code and base of the ASK portion of the project effort.

• kickoffMatch: As the name implies, this is called to kickoff a match and provides the matchup record (number of matches & win/loss record) if players are signed in.

• matchStorage.js: This is the code that is responsible for persistence. As a stateless app, each "Alexa, use pingpong and..." Lambda invocation requires establishing context, matchStorage does that.

• playerStorage.js: Stores player data including match preferences

• textHelper.js: Contains all the info when a player asks for help and describes what options the skill provides. Also includes the rules of ping pong.

• queryForSinglesMatchups.js: Pulls the match data from all previous historical singles matches in order to provide number of matches played & win/loss record.

• singlesPlayerHistory.js & doublesTeamHistory.js: Pulls historical match stats in support of the intent triggered by a player saying "give me a summary of the match" in order to compare the current match against the historical record.

Using the Alexa Skill

This is a full featured skill with more than 25 interactions. Alexa is a great score keeper, but lookout for her sharp humor! She has a sass-meter, so you can adjust it from 0 (off) to 10 (full up). She never forgets the score or when to switch serving. In fact every point and every game you play is stored for bragging rights against rivals.

You can play both singles and doubles matches. You can sign in so that all match stats are stored for you, or skip sign-in and just play. In addition to the game and match scores, you can also hear what percent of games and individual points were won/lost vs that opponent, what your win percentage was when serving, and what your longest point streak of the match was.

To get started, first register by saying 'register a new player'. Then just say, 'Use ping pong and start a match'. Whether it is singles or doubles, you will have a red team and a blue team. Once you've started a match, the only thing to do is tell Alexa who wins a point. Do that by either single or double clicking your IoT button. You can also say 'use ping pong and point blue', or 'point red'. All the rest is coordinated by Alexa.

When doing the one-time registration, you will choose a 4-digit number for your player ID and tell Alexa your first name. Then when starting a match, Alexa will prompt players for their IDs and kick off the match.

Basic things you can say to get started:

• “Start a match”

• “Start a doubles match”

• “Set the sass-meter to zero.” (0-10)

• “Tell me the rules of ping pong”

• “Register a new player”

• “Undo that”

• To give a point to a player, click a button or say “Point Red”, or “Point Blue”.

• “Tell me more things I can say”

Examples of more things you can say:

• “What's the score?”

• “Give me a summary of the match” (provides output to the Alexa app as well as verbal)

• “What's the match score?”

• “Who's serve is it?”

• “Tell me how to set preferences”

• “Tell me ways to change the match”

Examples of ways that you can manually override and control the flow of the match:

• “Play games to 21” (or 11)

• “Play best three out of five”

• “Play best three out of five”

• “Play a single game match”

• “Change the number of serves per turn” (to either 2 or 5)

• “Don't switch sides”

• “Change the score”

• “Start a new game”

Here are ways that you can set preferences for what Alexa says during the match:

• “Keep it short” This cuts down on the things Alexa says.

• “Don't switch sides”

You can save and load your match preferences. Preferences include:

• Playing to either 11 or 21 points in a game

• Playing matches with either one game, or best three out of five games

• Setting of the sass-meter between 0-10

• Switching sides after serve (on/off)

• Experienced user mode, where comments are kept short

When you say 'Save Preferences', the settings of the current match will be stored in your profile.

To load your preferences, say, Load Preferences after starting a new match.

You can always say “stop”, “cancel” or “exit”.

Part 2: Setting up the Raspberry Pi, AVS & IoT Button

Phase 1: Initial RPI Setup

There are a number of great resources out there that cover initial setup of an RPI for use with AVS. Instead of repeating the content, let's use this excellent project as a reference to follow. On your Raspberry Pi:

• Follow the instructions there from step 0 through step 2.2. Skip installation of the microphone, it isn't required.

• Install Node.js - this was the approach I used:

wget https://nodejs.org/dist/v4.3.2/node-v4.3.2-linux-armv6l.tar.gz
tar -xvf node-v4.3.2-linux-armv6l.tar.gz
cd node-v4.3.2-linux-armv6l
sudo cp -R * /usr/local/

Now node -v should return v4.3.2

• Add node to global PATH: sudo vi /etc/environment In that file, add:

export PATH=$PATH:/usr/local/bin/node

• Install mpg123 sound player:

sudo apt-get install mpg123

• Optional: follow these instructions to create a windows share on your RPI to facilitate moving files back and forth.

• Copy the source files in PingPong/RPI to your Raspberry Pi

• Navigate to the RPI folder for access to requirements.txt, and install the required python components:

Sudo apt-get update
Sudo apt-get install memcached python-pip
Sudo pip install -r requirements.txt

Phase 2: AVS Setup

Credentials are required from Amazon to use the Alexa Voice service. To get those set up, refer back to this setup guide from Amazon, and now complete Section 6: 'Getting started with Alexa Voice Service'.

• Important: under 'Web Settings' use http://localhost:5000 for 'Allowed Origins' and as a 'Allowed Return URL', put http://localhost:5000/code. Note that this is using "http" instead of "https" in both cases.

These steps will create a new product type as a Device and create a new security profile. Make a note of your credentials as you will need them in the next step.

The Amazon AVS credentials are stored in a file called creds.py which is used by auth_web.py, pointBlue.py and pointRed.py. Take the example_creds.py with the blank values, put your credentials in and save the file as creds.py.

The auth_web.py is a simple web server to generate the refresh token via oAuth to the Amazon user's account, it then appends this to creds.py and displays it on the browser. Credit here goes to Sam Machin for the code.

To initiate generation of the refresh token, execute:

python ./auth_web.py

Then go to http://localhost:5000, and authenticate with your Amazon login. It will return a refresh token that you then need to add to the creds.py file on your RPI.

Phase 3: IoT Button Integration

As I got in to this portion of the project, I was exploring various options and ended up trying 2 different approaches:

• The AWS IoT Button: uses a publish/subscribe methodology over the internet

• The Flic IoT Button : uses a direct bluetooth connection to the RPI

Phase 3 Option A: AWS IoT Button

This was the first option I explored. The advantage to this approach is using the complete AWS ecosystem of services and the tight integration between them.

The required AWS IoT Button components are in the RPI folder. AWS has solid documentation for all things around their IoT service, you can find it here.

For initial setup:

• First go to the AWS IoT console here.

• Pick 'Get Started', and then create a Thing as below:

• Now create a Certificate, pick 1-Click certificate create, and check Activate. Important: be sure to download your private key, public key and certificate - private and public keys will not be retrievable after closing this form.

• Now create a Policy as shown below. In theResource field, type the ARN of your AWS IoT button, and then select the 'Allow' check box. This allows your button to publish messages to AWS IoT. The ARN of your AWS IoT button has the following form: arn:aws:iot:your-region:your-aws-account:topic/iotbutton/your-button-serial-number - you can find your AWS account number in the Account Identifiers section of the Security Credentials page. The AWS IoT button serial number is on the back of the button. Pick 'Allow', click 'Add statement', and then 'Create'.

• Now you need to attach the policy to your certificate to give the device the permissions specified in the policy. From the AWS IoT console, choose your device certificate, and from the Actions menu, choose 'Attach a policy'.

• Next attach your device (thing) to your certificate. Pick your device certificate, and from the Actions menu, choose 'Attach a thing'.

• Now you need to connect your AWS IoT button to wifi. Follow the directions here to complete that.

• With that configuration work in place, now open aws-iot-pointScored.js, and replace the button serial number with your own. Also, a quick note on the clientId in the device variable - it doesn't seem to matter what you put there.

• Finally, you will need to create a new directory under the PingPong/RPI structure called 'certs'. Copy into this directory the three files from above (certificate.pem.crt, private.pem.key and root-CA.crt).

After initial setup and/or each time you restart your RPI:

• Optional: sudo ifconfig wlan0 down - this turns off wifi if you are using a hard wired connection, I have found button response time to be better this way.

• Turn on listening for AWS IoT messages:

cd ~/PingPong/RPI/
node aws-iot-pointScored.js

Phase 3 Option B: Flic IoT Button

Another option I built out was the Flic IoT button. The advantage here is in the direct bluetooth connection between the button and the RPI, streamlining communication and user response time. The required Flic components are also in the RPI folder. Here is a reference for the use of the Flic SDK for Linux (currently in beta) if you get stuck or want to learn more about it.

For initial setup:

• Add execute to flicd permissions with sudo chmod aug+x flicd

• Disable a running bluetooth daemon to avoid possible interference with sudo service bluetooth stop

• Now start the flicd daemon in one terminal with

sudo ./flicd -f flic.sqlite3

• In another terminal, scan for your Flic button with node scanwizard.js , and follow the instructions to connect your Flic.

• Take note of your bdAddr number, and in the flic-iot-pointScored.js file, replace the number with yours as below:

After initial setup and/or each time you restart your RPI:

• Optional: sudo ifconfig wlan0 down - this turns off wifi if you are using a hard wired connection, I have found button response time to be better this way.

• sudo service bluetooth stop - this turns off bluetooth in prep for flicd daemon connection

• Shell 1:

cd ~/PingPong/RPI/
sudo ./flicd -f flic.sqlite3

• Shell 2:

cd ~/PingPong/RPI/
node flic-iot-pointScored.js

Give her a spin

At this point you should be ready to test the combined solution of both the voice interface with Alexa as well as the IoT button.

To get started, say "Alexa, use ping pong and start a new match". You will need to register a couple of players to get them into the system first. Once the match is initiated, click or double click the button to give a point to a player.

Now after a bunch of screen time, you can stretch your legs with some ping pong and hopefully get a laugh out of your kids or friends with the cheesy jokes :)