Roxie The Voice Activated Pitching Machine

Introduction

Voice commands will revolutionize many products that today rely on keyboards and screen gestures. Have you ever said "I wish I had an extra set of hands?" That's a sign that potentially a voice interface would be useful given that it's a different way of interacting. In teaching my three daughters how to play softball, I've said that many times, and believe a voice activated pitching machine would be a great product - so I built one!

Problem

Hitting a moving baseball is a challenging skill to learn. It requires timing to connect a swinging bat to a moving ball, and a hands-on approach to teaching. I've found that much of the attention of the student gets focused on attempting to predict when the ball is coming vs. listening to tips on swing, footwork, etc. No doubt why - there's a ball coming at you!

Solution

A voice interface frees up our hands to do other things, and when applied to a pitching machine, makes it like having an extra person around to help out.

I've published the skill in the Alexa Store under the name "Robot Roxie", and all the source code is in the GitHub repo attached to the project below, or can be found at https://github.com/terrenjpeterson/roxie-skill.

Layout

We're going to assume that this is something done outdoors which begins to create some elements that can be challenging to work with. First we need to make sure there is enough distance between the pitching machine and hitter for swinging, and the Echo needs to be close enough to the hitter (or whomever is giving instructions) that it can effectively hear what is being said. Both the Echo and the Raspberry Pi need to be connected to the internet as well given that they aren't directly linked.

Key Assumption - Network Connectivity

When building this solution, we're assuming connectivity to the internet is available. This can be through a home Wi-Fi network, or if taking this to a park or field, a cellular phone can be used to tether the devices to the internet. I've found in testing within my front-yard that my Echo works to connect to my Wi-Fi network (probably 75 feet from the wireless access point) but the Raspberry Pi can't pick up a signal even 50 feet from it. One solution is to leverage another device like a phone or iPad to bridge by tethering. I've done this and it has the added feature that your phone can tell you when the RaPi is holding a connection.

Key Assumption - Power Supply

We can build this for a number of different skill levels and even sports. Throwing machines are also used in tennis, and I'd assume it could also be applied to cricket. The one major consideration is how heavy are the balls that are being thrown and for how far. For my example, I'm just using light-weight plastic balls, so the motors needed to drive them can be powered by batteries. If you're looking to use this for heavier balls, it can still work - just need to keep in mind that something stronger will be needed to throw them, and likely access to a 120v power supply is needed.

Solution Build

Step 1 - Create a Raspberry Pi Driven Pitching Machine

First you need to create a throwing mechanism. Pitching machines come in two types, a catapult which is mechanical, and a wheel driven solution that is driven by electric motors. For this project I'm using the electric version that uses two wheels driven by motors to send the ball flying through the air.

Next you need to create a release mechanism that controls the balls entering the throwing wheels.

Option #1 - Enhance an Existing Pitching Machine

A quick way to build this is to take an existing pitching machine that already has the existing wheels and release mechanism, so just find one and take it apart so you can get familiar with how the wiring works. As for the instructions, skip down to the section called "Controlling the Release" as the next option will be in how to build this from scratch.

Option #2 - Build a Pitching Machine from Scratch

For this project, I'm building a pitching machine that can throw plastic balls, so the size of the wheels and the motors aren't very large.

Making the Throwing Wheels

The throwing wheels will need to propel the ball in the air, so depending on the weight of the ball, larger motors and wheel sizes will need to be used. You will need to build two of these, and I made mine with car wheels, and 12V DC motors that are rated at 8,000 RPM.

The collet is what attaches the wheel to the motor shaft, and allows for tight coupling without using glue or apoxy that would permanently mount it. I strongly suggest using superglue to bind the tires to the plastic wheels as well in case these come in two pieces. Given the speed at which the wheels will be spinning at, the tires can come off the wheel in parts, requiring the game to be stopped.

Building the Motor Frame

Once two throwing wheels are made, a frame needs to be created that will space the two apart. Here's a view of how to make the measurements, and we will need two small pieces of wood that can easily be drilled.

Careful measurement is needed in how to space the two sets of holes. The rubber tires will have some grip, so allow for 1/8" overlap when adding the measurements together. Once these pieces are cut, drop the motors into the slots.

Then add pieces to the create the sides. This will allow mounting vertically on a small piece of plywood. Here's what the final product should look like.

Building a Ball Feeder

The next step is to build a ball feeder that can serve up the ball into the wheels. This can be done by building a small platform that will drop the ball at the right height underneath the top wheel, and into the bottom wheel. This one is done with some scrap pieces of plywood, and narrow (1/2" x 3/4") strips of pine. Once again this will need to be based on the size of ball you are attempting to throw.

Building a Ball Frame

Since we are going to want to have Roxie throw multiple balls, we need to build a frame that can stack the balls, then provide them to the feeder one at a time.

Once again I'm using narrow strips of pine that are sized based on the plastic balls that Roxie will be throwing. Here's a picture of the frame before the stand is attached, and this can load approximately ten balls. It's in a figure of a number four, and could be modified based on how many balls you would like to load.

Next we need to add a stand positions the frame such that gravity can feed the balls down to the bottom. This creates a little of a maze where the balls can then stack up.

Controlling the Release

Traditional electric pitching machines use a timing mechanism to release the ball, where the ball throwing frequency is on an interval - say 10 seconds. While this seems simple, it actually dictates much of the activity, as you find yourself focusing as much on the cadence of the balls coming out as fixing your swing. This is where improving the gameplay comes in as we're going to leverage the voice commands of Alexa. To do this, we will drive the timing from an electric relay that acts as a switch that is driven by an electric signal.

The attached schematics (see bottom of project) highlight how the wiring should go, and to control the relay from the Raspberry Pi, we will leverage the GPIO library. Here's the lines of python that do this within the rotate.py script.

# import GPIO library to connect to relay and time library for delays
import RPi.GPIO as GPIO
import time
# set parameters for IO on raspberry PI
relay_pin = 12
rotor_duration = 10
# configure IO on raspberry PI to communicate
GPIO.setmode(GPIO.BOARD)
GPIO.setwarnings(False)
GPIO.setup(relay_pin, GPIO.OUT)
# turn on the relay
print "Relay Active"
GPIO.output(relay_pin, GPIO.HIGH)
# pause
time.sleep(rotor_duration) 
# turn off the relay
print "Relay Off"
GPIO.output(relay_pin, GPIO.LOW)
# release resources when done
GPIO.cleanup()

By executing this script, the motor will turn on for the number of seconds. We can adjust the release mechanism based on changing the 'rotor duration' parameter as we only want to release one ball at a time to the wheels, and this will depend on the characteristics of the motor.

Next we will attach the release to the ball frame, and then connect the release motor, relay, and Raspberry Pi. Once these pieces are completed, we should be able to control the release by running the rotate.py script that was written above.

The next step is to connect the release with the chute that places the ball into the wheels. Here is an overhead view of how this connects the two parts together.

Now we have built a pitching machine that can be controlled by a Raspberry Pi.

Build a Physical Avatar

To make this a little more fun, I've added a cutout for Roxie. To do this, start with a piece of 2' x 4' particle board, and trace out the body. I've marked the dimensions on this for reference. With a jigsaw, cut out the body per the pattern.

After cutting out the shape, use some acrylic paints to give color and features to the avatar.

Step 2 - Create Alexa Skill to Interact With User

There are many great resources (like this) to explain how to create Alexa Skill, so I'm not going to repeat all of the steps here, rather focus on what is specific to this invention. To begin, let's start by just creating an utterance for "Pitch Ball". Below are the statements to add to the sample utterance file of the Alexa skill.

PitchBall pitch ball
PitchBall pitch the ball
PitchBall pitch

I'm going to use a Lambda function written in NodeJS to contain the rules. The source for this can be found in the GitHub repo under the lambda.js file. When starting the skill, the getWelcomeResponse(...) function gets called, and here is the section of code that sets the voice interaction back to the user.

function getWelcomeResponse(callback) {
 var sessionAttributes = {}; 
 var shouldEndSession = false; 
 var cardTitle = "Welcome to Robot Roxie"; 
 var speechOutput = "Hello, my name is Roxie and I'm a really big fan of playing baseball. If you would " + 
     "like to play a game, say Start Game and I will walk you through the steps needed to play a two " + 
     "player game. If you just want to practice, then please say " + 
     "Pitch Ball and I will throw you a ball."; 
 var cardOutput = "Roxie the Robot Pitcher"; 
 var repromptText = "Please tell me when you are ready to pitch a ball to you by saying, " + 
     "Pitch Ball or begin a game by saying Start Game."; 
 console.log('speech output : ' + speechOutput); 
 callback(sessionAttributes, 
     buildSpeechletResponse(cardTitle, speechOutput, cardOutput, repromptText, shouldEndSession)); 
} 

Once the user responds with one of the utterances intending to pitch the ball, the pitchBall(...) function gets invoked. More details on how this integrates will be described in the next section, and here is a video showing the voice interaction with an Echo.

As noted above, this skill is published in the Alexa store, so you can try out "Robot Roxie" for yourself!

Step 3 - Connect the Raspberry Pi to the Alexa Skill

Now we have a Raspberry Pi driven pitching machine, and a skill that can translate a voice command into functions. Connecting the two together will leverage a message queue, with the skill writing to the queue, and the RaPi reading the queue. Here's a picture showing how this works.

Alternatively, you can use MQTT and the AWS IOT to setup. Here are instructions.

http://docs.aws.amazon.com/iot/latest/developerguide/iot-console-signin.html

Security and Access

(note: given the secure nature of this information, I'm not showing screen shots of specifics around these steps).

To allow a Raspberry Pi to communicate with your AWS account, you first need to setup a "user" that will be representative of the Robot that we are building. Detailed instructions to do this can be found here, and there isn't anything unique about the user name (for example, I called the user "Roxie" on my account).

http://docs.aws.amazon.com/IAM/latest/UserGuide/id_users.html

Once you have created the user, you will need to create an access key for the user. Instructions for how to configure this will be found in the next section.

Raspberry Pi Configuration

We will be using the python libraries already natively available on the Raspberry Pi. In addition, we will be using the AWS SDK to connect, so some additional packages are required to be downloaded. Here's the link with more specifics.

https://aws.amazon.com/sdk-for-python/

Once you have the SDK installed, you'll need to download an access key from the AWS account you are running the Lambda function from.

Queue Setup

We will be using a service called SQS to communicate. Here's the AWS website that contains details on the service.

https://aws.amazon.com/sqs/

Follow the instructions on the console to create a new message queue that will be used to communicate between the Skill and the RaPi. We will be calling the queue "talkWithPitcher" and the configuration screen should look like this.

When troubleshooting, we can always use the AWS console to check if there are messages in the queue, or to manually put messages on the queue that can be consumed by the RaspberryPi.

Building the Interface

The json object that was created in the function gets passed in the queue, then consumed by the RaPi. There isn't much code to write for this, here are the lines that need to be included.

var sqs = new AWS.SQS({region : 'us-east-1'});
// set parameters for message queue to transport data
var AWS_ACCOUNT = <enter account as a string>;
var QUEUE_NAME = 'talkWithPitcher';
var QUEUE_URL = 'https://sqs.us-east-1.amazonaws.com/';
// set parameters for message queue to transport data
var params = {
MessageBody: JSON.stringify(sendData),
QueueUrl: QUEUE_URL + AWS_ACCOUNT + '/' + QUEUE_NAME
};
// send message to SQS and return back message to Alexa
sqs.sendMessage(params, function(err,data){
if(err) {
console.log('error:',"Fail Send Message" + err);
} else {
console.log('successful post - data:',data.MessageId);
callback(sessionAttributes,
buildSpeechletResponse(cardTitle, speechOutput, speechOutput, repromptText, shouldEndSession));
}
});

Then reading the queue uses the Python SDK (called Boto).

# import boto library that handles queuing functions
import 
# connect to queue that will have incoming requests
conn=boto.sqs.connect_to_region("us-east-1")
my_queue = conn.get_queue('talkWithPitcher')
# check queue to see if a request exists
incomingMsgs = my_queue.get_messages()
# if messages are found, process them
if len(incomingMsgs) > 0:
<execute the python script>

With all of the pieces working, we can take it outside and begin to practice!

Step 4 - Play Ball

Now that all of the individual pieces have been constructed, we can connect them all together. The ball frame and feeder is what communicates with the Alexa skill, and the throwing wheels then send the ball to the hitter. The physical avatar for Roxie acts as a stand that sits in front of the pitching machine, and helps tie the automation together and prepare the hitter for where the ball is coming from.

Now we have a voice activated pitching machine, so lets play ball!

Step 5 - Enhance the Experience by Expanding the Alexa Skill

Now that we've got a basic version of this working, let's add a little fun to it by leveraging the full power of the Alexa Voice platform. This is where the "Robot Roxie" skill comes in that is published in the Alexa store.

We started by having Alexa process the utterance of "Pitch Ball", but let's expand this into a simple two-player game where it tracks who's turn it is, and include some basic scoring. Here's the Voice User Interaction that highlights the capabilities of the game.

This supports two players, and the names can be configured and then pulled into the voice interaction. Here's a demonstration of starting a game.

The configurations for this can be found in the GitHub repo, and to understand how we're managing the gameplay, we're using the session attributes passed between the request/response. Here is the section that contains the attributes.

"attributes":{
"data": {
"gameMode": true,
"playerOne": {
"name": "Player One",
"score": 0
},
"playerTwo": {
"name": "Player Two",
"score": 0
},
"homeTeamAtBat": false,
"gameLength": {
"innings": 3,
"pitchPerInning": 10
},
"currInning": 1,
"currPitch": 0
} 
}

Included in the voice commands is the ability to configure the game, for example, customizing the player names. This makes "Roxie" more user friendly when pitching as it then speaks the name when it's ready to throw a ball.