Game Station

I've been wanting to build some interactive games with the EV3 for many years, but I was always a bit disappointed with the limited options for providing feedback and interactivity.

When I discovered the possibility of using an Alexa Echo to provide some voice interaction with the EV3, it really opened my eyes to the possibility of not only using voice commands to trigger and play games, but also to provide a level of feedback and interactivity that would provide an amazing play experience.

I wanted to design the game console as simply as possible, so that it would be easy to reproduce and customize, yet still provide a versatile play experience. With Alexa providing audio feedback and instruction, I discovered that all that was needed to play many games were 4 buttons for input and 4 tracks to visually represent the position of players.

Using this setup, I was able to implement 5 different games, and I already have ideas for more to add in the future. The games currently implemented are Simon, Trivia, Musical Chairs, Hot Potato and Race to the Top.

A demonstration video for each game is included below, along with details on how they are played. I also go over some of the key coding concepts and challenges for some of them.

Project Overview

There are two main components to this project: the game console, built using the LEGO MINDSTORMS EV3 kit and some additional LEGO pieces; and an Alexa powered device, like the Amazon Echo Dot.

The program that runs on the EV3 manages all of the logic behind each game - responding to button events, moving the players, and managing the progress of the game. An Alexa Skill manages all of the logic on the Alexa device, issuing voice commands, playing music and testing answers for the trivia game. The EV3 and Alexa device communicate via messages sent over Bluetooth.

If you would like to recreate this project, and are unfamiliar with creating Alexa Skills or writing ev3dev programs for the EV3, I would strongly recommend going through the Setup Guide and the 4 Missions described on the LEGO MINDSTORMS Voice Challenge page - https://www.hackster.io/alexagadgets

These tutorials will go through all the details of setting up the development environment for your EV3 brick (python code running on ev3dev), writing an Alexa skill (Node.js code using the Alexa Skill Kit), and configuring your EV3 and Alexa skill to communicate with each other. The overall setup and configuration of this project matches that of these Missions.

Building the Game Station

You can find step by step building instructions for the console attached to this project. It is essentially just four vertically mounted tracks, each powered by an EV3 large motor.

Each track has an indicator on it to represent the position of each player.

Note that you do not need to build it the exact same way. All that is really required are four tracks driven by the four motors, built in whatever orientation or format you like.

The motors are connected to the four motor ports in order, from A to D.

For the buttons, if you want to get up and running quickly, you can just use the touch sensors on their own. As the only input device, I wanted a large, solid button design that would give the players a satisfying play experience.

I also knew the buttons might need to handle a fair amount of force, as the button pressing can get pretty hectic, especially playing the Race to the Top game. I wanted to design a frame around the touch sensor that would absorb the brunt of the impact. The design of the button has evolved through several iterations to get to the current point, and the result is a very sturdy and solid frame.

You can find the step by step building instructions for the buttons attached to this project.

Programming the Game Station

All of the source code for this project can be found in the following github repository: https://github.com/jasonallemann/gamestation

The EV3 code for the Game Station is written in Python and runs on ev3dev, an open source operating system that can run on the MINDSTORMS EV3 brick. The code is comprised of several files, which you can find in the gameStationEV3 folder in the github repository.

The gameStation.py file is the main program to run for the Game Station. It derives a class from the AlexaGadget class, which implements the code for responding to and sending messages between the EV3 and Alexa device.

The code for each game is implemented in its own class, in a separate file. When the MindstormsGadget class initializes, it creates an instance of each game as seen in the code below.

class MindstormsGadget(AlexaGadget):
    # Initialize the game objects.
    def __init__(self):
        super().__init__()
        console.initialize()
        self.s = simon.Simon()
        self.m = musicalChairs.MusicalChairs()
        self.h = hotPotato.HotPotato()
        self.r = race.RaceToTheTop()
        self.t = trivia.Trivia()

The following code snippet shows how the EV3 will process the directive for starting a game of Simon ("simonStartGame"). As you can see, it will ask the Simon object to start the game, and then send an event back to the Alexa device with the first colour sequence.

class MindstormsGadget(AlexaGadget):
...
    def on_custom_mindstorms_gadget_control( self, directive ):
        try:
            payload = json.loads( directive.payload.decode( "utf-8" ) )
            controlType = payload["type"]

            #
            # Events for Simon
            #
            # The skill has started a game of Simon.
            # Start the game, then tell the skill to read the first colour sequence.
            if controlType == "simonStartGame":
                self.s.startGame()
                self._send_event( "simonSequence", self.s.Sequence )

The program execution for each game follows a similar pattern. Alexa will send a message to start the game and the EV3 code will respond with an event that the game has started, with any additional information required. Events will continue to be sent back and forth as the player(s) progress through various stages of the game, in order for Alexa to provide feedback or further instructions. I illustrate this program flow in more detail in the description for the Simon game below.

The purpose of each python file in the gameStationEV3 folder is as follows.

• console.py - This module contains common code for controlling the console. This includes helper functions for moving the players up and down the console and responding to button presses.
• gameStation.py - The main program to run for the Game Station.
• hotPotato.py - Code for the Hot Potato game.
• musicalChairs.py - Code for the Musical Chairs game.
• offlineTests.py - This can be run as a separate program to test code independent of Alexa. Useful for testing the console code and includes functions illustrating the program flow for each game.
• race.py - Code for the Race to the Top game.
• simon.py - Code for the Simon game.
• trivia.py - Code for the Trivia game.

Programming the Alexa Skill

The code for the skill is written in Node.js and can be found in the alexa-skill folder in the github repository.

These files can be uploaded into your skill once it has been created in the Alexa Developer Console. Again, it is recommended to go through the Setup Guide and Mission tutorials linked in the Project Overview section if you are unfamiliar with creating a skill.

The Alexa skill also contains a model for the dialog interaction with the user, which is described in JSON format in the model.json file. For example, here is the dialog definition for playing the Simon game.

{
    "name": "SimonIntent",
    "slots": [
        {
            "name": "gameMode",
            "type": "GameModeType"
        }
    ],
    "samples": [
        "simon",
        "I would like to play simon {gameMode}",
        "simon {gameMode}"
    ]
},

Below is the Alexa skill code that gets executed when the user says they would like to play Simon. It creates the event to send to the EV3 to start the Simon game (the "simonStartGame" directive), as well as starts the event handler to listen for events that the EV3 will send back to the Alexa device.

const SimonIntentHandler = {
...
    handle: function( handlerInput )
    {
        const attributesManager = handlerInput.attributesManager;
        const endpointId = attributesManager.getSessionAttributes().endpointId || [];

        console.log( '#### Start playing Simon. ####');
        
        // Set up the directive to tell the EV3 to start playing Simon.
        const directive = Util.build(endpointId, NAMESPACE, NAME_CONTROL,
            { type: 'simonStartGame' } );

        // Set the token to track the event handler
        const token = handlerInput.requestEnvelope.request.requestId;
        Util.putSessionAttribute( handlerInput, 'token', token );

        return handlerInput.responseBuilder
            .speak( randomConfirmation() + `let's play Simon.` )
            .addDirective( directive )
            .addDirective( Util.buildStartEventHandler( token, 60000, {} ) )
            .getResponse();
    }
};

The code files for the skill are as follows.

• index.js - The code for all the main intent handlers, and the handler for events sent from the EV3.
• util.js - Utility functions.
• common.js - Common intent handlers.

Natural Responses

In many of the responses during game play, I've tried to randomize what Alexa says to make her sound more natural. For example, when winning a game, Alexa will congratulate you in one of three different ways.

function randomSuccess()
{
    var Response = "Congratulations ";
    var option = Math.floor( Math.random() * 3 );

    if( option === 0 ) { Response = "Congratulations "; }
    else if( option === 1 ) { Response = "Great job "; }
    else { Response = "Good job "; }

    return Response;
}

I have used the same strategy when assembling responses for correct answers, incorrect answers, failure notifications and confirmation responses.

Here's another example for the beginning of a confirmation response.

function randomConfirmation()
{
    var Response = "Okay, ";
    var option = Math.floor( Math.random() * 4 );

    if( option === 0 ) { Response = "Okay, "; }
    else if( option === 1 ) { Response = "Sure, "; }
    else if( option === 2 ) { Response = "Allright, "; }
    else { Response = "You got it, "; }

    return Response;
}

Playing the Games

Once the gameStation.py program is running on the EV3, you can launch the Game Station skill by saying the following.

"Alex, open the Game Station".

This invocation name is stored in the model.json file as shown below, so you can modify it if you wish to whatever you like.

...
"languageModel": {
"invocationName": "the game station",
"intents": [
...

Once the skill is activated, Alexa will ask you what you would like to play. You can respond in many different ways.

"I would like to play a game of Simon."
"Musical Chairs"
"Let's play Hot Potato"

You can also ask the Game Station which games are available and how to play them.

"What games can I play?"

"How do I play Race to the Top? "
"What are the instructions for Trivia?"

Alexa will answer appropriately, saying which games are available or explaining how to play them.

Simon

1 player - Memory

The first game I wanted to implement was Simon, the memory game where you are asked to remember and replicate a sequence of colours. I'd been wanting to build a Simon game for many years, but I could never find a satisfactory way of implementing it. Using Alexa to speak the colour sequence for each level immediately struck me as a perfect way to do it.

As you can see in the video, in the first level there is only 1 color and each subsequent level adds a randomly selected colour to the sequence. For every level, Alexa will speak the sequence of the colours, then the player needs to replicate the sequence by pressing the coloured buttons in the same order.

The game can, in theory, go on forever, but I have set the game to have a maximum of 10 levels, so the player can feel good about achieving an end.

The following list outlines the general program flow of the game, and the communication between Alexa and the EV3.

1 - Player asks Alexa to start game "Let's play Simon".

2 - Alexa sends "simonStartGame" event to the EV3. See the SimonIntentHandler function in index.js.

// Set up the directive to tell the EV3 to start playing Simon.
const directive = Util.build(endpointId, NAMESPACE, NAME_CONTROL,
{ type: 'simonStartGame' } );

3 - EV3 starts the game and generates the initial colour sequence (a single colour for the first level). Code from simon.py:

def startGame( self ):
    self.CurrentLevel = 1
    self.Sequence.clear()
    self.Sequence.append( random.choice( ["red", "green", "blue", "yellow"] ) )

4 - EV3 sends "simonSequence" event to Alexa, with JSON payload of the colour sequence. Code from gameStation.py:

self._send_event( "simonSequence", self.s.Sequence )

5 - Alexa speaks the colour sequence and sends the "simonTest" event to the EV3. Code from index.js.

for( var i = 0; i < payload.length; i++ )
{
    if( i !== 0 ) { speechOutput = speechOutput + " "; }
    speechOutput = speechOutput + payload[i];
}

const directive = Util.build(endpointId, NAMESPACE, NAME_CONTROL,
{ type: 'simonTest' } );

return handlerInput.responseBuilder
    .speak( speechOutput, "REPLACE_ENQUEUED" )
    .addDirective( directive )
    .getResponse();

6 - EV3 waits for button presses and compares them to the colour sequence of the current level. See test function in simon.py.

7a - If the player successfully matched the sequence, a colour is randomly added to the sequence and the process is repeated from step 4. Code from test function in simon.py.

self.CurrentLevel += 1
self.Sequence.append( random.choice( ["red", "green", "blue", "yellow"] ) )

7b - If unsuccessful, or the player successfully matched the last level, the game is ended and the EV3 sends a "simonEnd" event to Alexa, with information on how the game ended in the payload. Code from gameStation.py.

endSimonData = { "level": self.s.CurrentLevel - 1, "passed": result }
self._send_event( "simonEnd", json.dumps( endSimonData ) )

8 - Alexa reports the results of the game and sends the "simonEndGame" event back to the EV3. Code from index.py.

if( payloadJSON.level >= 10 && payloadJSON.passed === true)
{
    speechOutput = CorrectSound + "Congratulations! You successfully finished level ten. What would you like to play next?";
}
else if ( payloadJSON.level <= 1 )
{
    speechOutput = IncorrectSound + "Wow, you aren't very good at this one. Maybe you should try another game.";
}
else
{
    speechOutput = IncorrectSound + randomFail() + ", that wasn't right, but you successfully completed level ";
    speechOutput += String( payloadJSON.level - 1 );
    speechOutput += ". What would you like to play next?";
}

const directive = Util.build(endpointId, NAMESPACE, NAME_CONTROL,
{ type: 'simonEndGame' } );

9 - The EV3 does the final game cleanup. Code from simon.py.

def endGame( self ):
    console.resetPlayers()
    console.waitForPlayers()

Trivia

2-4 players - Knowledge

A trivia game was an absolute must for the Game Station, since it so perfectly illustrates the benefit of using Alexa to interact with the EV3. This video shows the trivia game being played by 2 players.

For this game, I decided to use LEGO as a theme for the questions, but, of course, this can be easily customized. The questions are stored in JSON format in the triviaQuestions.json file in the gameStationEV3 folder.

{
    "questions": [
    {
        "question":"In which country was LEGO founded?",
        "hint":"Sweden, Denmark, or Germany",
        "answer":"denmark"
    },
...
}

The "question" and "answer" values are self explanatory. The "hint" value is optional, and can be used to give the players a hint for the answer. The "hint" text will be spoken after the question is asked and repeated if a player incorrectly answers the question. It can be any text at all, and need not be a list of 3 possible answers.

This makes for some interesting game play, as when a player presses a button while the question is being asked, the audio will be interrupted, so even though they are rewarded with the first try at the answer, they may not hear the hint.

At the start of each game, all of the questions are loaded from the file.

def startGame( self ):
    try:
        f = open( "triviaQuestions.json", "r" )
    except FileNotFoundError:
        self.TriviaQuestions = [ { "question":"none" } ]
    else:
        contents = f.read()
        questionData = json.loads( contents )
        self.TriviaQuestions = questionData["questions"]
        f.close()

Whenever a question is asked, it is randomly selected from the list, then removed so that it is not asked again.

def randomQuestion( self ):
    length = len( self.TriviaQuestions )
    questionIndex = random.randint( 0, length - 1 )
    
    question = self.TriviaQuestions[questionIndex]
    self.TriviaQuestions.remove( question )
    
    return question

Each time a player answers a question correctly, their player is moved up the console, and the first to answer three questions correctly is the winner. This, again, can be easily modified, or even be set as a parameter when launching the game.

def correctAnswer( self ):
    gameOver = False
    self.playerScores[self.AnswerPlayer] += 1
    console.movePlayer( self.AnswerPlayer, console.Third )

    if self.playerScores[self.AnswerPlayer] == 3:
        gameOver = True

    return gameOver

Musical Chairs

4 players - Reaction Time

The ability for Alexa to play music made it clear that a game of musical chairs was in order. You can see it being playing in the video below.

Since there are obviously no chairs involved, I simply use the position of a player on the console to indicate if they 'have a chair' or not. At the beginning of the game, all the players are moved up the console and Alexa will begin to play music.

There are currently three different music tracks, which Alexa randomly chooses from. They are stored in, and loaded from, the audio folder in the github repository.

const Music1 = '<audio src="https://raw.githubusercontent.com/jasonallemann/gamestation/master/audio/muffinman20s.mp3"/>'
const Music2 = '<audio src="https://raw.githubusercontent.com/jasonallemann/gamestation/master/audio/bunnyhop20s.mp3"/>'
const Music3 = '<audio src="https://raw.githubusercontent.com/jasonallemann/gamestation/master/audio/twirlytops20s.mp3"/>'

function randomMusic()
{
    var AudioSSML = Music1;
    var option = Math.floor( Math.random() * 3 );
    if( option === 0 ) { AudioSSML = Music1; }
    else if( option === 1 ) { AudioSSML = Music2; }
    else { AudioSSML = Music3; }
    return AudioSSML;
}

For each round, the music is played for a random amount of time between 2 and 10 seconds.

At the end of each round, the player that is slowest to press their button is moved to the bottom of the console, indicating that they did not get a chair, and the last player remaining is the winner.

Accurately determining what order the buttons are pressed was quite challenging, as they can be pressed almost simultaneously. Just before the music stops, I record the time as a reference point.

self.startTime = time.time()
self.redTime = self.greenTime = self.blueTime = self.yellowTime = 0

Then start a thread to monitor each button, here is how the red button thread is started.

if self.PlayerStatus["red"] == True:
    self.tRed = threading.Thread( target=self.redButtonTime, daemon=True )
    self.tRed.start()

The four threads simply wait for them to be pressed, and record the time that they were pressed. Here is the thread function for the red button.

def redButtonTime( self ):
    console.waitForButton( console.RedPlayer )
    self.redTime = time.time()

Once all four buttons are pressed, the times are then compared to the start time and put in order to determine the results.

Hot Potato

4 Players - Luck

Once I finished the musical chairs game, it seemed only natural to write a Hot Potato game as well, since they both involve playing music and something happening when it ends.

Just like in the musical chairs game, I use the players position on the console to indicate if they 'have the potato' or not. At the beginning of the game, all the players are moved up the console, the music starts, and the player with the potato is moved up even higher.

A piece of music is randomly selected and plays for a random amount of time between 5 and 12 seconds.

The player with the potato can pass it on to the next player by pressing their button. If they have the potato when the music stops, they are eliminated and the remaining players play again. The final player remaining is the winner.

Race to the Top

2-4 players - Speed

As a child of the 70s and 80s, I have fond memories of some of the early arcade and console games where the goal was simply to tap the button as fast as possible. I simply had to replicate that idea here.

The goal of this game is quite simple. The faster you tap you button, the quicker your player will move up the console. First player to the top wins!

It was particularly challenging to capture all the button presses and update the position of the player in a timely manner. Again, separate threads were created for each button to maintain a count of how many times they are pressed. Here's the thread function for the red button.

def redButtonCountThread():
    global RedCount
    while TouchSensor( INPUT_1 ).wait_for_bump( None, 50 ):
        RedCount += 1

When the game is started, all of the button counts are reset and the individual button threads are started.

While the game is being played, the press count for each button is continuously being polled and the player position on the console is updated to reflect the count.

while self.PlayingMadDash:

...

    self.RedCurrent = console.RedCount

...

    self.RedCurrent = min( self.winCount, self.RedCurrent )
    if self.RedCurrent > RPrevious:
        console.moveRedPlayer( ( self.RedCurrent - RPrevious ) * self.motorStep )
        RPrevious = self.RedCurrent

Future Possibilities

There are a lot of possibilities for modifying these games and adding more. Pretty much any parameter of the current games can be easily changed. I have listed a few ideas below, but of course, there are many others, and you are only limited by your imagination!

The Musical Chairs and Hot Potato games could be changed to be double or triple elimination, so that each player is only eliminated after 'failing' two or three times.

The number of correctly answered Trivia questions can be increased, or a player could be penalized for incorrectly answering a question, moving down the console.

The Simon game can be modified to randomly reorder all of the colours each round, which would make it more difficult. The 10 round limit can also be changed.

A 'Whack-A-Mole' game could be created, where the player has to tap the corresponding button every time a minifig moves up the console to bring them back down.

I also think it would be awesome to have some kind of tournament mode, where players must play multiple games, or a single game multiple times, and stats are stored and accumulated throughout.