Unity Game Engine and Arduino Serial Communication

History

I have been working on an indie game (Son.Light.Sleepwalker on Steam) for a few years now. With VR finally being within reach of the general consumer, I thought - how could I make it even more immersive? What if I could make wind blow on the player by turning on a fan or turn on lights or a radio in their house?

This led to a lot of research to get Unity, the game engine, to send serial commands to the Arduino. Then from there, I could control anything imaginable in response to the game.

Hidden Maker Mode

This led to a hidden feature in my indie game - Maker Mode. When you pause the game and press the letter M, it polls the COM ports for a USB connected microcontroller and begins to interact by sending it commands. In Son.Light.Sleepwalker, you will see the LED blink 5 times when it connects.

Unity Setup

In this project, we will use C# in Unity to build our app. Download Unity from Unity3D.com. It is an incredible, free 3D Game Engine if you haven't already made $100, 000 in a year using Unity.

We will not get into the details of 3D game programming in this guide. Rather, we'll dive into the minimum requirements needed to get your game talking to your microcontroller. If you have any other questions, though, feel free to post a comment.

When you first open Unity, you will have a default game "scene". A scene is basically the stage at which the camera is first set where the first action occurs. The only objects (called game assets) in the scene are the main camera and a light source.

In the Unity Editor Hierarchy pane, click on the Directional Light. The Inspector Pane on the right will update to show you properties of the light.

Scroll to the bottom and click the "Add Component" button. Scroll to the bottom of the options presenting and select "New script". I named my script "SeanSerialTesterScript". You can name yours differently, but you will need to alter the code in Step 3 accordingly. Last, click "Create and Add".

Before we proceed, we need to set up your Unity Player to access the Serial Ports.

To adjust the Unity Player for Serial Port Access, we have to tell it to use the full.Net references, not just a subset as with the default.

On the menu bar at the top of the Unity Editor, click Edit-->Project Settings-->Player. The Inspector Pane on the right will now show the "Player Settings" properties. Scroll to the section called "Other Settings". If it is not already expanded, click on it. Now scroll to the Configuration sub-section.

Refer to the image above to for the settings you need. You may have newer.Net implementations you desire (ie, 4.x versus 3.5). The big thing is to not use the "subset" version for the API Compatibility Level as it does not include Serial Port access.

Go ahead and click save and give your project a name. You are now ready to paste the code.

Click back on your Directional Light and scroll to the script component you added. Doubleclick on the name of the script and the code editor will open.

Paste the following code into the editor:

using UnityEngine;
using System.IO.Ports;
public class SeanSerialTesterScript : MonoBehaviour {
    SerialPort sp;
    float next_time; int ii = 0;
    // Use this for initialization
    void Start () {
        string the_com="";
        next_time = Time.time;
        
        foreach (string mysps in SerialPort.GetPortNames())
        {
            print(mysps);
            if (mysps != "COM1") { the_com = mysps; break; }
        }
        sp = new SerialPort("\\\\.\\" + the_com, 9600);
        if (!sp.IsOpen)
        {
            print("Opening " + the_com + ", baud 9600");
            sp.Open();
            sp.ReadTimeout = 100;
            sp.Handshake = Handshake.None;
            if (sp.IsOpen) { print("Open"); }
        }
    }
    // Update is called once per frame
    void Update() {
        if (Time.time > next_time) { 
            if (!sp.IsOpen)
            {
                sp.Open();
                print("opened sp");
            }
            if (sp.IsOpen)
            {
                print("Writing " + ii);
                sp.Write((ii.ToString()));
            }
            next_time = Time.time + 5;
            if (++ii > 9) ii = 0;
        }
    }
}

Note: You may need to rename the main class if you chose a different name in Step 1.

This code will send a number from 0 to 9 every five seconds to the microcontroller. Once it passes 9, it starts back at 9 until you stop the program.

There are some "print" statements that are there just for debugging purposes. You can see them in the Unity Editor Console as the code executes.

Any microcontroller that can connect to a PC Serial Port via USB will work. In this example, we are using the Arduino MKR1000. It has WiFi capability allow for further real world effects.

Assuming you have an Arduino and the Arduino IDE installed, paste the following code into a new Arduino Sketch:

int inByte;
void setup() {
  // start serial port at 9600 bps:
  Serial.begin(9600);
  pinMode(LED_BUILTIN, OUTPUT); 
  digitalWrite(LED_BUILTIN, LOW);
}
void loop() {
  // if we get a valid byte, read analog ins:
  if (Serial.available() > 0) {
    // get incoming byte:
    inByte = Serial.read();
    if ((57-inByte)==0)
    { //after pressing m in pause screen and connected
      blink_it (10);
    }
    
    if ((57-inByte)==1) 
    {
      //Saw kid    
      blink_it(1);
    }
    if ((57-inByte)==2) 
    {
      //Lightening/Thunder
      blink_it(2);
    }
    if ((57-inByte)==3) 
    {
      //Spooky Noises
      blink_it(3);
    }
    if ((57-inByte)==4) 
    {
      //Sudden sound like the door being beat on
      blink_it(4);
    }
    if ((57-inByte)==5) 
    {
      //Chilling Ambient Sound
      blink_it(5);
    }
    if ((57-inByte)==6) 
    {
      //door creak
       blink_it(6);
    }
    if ((57-inByte)==7) 
    {
      //voices talking
       blink_it(7);
    }
    if ((57-inByte)==8) 
    {
      //drone woke kid
       blink_it(8);
    }
    if ((57-inByte)==9) 
    {
      //next chapter
       blink_it(9);
    }
  }
  delay(500);
}


void blink_it(int the_count){
  for (int ii=0;ii<the_count;ii++){

     digitalWrite(LED_BUILTIN, HIGH);
     delay(150);
     digitalWrite(LED_BUILTIN, LOW);
     delay(150);
  }        
}

Deploy the code by clicking the icon at the top left of the Arduino IDE.

The code polls for available Serial commands from the PC. Once it receives one, it then compares the value with If statements to determine which routine to run. To keep it simple for this guide, we just have it blink the onboard LED to let you know what command was sent. If a 5 was sent, it will blink 5 times, etc.

Leave the Arduino connected to the PC and go back to Unity, or run Son.Light.Sleepwalker to test it. In Unity, click the Play button. You should see output similar to what is pictured.

Summary

If you are a Maker, you know that once you blink an LED on a microcontroller, the sky is the limit as to what you can do. With so many open source tools, you are only limited by your imagination.

For example circuits to use with Unity, check out my 4D IoT game engine blog: 4D IoT Game Engine Blog | element14 | sjmill01

For fun deep dive into the project, watch us on YouTube:

Happy Making!

Sean