Low-Latency Teleoperation with Raspberry Pi and Surrogate.tv

Surrogate.tv is a platform that allows you to easily create remote play and teleoperation experiences, even if you don't know how to code. You can connect things like consoles, robots, RC cars, robots, or anything else to the internet and let anyone in the world control it remotely. In this guide we will show you how to set up your hardware, connect to the platform and become a creator on Surrogate.tv.

This guide is aimed at people who've never created a game on Surrogate.tv: we will begin from scratch, and within an hour you should have your very own game up and running on Surrogate.tv! After following this guide, you can go on to use one of our easy-to-set-up template games or create a custom game using our open-source Python SDK. For instance, you can create an internet controlled RC car in just 20 minutes!

If you encounter any issues during the setup, or just want to say hi, come join our Discord community!

Step One: Gathering the Hardware

To get started, you only need a Raspberry Pi, a microSD card with at least 16 GB of capacity, a camera, and an SD card reader. Make sure that the microSD card does not have any files you don't want to lose, as all files in it will be lost during the flashing process.

Plug the camera into the Raspberry Pi. If using a USB camera (or capture card), plug the camera into a blue USB port if possible (blue ports support USB 3.0). If using the Raspberry Pi camera module, make sure that the contacts of the ribbon cable are on the correct side on both the camera and the Raspberry Pi (facing away from the black bit) and that the ribbon cable is inserted into the camera port, not the display port.

Don't connect the power supply to the Raspberry Pi just yet.

Pro tip: to make theprocess even speedier, start downloading the Surrogate Image file (approximately 1.2 GB) now.

Step Two: Signing Up

Go to Surrogate.tv and create an account by clicking the "Sign up" button in the upper right corner. You can either use your Google or Facebook account, or create an account using your email address. If signing up via email, you will receive an email with a confirmation code that you must submit to complete the account creation process. Check your spam folder if you haven't received the confirmation email within a minute or so.

Next, log in to your Surrogate.tv account by clicking the "Log in" button, also in the upper right corner. When logging in for the first time, you will be prompted to enter your username. Don't worry, you can change it later!

Step Three: Create a Game

Once you've logged in, click the "Create a game" button in the upper right corner (déjà vu anyone?). As this is your first time creating a game, our goal is to just get your system up and running — no prior development is needed!

Check that you have everything required to create a game and then click "Start".

Next, choose the title and URL (web address) for your game. These, too, can be changed later. To continue, click "Next".

Step Four: Preparing the Raspberry Pi

Download the Surrogate Image if you haven't already done so. Also download and install the Raspberry Pi Imager.

With the image downloaded and imager installed, put the SD card in your SD card reader and open Raspberry Pi Imager. Click "Choose OS" and select the Surrogate Image file you downloaded. Then click "Choose storage" and choose the SD card you want to use. Remember that all its contents will be destroyed in the process. Finally, click "Write" and wait for the flashing process to complete. It may stay at "Writing... 99%" for a while; just be patient. It's best to wait for the verification step to complete as well, but if you're in a hurry you can opt to cancel the this step at a small risk of the image not working.

When the image has been written to the SD card, you may remove the card from the reader and insert it into the Raspberry Pi. Make sure that the SD card and camera are properly in place.

Next, connect the power supply to the Raspberry Pi. The red LED at the end of the device should light up.

Step Five: Connecting the Raspberry Pi

If you don't remember the name or password for your WiFi network, look them up now. They are needed in the next part of the process.

Click "Next" on the game creation page.

Copy the token displayed on the page, then connect to the WiFI hotspot provided by the Raspberry Pi (network name: surrogate-rpi, password: surrogatetv). It may take a while for it to show up.

Once you've connected to the surrogate-rpi network, click "Continue". This will open up a new tab, which should show the Controller Setup page. Paste the token you copied previously to the token input field. (If you forgot to copy it, just go to the other tab and copy it now.)

Click "Continue" to progress to the next step.

On this page you only have to add the name of your WiFi network to the SSID field and WiFi password to the Password field (to be clear: these refer to your WiFi network you use to connect to the internet, not the hotspot network SSID and password given on the "Connect to Device" page). The rest of the options can be left in their default state.

Click "Connect". This will cause the hotspot to disconnect, and the Raspberry Pi will attempt to connect to your WiFi network.

Now you can connect back to your regular WiFi, go back to the previous tab and click "Skip to dashboard". This will bring you to the dashboard page from where you can control your game.

Step Six: Mic.. Er, Robot Check

If everything went well, the robot you created should soon show up in the "Sets" category. If it doesn't show up, or the status indicators are red, try rebooting the Raspberry Pi by removing and re-inserting the power supply. Also check that the Game Engine is online; though it starts running automatically, it can take a few minutes to spin up.

Once the robot shows up in the "Sets" category with yellow status indicators, it's time to ensure that everything works.

To check that the camera works, click "Preview" near the bottom of the page. The camera feed should then appear in the preview window.

Next, we need to check that the whole system works. First, make sure that the robot is enabled and the game is online.

If everything shows green, click the "Go to game" button on the left-hand side of the page.

If the watcher stream is working, you should immediately see a video feed on the game page (the system uses two video streams: a low-latency stream for players in the game, and another for people watching the game, called the "watcher stream"). Whether or not there is a video stream visible, click "Play now" to start the game. After a brief period in the queue, you will be moved into the game. You should see the in-game video stream now.

The simple game template we're running does not include any actual gameplay, but it does show that the system is working. To get an idea of how the game loop works, open two tabs or browser windows – one with the game page open and the other with the dashboard open. Then play the game once again, while monitoring the Game Loop section of the dashboard. The text at the bottom of the Game Loop section will show the current state. Notice also the "Restart Game Loop" button: this can be very useful if the game state gets stuck while developing a new game. For more information on the game loop, see the documentation.

Step Seven: The Fun Part

Now that you have the whole system working, it's time to do something with it! You can either choose to use one of our ready-to-use templates, or create a custom game by using our Python SDK. Check our projects-page on hackster.io, our Discord community, and Youtube channel for inspiration.

For more detailed information on our platform, see the documentation.

We are constantly publishing new guides on Youtube and hackster.io. Some of them help you create a complete game from start to finish, while others focus on a certain feature of our platform or Python SDK.

Here's a list of guides to get you started:

• Internet controlled RC car: VIDEO | WRITTEN GUIDE
• Let anyone play your Nintento Switch over the internet: VIDEO
• Internet controlled Sphero RVR: VIDEO
• How to create a custom game: VIDEO | WRITTEN GUIDE
• Let others remotely control a servo: VIDEO | WRITTEN GUIDE