Internet Controlled RC Car with HD Video Using Raspberry Pi

In this guide, we will be extending the range of our RC Car to be controlled over the internet. To see what's happening, we'll be connecting an HD camera to the setup, allowing us to glance at the surroundings in near-real-time with a crisp image. The best part is that there's no coding required, as everything should work out of the box using the Surrogate.tv platform.

Additionally, you can let anyone else in the world take control of your car! Or, if that sounds too dangerous, you may choose to keep your game private. For a more amibitious project, add gameplay objectives to your game logic using our Python SDK and create your own real-life game for others to play over the internet! In order to complete this project, you must first create a game on Surrogate.tv by following our getting started tutorial (which takes less than an hour!)

The car we're using to showcase this process is a model from Traxxas, but all the steps we go through could probably be applied to any standard RC car.

Before you begin, make sure that the battery of your RC car and your power bank have enough juice to at least do some test runs once you're all set.

Step One: Gathering the Hardware

First, make sure you have all the required hardware.

Obviously, you need an RC car. We're using a model from Traxxas.

A Raspberry Pi with the surrogate.tv software installed (see the guide if you haven't already installed it) is used to stream video and control the car. A camera is also needed so you can see where you're going! Most USB cameras will work, as will the Raspberry Pi camera module. Even a GoPro can be connected via a USB HDMI capture card! (micro-HDMI to HDMI cable required)

To power the Raspberry Pi, you’ll need a power bank. Theoretically you could get power from the wall, but then you couldn't drive very far. (Pro tip: Choose the powerbank that best matches your project requirements. Use a small powerbank if space is at a premium, or just to lower the weight of the system... or a large powerbank if you wish to make longer journeys.)

Finally, you need a breadboard, one male-to-male cable, three male-to-female cables and two 3-pin male-to-male connectors. We recommend soldering your own circuit board to replace the breadboard as soon as possible! A breadboard will do fine to get you started, but a proper circuit board will be more reliable.

Step Two: Preparing the RC Car

Remove the body shell covering the car and then disconnect the radio receiver. In our Traxxas car the receiver is housed inside a waterproof container, which must be opened first to gain access to the receiver.

After removing the cover of the receiver (if there is one), disconnect the cables going from the receiver to the ESC (electronic speed control, whichmakes the car go forwards/backwards) and steering servo.

If the receiver has more cables connected than those going to the ESC and steering servo, it may be better to leave the other cables connected to the receiver. You can either leave the receiver in its place or remove it completely to save some weight and make space for a future addition to your internet-connected car.

Step Three: Connecting the Wires

With the insides of the car available to us, it's time to wire up the ESC and steering servo to the Raspberry Pi, which will be controlling the car. (The black wire is ground, red is power, and white is the control wire which the Raspberry Pi will use to control the car.)

Prepare the board by adding the two 3-pin connectors.

Connect the ESC header so that the black wire is adjacent to the other 3-pin connector.

Connect the servo header, again placing the black wire next to the other black wire.

Connect the two red wires together by placing a jumper cable between them.

Add the wires that will go into the Raspberry Pi. Again the black wire must be on the same row as the other black wires, and the white wires must be on the same rows as the other white wires.

Finally, connect the wires to the Raspberry Pi GPIO pins. (The ESC/servo ground is connected to the Raspberry Pi ground, and the two control wires are connected to GPIO pins 12 and 16.)

Now that the wiring is complete, place the components inside the RC car. For this first prototype the components can just be crammed wherever they fit.

Step Four: Setting Up the Game Logic

Now it's time to power up the Raspberry Pi. Connect the power bank to the power port, and check that the red LED on the device turns on.

Because we're using a ready-made template for this game, the set-up is very simple. Log in to your surrogate.tv account and either create a new game according to the instructions in our guide, or navigate to a previously created game by clicking your profile picture in the upper right corner.

Once you're on the dashboard page of your game, check that your Raspberry Pi has connected to the platform. If the indicator icons show green or yellow, click the link below "Controller" to go to the Device Setup page. On the token page, click "Continue". Now, change "Game type" to "RC Car" and click "Continue". The device is now using the RC Car game template, and the game is almost ready to be played – yay! Go back to the dashboard page of your game. Now click the "Restart game loop" button to ensure that the game gets reloaded with the new template.

The final step we must do is to set the key binds for the game (this step will be removed in a later update.) On the dashboard, click "Settings" in the bottom left corner, and then click "In-game configuration" in the upper menu. Scroll down to "Control settings". Click "MIN" for the control named "motor", and then the key you wish to use for breaking/reversing the car. Set the MIN and MAX controls for at least the "motor" and "steering" controls in the same manner. Here are our recommended key bindings:

Motor: MIN - S, MAX - W

Steering: MIN - A, MAX - D

The rest can be left empty for now. Binding the throttle_shifting and steering_shifting controls allows you to change the degree the throttle and steering change the current speed and direction. Note that you can remove a key binding by choosing a control and clicking the escape button on your keyboard.

Step Five: Testing

Warning: it's best to hold the RC car in the air for the first tests, or at least set it somewhere where it can't destroy anything expensive or fall down and break itself.This advice may or may not come from first-hand experience.

Now that you followed the instructions to the letter, everything should work flawlessly right away! Unless they don't work, which is fine and quite possible.

First, turn on the RC car. On our Traxxas car the power button is found on the ESC. Press the "Ez Set" button. The LED next to it should turn on. When the LED is showing a constant green, the connection with the Raspberry Pi is working.

If the LED is flashing green, it means that the ESC is not recognizing the Raspberry Pi controls. It may take a short while for the connection to work. If the flashing continues, first make sure that the wires are properly and correctly connected. If that doesn't help, you may have to calibrate the ESC. Check the manufacturer's instructions for how to do this calibration. You will most likely need to use the throttle and steering controls in the calibration process; to see how to control the car, check the instructions below.

With the RC car turned on, we can straight away see whether the system is working by going back to the dashboard and clicking "Preview". You should see the video feed from your camera and be able to control the RC car with the keys you set previously. It's possible that the power and steering controls are mixed up. In that case, just switch the locations of the non-ground wires on the Raspberry Pi.

Step Six: Terrorizing the Neighbourhood

Just kidding – but please use your newfound powers responsibly! Have fun and come share your project with our Discord community!

Take a look at our Robot Explorer missions to see what's possible with an internet-connected RC car:

The first prototype you've made is probably not that sturdy. Improving the outdoor survivability of your internet-connected car might be a wise next step for the project. You could also add mobile connectivity with a USB 4G modem, which would allow you to operate the car outside the confines of your WiFi network. For extra fun, strap a mobile phone to the car so you can ask your neighbour to borrow something without leaving the house. :D

You can also use the surrogate.tv platform to create games where other people can control your RC car. Stay tuned for more tutorials on how to add gameplay mechanics to surrogate.tv games! Or, if you're technically inclined, jump straight in and create a custom game using our Python SDK. See also the documentation for more information on our system.

For our Robot Explorer series, we used a 4G USB modem and a GoPro attached to an HDMI capture card. We also powered the GoPro via the power bank because the battery of the GoPro didn't last long enough for our use case. All the components are housed inside a weatherproof plastic case. Zip ties were used to secure things in place. Here's what our set-up for the Robot Explorer series looks like:

In the picture below, you can see the components better. The white thing is the 4G modem, the brownish thing is the HDMI capture card, and the green PCB is a soldered version of the circuit pictured in the image in step three.

And here's an image of our GoPro mounting system.

For more project ideas, see our projects-page on hackster.io, our Discord community and our YouTube channel for inspiration.