Donkey Car Simulator with Real RC Controller

Background

This is a guide for using the Robo HAT MM1 board / RC controller in the Donkey Car Simulator. This is a request that I have seen from a number of people now.

The question I get asked is How do I use the RC controller in the simulator?. There are a number of reasons you might want to do this:

• Practice - You might want to practice your driving with the RC controller but do not want to go through the trouble of setting up a physical Donkey Car.
• Practicality - You might only own the RC controller and not have access to a PlayStation or Xbox controller.
• Experimentation - You might just want to see if it works. I don't blame you... it is pretty cool to get working.

This guide will be a complete guide so that you can use this guide even if you are not using a Robo HAT MM1 and just the simulator.

Getting Started

We will be using the Donkey Car Docs - Simulator page as a starting point. I use a Windows PC, so these steps may be different if you are using a Mac or Linux machine.

I would recommend that you have a computer that uses a GPU for this project. If you do not have a GPU, it will take longer to compile Neural Networks and your experience won't be as good.

Summary

• Set Up Environment
• Install Donkey Car (dev branch)
• Install Donkey Simulator
• Test Simulator
• Compile Neural Network
• Run NN in simulator

Environment Setup - Donkey Car

This process is the same as the Donkey Car setup. All operating systems have different instructions for setting up the environment. Please follow the relevant for your OS below:

• Linux/Ubuntu (miniconda)
• Windows (miniconda)
• Mac (miniconda)

Please make sure that when you are following the instructions on the other page that you do not change to the master branch. To make sure, after you clone and change into the donkeycar folder, run the following command.

git checkout dev

HINT: If your PC has a NVIDIA GPU, you should install Tensorflow GPU instead of the normal one. The above instructions outline how to do this.

Environment Setup

Part 1 - Download Donkey Simulator

You need to download the latest release of the simulator from below (make sure you get the correct one for your operating system):

https://github.com/tawnkramer/gym-donkeycar/releases

Extract the ZIP and put into a folder. For the purposes of this tutorial I have created a folder ~/Documents/DonkeySim2020. I am using Windows. We will end up with a final directory structure like this:

~/Documents/DonkeySim2020
   |-- DonkeySimWin
       |-- donkey_sim_Data
       |-- gym-donkeycar
       |-- MonoBleedingEdge
       |-- mysim
       - donkey_sim.exe
       - UnityCrashHandler64.exe
       - UnityPlayer.dll

At the moment, we only have the files from the DonkeyCarSim.zip. We need to put the other files in here. We will start by installing the Donkey Gym repository.

Part 2 - Installing Donkey Gym

The Donkey Gym repository contains all the information for the simulator to launch correctly in Python. Without this being installed, you will not be able to launch the simulator.

In the folder above (DonkeySimWin) you need to run the below commands...

git clone https://github.com/tawnkramer/gym-donkeycar 
conda activate donkey 
pip install -e .

Now we have this installed in the Conda environment, we can move onto create a car.

Part 3 - Create Donkey Car

This is the same process as the normal Donkey Car procedure. We will just be putting the simulator in the same folder as the simulator - otherwise it won't be able to find it when we go to launch it.

donkey createcar --path ~/Documents/DonkeySim2020/DonkeySimWin/mysim 
cd ~/Documents/DonkeySim2020/DonkeySimWin/mysim

We are now ready to configure the car for the simulator.

Configuring Simulator

Similar to all Donkey Cars, you need to configure the car for use based on your hardware. In this case, we are using the simulated hardware.

The is a fine in the mysim folder we created in the last step named myconfig.py. We will be changing some settings in this file to make the Simulator work, along with the Robo HAT MM1 HAT as a controller.

Let's first enable the simulator. Find the lines below and un-comment in myconfig.py:

DONKEY_GYM = True 
DONKEY_SIM_PATH = "~/Documents/DonkeySim2020/DonkeySimWin/donkey_sim.exe"
DONKEY_GYM_ENV_NAME = "donkey-generated-track-v0"

You will need to update the DONKEY_SIM_PATH to the correct folder that you saved the executable. See below examples for each operating system.

Windows: DonkeySimWin/donkey_sim.exe Mac OS: DonkeySimMac/donkey_sim.app/Contents/MacOS/sdsim Linux: DonkeySimLinux/donkey_sim.x86_64

Robo HAT MM1 - Enabling for Simulator

The next step is enabling the Robo HAT MM1 Hardware for the RC controller. Please note if you are following this guide and do not plan on using the Robo HAT MM1 or RC controller, then you can skip the next steps.

Find the lines below and un-comment in myconfig.py:

. . .
CONTROLLER_TYPE='MM1'
. . .
HAVE_ROBOHAT = True            
MM1_STEERING_MID = 1500        
MM1_MAX_FORWARD = 2000          # Max throttle to go fowrward. The bigger the faster
MM1_STOPPED_PWM = 1500
MM1_MAX_REVERSE = 1000          # Max throttle to go reverse. The smaller the faster
MM1_SHOW_STEERING_VALUE = False

Make sure to set HAVE_ROBOHAT = True. By default it is False.

Make sure you set CONTROLLER_TYPE='MM1'. By default it is 'ps3'.

The final thing to un-comment in myconfig.py is the MM1_SERIAL_PORT parameter. You will need to change this based on your operating system. You may also have to search for the correct serial port on Windows, Mac and Linux. The default is set for the Raspberry Pi.

MM1_SERIAL_PORT = '/dev/ttyS0'

SERIAL HELPFUL HINT
# -- Default Pi: '/dev/ttyS0'
# -- Jetson Nano: '/dev/ttyTHS1'
# -- Google coral: '/dev/ttymxc0'
# -- Windows: 'COMn' (Check Device Manager for correct port)
# -- MacOS/Linux: please use 'ls /dev/tty.*' to find the correct serial port
# eg.'/dev/tty.usbmodemXXXXXX' and replace the port accordingly

The very last step is to ensure that you have the latest version of code.py on the CIRCUITPY drive of the Robo HAT MM1. Plug your Robo HAT MM1 into your computer and copy the code.py file from the below link:

https://github.com/autorope/donkeycar/blob/dev/donkeycar/contrib/robohat/code.py

One change to this file is also required. Near the top of code.py is a few parameters that can be changed. We want to turn on USB_SERIAL and turn off DEBUG. Otherwise the simulator will get confused. Make sure they are set as below in code.py:

# Customisation these variables
SMOOTHING_INTERVAL_IN_S = 0.025
DEBUG = False
USB_SERIAL = True
ACCEL_RATE = 10

In the below images, you will see the receiver and the HAT. Follow the diagram for how to connect the receiver to the HAT so that the software works correctly with Donkey Car. Make sure you go through all three (3) pictures below. NOTE: you do not need a Raspberry Pi for this.

Default configuration: RCC4 is channel 1, RCC3 is channel 2.

AND OF COURSE - Plug in the USB cable to your computer you are running the simulator on.

You will need a battery or power supply for the Robo HAT MM1 as well. The USB only provides data and does not power the RC Receiver.

Power Options:

• IN - a lipo battery 7 - 15 V.
• BAT - a 1S lipo battery
• External 5V Power or Raspberry Pi 5V Rail

Simulation Time!

We are now ready to launch the simulator. We have done the following to get to this point:

• Installed Donkey Car
• Installed Simulator
• Set up folder structure
• Configured mysim car
• Setup Robo HAT MM1

Now for the fun part! You get to drive the car around as fast as you like without risk of damaging anything. This is achieved using the same commands as if you are using a Real Donkey Car + Raspberry Pi / Jetson Nano.

Driving (with Robo HAT MM1)

python manage.py drive --js

Using a real-world controller in the simulator

Driving (without Robo HAT MM1)

python manage.py drive

Using the web page running at: http://localhost:8887

Training your model

python manage.py train --tub tubs/<tub name> --model models/mypilot.h5

Make sure to include only the tubs you want to create the neural network. More information here.

Run AI Mode

python manage.py drive --model models/mypilot.h5

You will need to start it by going to the web controller (http://localhost:8887) and selecting either 'local angle' or 'local pilot'. More information here.

Conclusion

We have now successfully got the simulator and Robo HAT MM1 working together to achieve a simulated donkey car.

I hope that you have found this guide useful. Let me know in the comments. I am open to suggestions and edits.

What to do next?

• Join the Donkey Car Discord Channel
• Order Robo HAT MM1 from CROWD SUPPLY
• Buy a Real Donkey Car from the Robocarstore

Future Work

There are a number of future improvements that I would like to implement in the simulator. Some of them are being worked on at the moment.

• Real-Time Leader Board: This is something I have seen before at some events. It would be good for the large scale racing events in the simulator if it can do real time laptop reporting of each racer. At the moment the times are manually exported.
• Multi-Racers, Multi-Simulators: It would be cool if you could have a multi-dimensional race; a single race but with multiple instances of the simulator running all reporting times out to the leader-board. The reason for this is so that you can avoid collisions and reduce lag for people with poorer internet connections. It also opens the possibility for events or divisions taking place in separate sessions.
• More Tracks: There a few different track models I would like included (mainly the one that Robotics Masters makes). Having a deep racer track as well would be cool (I think this is already in there).