Published February 13, 2021

Machine desirous of oranges

Raspberry PI driven car with onboard Tensorflow used to recognize pictures in real time

IntermediateWork in progress3 hours255

Things used in this project

Hardware components

Raspberry Pi 3 Model B

SparkFun Dual H-Bridge motor drivers L298

Raspberry Pi Camera Module

Software apps and online services

TensorFlow

Raspberry Pi Raspbian

Story

This is a project I created with Raspberry and car chassis to prove to myself whether I can run TensorFlow in real time on Raspberry. Polish description of the project can be found here https://lukaszkuczynski.github.io/Machina-ze-slaboscia. Github repo is here https://github.com/lukaszkuczynski/rpi80b3.

a kid

From the very moment I started playing with micro-controllers I wanted to have some connection with the real world, to get real data I could analyze. Before, when I was studying Big Data solutions the only issue was: where to get data from :) So how about leveraging your own devices and measurements?

Everything started with simple Temperature sensor and Raspberry. Then I realized, Raspberry is too much for such a simple task. But how to get WIFI connection from a micro-controller. ESP is the answer. After getting 8266 and ESP32 time for other sensors has come. One of things bought in the one package was a servo motor, once. But it was to no use. What can I do with it.

A car! Everyone is a kid, some of us just grown too much. But we still have some childish desires. We want to explore and have fun! When I was a kid (oh, so many years ago!) I wanted to have a remote controlled car. So hereby I created one!

building process

I bought some simple chassis for a car. It already had some motors fitting to it in the package. After putting things together I had a solid basis to have a car. Then, I needed to have a driver to manage motors direction and speed. The no-brainer was to get L289N and mount it on the chassis. When it comes to serving digital signals to inputs of the driver I had to use some micro-controller. I could have mounted simple Arduino or ESP to send signals and remotely control it. However, with built-in ML in mind I wanted to play with Python and regular Linux software, so Raspberry was a natural choice.

two ways to have fun

I have put the Raspberry and mounted on the chassis. There are 2 ways I use to play with my machine:

1) remotely controlled car

The first solution I built was to run remote control system. It was easy as some Flask application is easy to write and control. With this Flask app I can send PWM signals straight to GPIOs of my Raspberry. I have a running camera RPi service which gives me real-time view from a driver perspective.

2) AI powered Orange lover

Another way of having fun with my machine is to run Tensorflow service which is mostly based on the example from Tensorflow team. It just detects an object - using model known based on well known COCO dataset. Then, if it is an orange, it just sends a request to go forward for few seconds.

goals

At the very moment of writing this article the project still requires lots of improvements:

not enough sensors, I could have a sensor for distance measuring to make emergency stop feature
plain GO/STOP choice built on the machine, I can not speed up or slow down when driving a car
no adjustment of driving course, when an orange is detected a car just goes forward

what I liked

Tensorflow Lite gives much freedom and lots of possibilities for makers. You can use ML models just onboard of your device. It works as a charm. Like it!

Code

remote control with Flask

import RPi.GPIO as GPIO

app = Flask(__name__)


import RPi.GPIO as GPIO
import time
 

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(12, GPIO.OUT)
GPIO.setup(16, GPIO.OUT)
GPIO.setup(13, GPIO.OUT)
GPIO.setup(26, GPIO.OUT)  

motor1_f = GPIO.PWM(12, 50)
motor1_b = GPIO.PWM(16, 50)

def changeDuty(left_forward, right_forward, left_back, right_back):
    motor1_f.ChangeDutyCycle(left_forward)
    motor2_f.ChangeDutyCycle(right_forward)
    motor1_b.ChangeDutyCycle(left_back)
    motor2_b.ChangeDutyCycle(right_back)

def left(speed=30):
    print("turning left, with speed %d"%speed)
    changeDuty(0,speed,speed,0)

@app.route('/')
def main():
    return render_template("b3.html")

def change_dir(direction):
    if direction == 'left':
        left(25)
    elif direction == 'right':
        right(25)
    elif direction == 'forward':
        forward(40)
    elif direction == 'backward':
        backward(30)
    else:
        print("dont know that direction %s" % direction)
    #time.sleep(3)
    #init()
    

@app.route("/dir/left")
def route_left():
    change_dir('left')
    return redirect(url_for("main"))