Use Teachable Machine AI to Control Anything

Note: I've cleaned up the code and upgraded libraries, also created a local version that you can run on your computer. Check out the Github repository.

Teachable Machine is a web-based AI tool developed by Google, which enables you to train and use AI models without any AI knowledge. An AI tool for dummies. Neat, right?

But can it do more than simply demonstrating machine learning in the web browser? Can it be a practical tool too?

Interestingly, there's a demo on the Teachable Machine website called Tiny Sorter which use p5.js (an online JavaScript editor/compiler) to talk to an Arduino Leonardo with a servo. So it is entirely possible to run AI on your computer and do something useful.

This tutorial is about how to make Teachable Machine AI output its recognition results to an external device, such as a microcontroller. My method is not the same as the Tiny Sorter demo, however the principle is similar. This would be especially useful for people who has little knowledge about AI, but interested to use it for their Maker or physical computing projects.

(Note: the following steps only tested on Windows 10.)

Pros of this method:

• No AI knowledge required
• Fast, easy and free model training/hosting
• Almost no need to install anything
• Pretty low cost if you already have a computer
• Can be used on any microcontrollers as long as you know how to program them to read serial inputs and do stuff

Cons of this method:

• Recognition has to be done on your computer and thus reduce the range/mobility of your microcontroller
• Computer needs Internet connection
• The AI models have limitations, it may get false readings and cannot be further customized
• May be more difficult to debug on the microcontroller's end
• Some AI experts might tell you this is cute but not the real thing. Yeah, ok...

Step 1: Generate and upload your AI model

When you train your model, I would use Class 0 as the background image (nothing), then the things to be recognized would be named as "1", "2", "3"...etc. This will be easier for the devices to read results.

For image recognition, you can move your object around to improve AI accuracy, and there should be enough difference between each objects.

Or if you are training the sound recognition model, make sure to input enough background noise samples first. Same as above, I would name the sounds to recognized as "1", "2", "3"...

After the model finished training, click Export Model than select Tensorflow.js -> Upload. Copy the model link after it is done. (For now sound recognition can only choose Tensorflow.js.)

Step 2: Prepare the p5.js script

The p5.js scripts template can be found here:

• Image recognition: https://editor.p5js.org/krantas/sketches/IKUf43rB
• Sound recognition: https://editor.p5js.org/krantas/sketches/3wZ9hAwG

Or you can find the complete code here. I haven't done the gesture recognition version, but it should be easy to modify.

This script is basically the combination of Teachable Machine's and p5.js's example code. There are 4 files total:

• sketch.js (main code)
• index.html (import libraries)
• style.css
• p5.serialport.js (serial port functions)

You can login p5.js and duplicate/save your script. You'll only need to change two places:

• microcontroller's serial port on your computer (see step 4)
• the link of your model (step 1)

Note: later I'll see if it's possible to update both code to the new p5.js 1.0.

Remember to change the COM__ to your serial port!

Step 3: Program your microcontroller

Your device (in this case, a BBC micro:bit) will need its own program to be able to read serial inputs and do something about it.

The serial outputs are strings or characters and have no new line characters (like \r\n).

The following example is to indicate received image recognition results by lighting up different LEDs on my BBC micro:bit:

• Result 1 (a BPI:bit ESP32 board) - red LED (Pin 0)
• Result 2 (a Arduino Nano 33 IoT with shield) - green LED (Pin 1)
• Result 3 (a Adafruit Metro M4 Express) - blue LED (Pin 2)
• Result 0 (background only) - white LED (Pin 8)

The serial output baud rate is 9600. Since micro:bit's default rate is 115200, I had to reset it in the beginning.

let result = ""
serial.redirect(
    SerialPin.USB_TX,
    SerialPin.USB_RX,
    BaudRate.BaudRate9600
)

basic.forever(function () {
    result = serial.readString()
    if (result == "0") {
        pins.digitalWritePin(DigitalPin.P0, 0)
        pins.digitalWritePin(DigitalPin.P1, 0)
        pins.digitalWritePin(DigitalPin.P2, 0)
        pins.digitalWritePin(DigitalPin.P8, 1)
    } else if (result == "1") {
        pins.digitalWritePin(DigitalPin.P0, 1)
        pins.digitalWritePin(DigitalPin.P1, 0)
    pins.digitalWritePin(DigitalPin.P2, 0)
        pins.digitalWritePin(DigitalPin.P8, 0)
    } else if (result == "2") {
        pins.digitalWritePin(DigitalPin.P0, 0)
        pins.digitalWritePin(DigitalPin.P1, 1)
        pins.digitalWritePin(DigitalPin.P2, 0)
        pins.digitalWritePin(DigitalPin.P8, 0)
    } else if (result == "3") {
        pins.digitalWritePin(DigitalPin.P0, 0)
        pins.digitalWritePin(DigitalPin.P1, 0)
        pins.digitalWritePin(DigitalPin.P2, 1)
        pins.digitalWritePin(DigitalPin.P8, 0)
    }
})

Step 4: Connect your device's serial port

Download and unzip p5.js serial control program. Connect your device to USB and run the p5.serialcontrol.exe (for Windows). You may have to set it allowed to run in your antivirus program.

Once the program launched, open the serial port of your device. If you are not sure, open Device Manager to see which COM port it has.

Remember to close the COM port in p5.serialcontrol whenever you need to flash code onto your board! Otherwise it will be blocked.

After the serial port is connected and set in the p5.js sketch, everything would be good to go.

Step 5: Employ the power of AI!

Go back to the p5.js script editor and click Run. After a short while a webcam window should appear beside the script. You may see your device's serial port light blinking, indicating the data it received.

If you forget to open the serial port, nothing wrong will happen. The p5.js script would simply run as usual.

In the demo below I can indeed send image recognition results to my micro:bit:

Another demo here is using an Arduino Nano and a mini servo to indicate the sound recognition results: (The background noise is not a result. A result would only be returned when the model recognized something.)

• Result 1 (finger snapping) - turn to 0 degrees
• Result 2 (clapping) - turn to 180 degrees

#include <Servo.h>

Servo myservo;
char result;

void setup() {
    Serial.begin(9600);
    myservo.attach(9);
    myservo.write(90);
}

void loop() {
    while (Serial.available() > 0) {
        result = Serial.read();
        switch (result) {
            case '1':
                myservo.write(0);
                break;
            case '2':
                myservo.write(180);
                break;
        }
    }
    delay(100);
}

I added a small delay at the end, for the servo needs some time to finish its turn.

Below is the Arduino code for controlling two LEDs based on 3 labels.

char result = '0';

void setup() {
  Serial.begin(9600);
  pinMode(7, OUTPUT);
  pinMode(8, OUTPUT);
}

void loop() {
  if (Serial.available() > 0) {
    result = Serial.read();
  }
  switch (result) {
    case '1':
      digitalWrite(7, HIGH);
      digitalWrite(8, LOW);
      break;
    case '2':
      digitalWrite(7, LOW);
      digitalWrite(8, HIGH);
      break;
    default:
      digitalWrite(7, LOW);
      digitalWrite(8, LOW);
  }
}

Remote control possibilities?

It is possible to control something not directly connected to your computer. One possible way is to use HC-05 Bluetooth module on the microcontroller as a wireless serial port.

If you are using micro:bit, you can use one to simply resend results to others via the "radio" function.