CangHai • Posted by vany5921

Published April 7, 2020 © MIT

M5StickV-M5Train

Stickv is used to realize the AI identification of traffic signs, and stickc is used as the executive component. Thanks @CangHai

IntermediateFull instructions provided3 hours1,777

Things used in this project

Hardware components

Toy train

M5Stack M5StickV

M5Stack M5Stick

L298N motor driver

Charge Management Board

Software apps and online services

V-training

Thonny IDE

Vscode

Story

Toy train has a long history and is a popular toy for children. Mickey rabbit small train track is compatible with IKEA's track, and children can't get tired of playing. However, after all, the electric small locomotive has only one speed, and the small locomotive drives the carriage forward at a constant speed for a long time, which is inevitably boring. How to enhance interest?

Train your own model through M5StickV and upload it to the server to get the returned training model. In this way, the small train is equipped with a pair of smart eyes that can recognize traffic signs.

Seven traffic signs have been trained in this project. I try to use it in the little track train, and use M5StickC as the executive component to build a funny track train based on visual recognition.

My idea is that such a small locomotive pulls a car and starts from the departure station, passing through seven actions: speed limit, pay attention to the train, release the speed limit, light up, whistle and stop. The small locomotive transforms into an AI small train that can recognize traffic signs. According to the traffic signs, it completes such actions as changing speed, flashing lights, honking various sound effects, starting and stopping. More fun for children's toys.

（1）M5sitckV

Training model

Use the training program of M5StickV to take more than 100 photos of each of the seven traffic signs (700 + photos in total) and send them to http://v-training.m5stack.com/. If your training meets the requirements, the calculation results will converge, so that in about half an hour, you will receive an email with a download link. After downloading, you will get an independent training model library and a boot.py demo code. Don't underestimate this demo, we will easily migrate it to the main program. Note: when training, try to put the model in the viewfinder and try different lighting conditions.

Design code

This part mainly sends the result of recognition to M5StickC, so UART sending programming is the main content. In addition, because m5sitckv has a horn, the train sound effect of our project is realized by M5SitckV, the locomotive light mentioned above is not on, and M5StickV itself has a very bright full-color LED, and the task of lighting is also handed to M5StickV.

（2）M5sitckC

The code of M5StickC is relatively simple. After receiving the recognition character from UART, the action behavior can be sent to L298N through a series of judgment statements.

When designing the code of M5SitckC, I use uiflow as the UI design and framework construction, and then use thorny to debug the generated code. Players may ask two questions here: first, why not continue debugging with UIFlow? After all, uiflow has some limitations, and thorny can get a very comprehensive debugging and error information when debugging ESP32. Second, why does not use Vscode + M5 plug-in when debugging M5StickC? Because the screen of M5StickC is very small, when debugging Vscode + M5 plug-in, no specific error information can be obtained. So according to my experience, when debugging M5StickC, the most suitable way for me is UIFlow + thorny IDE.

On the screen of M5StickC, I designed three parameters, one is the battery voltage (the endurance of C is weak, it is very important to display the battery voltage in real time, otherwise there will be a lot of trouble in debugging), the other is to display the identification feature code obtained from V, and the third is the action behavior of Real C (such as stop / go /...) to facilitate the understanding of the identification when it is in joint debugging with V.

L298N motor drive board commissioning

In the joint commissioning, I used a small skill to weld the input wire end of a test motor with two DuPont wire bus heads, while the output and input copper wire ends of L298N were welded with pins and heat shrinkable. In this way, it is very convenient to connect L298N with battery, motor and M5StickC. In the formal deployment, only the copper conductor needs to be cut to the appropriate length. Such a small skill can greatly shorten the debugging time and increase the reliability of debugging. The purpose of L298N debugging is to test each control behavior and get PWM parameters according to the actual situation of the motor (this parameter will be modified after assembling the small locomotive motor).

import audio
import gc
import image
import lcd
import sensor
import sys
import time
import uos
import os
import KPU as kpu
from fpioa_manager import *
from Maix import I2S, GPIO
from machine import I2C
from board import board_info
from pmu import axp192
pmu = axp192()
pmu.enablePMICSleepMode(True)
fm.register(board_info.SPK_SD, fm.fpioa.GPIO0)
spk_sd=GPIO(GPIO.GPIO0, GPIO.OUT)
spk_sd.value(1)
fm.register(board_info.SPK_DIN,fm.fpioa.I2S0_OUT_D1)
fm.register(board_info.SPK_BCLK,fm.fpioa.I2S0_SCLK)
fm.register(board_info.SPK_LRCLK,fm.fpioa.I2S0_WS)
wav_dev = I2S(I2S.DEVICE_0)
from machine import UART
fm.register(board_info.CONNEXT_B,fm.fpioa.UART1_TX)
fm.register(board_info.CONNEXT_A,fm.fpioa.UART1_RX)
uart_A = UART(UART.UART1, 115200, 8, None, 1, timeout=1000, read_buf_len=4096)
fm.register(board_info.LED_W, fm.fpioa.GPIO3)
led_w = GPIO(GPIO.GPIO3, GPIO.OUT)
led_w.value(1)
passlable=''
global v_state
v_state='no'
lcd.init()
lcd.rotation(2)
try:
        img = image.Image("/sd/startup.jpg")
        lcd.display(img)
except:
        lcd.draw_string(lcd.width()//2-100,lcd.height()//2-4, "Error: Cannot find start.jpg", lcd.WHITE, lcd.RED)
task = kpu.load("/sd/3f758421b4b1db32_mbnet10_quant.kmodel")
labels=["1","2","3","4","5","6","7"]
sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.set_windowing((224, 224))
sensor.run(1)
lcd.clear()
def play_sound(filename):
        try:
                player = audio.Audio(path = filename)
                player.volume(30)
                wav_info = player.play_process(wav_dev)
                wav_dev.channel_config(wav_dev.CHANNEL_1, I2S.TRANSMITTER,resolution = I2S.RESOLUTION_16_BIT, align_mode = I2S.STANDARD_MODE)
                wav_dev.set_sample_rate(wav_info[1])
                spk_sd.value(1)
                while True:
                        ret = player.play()
                        if ret == None:
                                break
                        elif ret==0:
                                break
                player.finish()
                spk_sd.value(0)
        except:
                pass
while(True):
        img = sensor.snapshot()
        fmap = kpu.forward(task, img)
        plist=fmap[:]
        pmax=max(plist)
        max_index=plist.index(pmax)
        a = lcd.display(img)
        if pmax>0.99:
                lcd.draw_string(40, 60, "Accu:%.2f Type:%s"%(pmax, labels[max_index].strip()))
                passlable=str(labels[max_index])
                if passlable==v_state:
                        pass
                else:
                        v_state=passlable
                        uart_A.write(passlable)
                        if v_state=='1':
                                led_w.value(0)
                                time.sleep_ms(2000)
                                led_w.value(1)
                        elif v_state=='2':
                                play_sound("/sd/whistle3.wav")
                                time.sleep_ms(200)
                        elif v_state=='3':
                                play_sound("/sd/train.wav")
                                time.sleep_ms(200)
                        else:
                                pass
        else:
                pass
a = kpu.deinit(task)
uart_A.deinit()

from m5stack import *
from m5ui import *
from uiflow import *
import machine
import time
 
#UI
setScreenColor(0x111111)
title0 = M5Title(title="Train0108", x=3 , fgcolor=0xFFFFFF, bgcolor=0x0000FF)
 
label0 = M5TextBox(31, 45, "Ready", lcd.FONT_Default,0xFFFFFF, rotate=0)  #显示火车状态
label1 = M5TextBox(33, 89, "Inbox", lcd.FONT_Default,0xFFFFFF, rotate=0) #显示接收到的指令情况
circle0 = M5Circle(17, 52, 3, 0xFFFFFF, 0xFFFFFF)
circle1 = M5Circle(17, 97, 3, 0xFFFFFF, 0xFFFFFF)
#lcd setup
axp.setLDO2Volt(2.7)
title0.setTitle(str(axp.getBatVoltage()))
#setup
#===uart
uart = None
uart = machine.UART(1, tx=32, rx=33)  
uart.init(115200, bits=8, parity=None, stop=1)
 
#===GPI0 SETUP
#pin1 = machine.Pin(5, mode=machine.Pin.OUT, pull=machine.Pin.PULL_UP)
pin0 = machine.Pin(0, mode=machine.Pin.OUT, pull=machine.Pin.PULL_UP)
PWM1 = machine.PWM(26, freq=10000, duty=0, timer=0)
PWM1.resume()
wait(1)
 
#===Train Action
def go(): #normal
  pin0.value(0)
  PWM1.duty(60)  #speed=60
  label0.setText('Go')
  label1.setText('no')
 
def light(): #light of train
  #light on m5stickV
  label0.setText('Light')
  label1.setText('1')
  pin0.value(0)
  PWM1.duty(50)  #speed=50
 
def train(): #Train comming 
  # Train sound play on m5stickV
  label0.setText('Train')
  label1.setText('2')
  pin0.value(1)
  PWM1.duty(100)   #IN1=1   IN2=1  停车3S后正常速度，相当于等待火车通过
  time.sleep_ms(3000)
  pin0.value(0)
  PWM1.duty(50)   #3S为speed=50
  
  
def whistle(): #whistle 
  # whistle play on m5stickV
  label0.setText('Whistle')
  label1.setText('3')
  pin0.value(0)
  PWM1.duty(50)  #speed=50
 
def limit(): #train limit 5KM/h
  pin0.value(0)
  PWM1.duty(45)   #speed=45
  label0.setText('Limit')
  label1.setText('4')
 
def nolimit(): #train no limit 5KM/h
  pin0.value(0)
  PWM1.duty(75)   #speed=75
  label0.setText('NoLmt')
  label1.setText('5')
  time.sleep_ms(2000)
  pin0.value(0)
  PWM1.duty(50)   #2S后降速为speed=50
 
def stop():#train stop
  pin0.value(1)
  PWM1.duty(100)   #IN1=1   IN2=1
  label0.setText('Stop')
  label1.setText('6')
  
def greenlight(): #train Go
  pin0.value(0)
  PWM1.duty(50)   # speed=50
  label0.setText('GLight')
  label1.setText('7')
 
 
 #=====Loop
while True:
  title0.setTitle(str(axp.getBatVoltage()))
  if uart.any():
    bin_data = uart.readline(1)
    decode_bin_data=bin_data.decode()
    wait_ms(200)
    if decode_bin_data=='1': #如果识别到1开灯
      light()
      time.sleep_ms(200)
    elif decode_bin_data=='2':#如果识别到2===火车通过
      train()
      time.sleep_ms(200)
    elif decode_bin_data=='3':#如果识别到3===鸣笛
      whistle()
      time.sleep_ms(200)
    elif decode_bin_data=='4':#如果识别到4===限速
      limit()
      time.sleep_ms(200)
    elif decode_bin_data=='5':#如果识别到5===解除限速
      nolimit()
      time.sleep_ms(200)
    elif decode_bin_data=='6':#如果识别到6===停车
      stop()
      time.sleep_ms(200)
    elif decode_bin_data=='7':#如果识别到7===绿灯
      greenlight()
      time.sleep_ms(200)
    else: #其他数据正常前进即可
      go()  
  else:
    pass

Credits

CangHai

Posted by

vany5921

M5StickV-M5Train

Things used in this project

Hardware components

Software apps and online services

Story

（1）M5sitckV

Training model

Design code

（2）M5sitckC

L298N motor drive board commissioning

Code

M5StickV

M5StickC

Credits

Comments

Embed the widget on your own site

M5StickV-M5Train

M5StickV-M5Train

Things used in this project

Hardware components

Software apps and online services

Story

（1）M5sitckV

Training model

Design code

（2）M5sitckC

L298N motor drive board commissioning

Code

M5StickV

M5StickC

Credits

Comments

Related channels and tags