Device to help colorblind person detecting traffic light

One day I was watching a talk show, one of the attendants is colorblind and couldn’t tell the difference between color green and red. Even though he was making fun of himself saying that one of the biggest challenges and problems that he meet on his daily life, is that it’s difficult for him to cross the street alone at the beginning, because he couldn’t read the traffic light correctly. So when he is alone and needs to cross the street, there is nothing he can do, but to follow other people. And sometimes, people would misunderstand him as a stalker.

It really hit me, sometimes the things that you can easily managed, might be difficult tasks to others, so we really need to appreciate the life and the things we have now. And I am also thinking that it would be great if there is a portable device with a camera, which is easy to carry around, and when people uses the camera to take a photo of the traffic light, the device will tell what light it is.

And since I am also quite interested in AI and machine learning, and build a device all by myself, is a good way to experience the beauty of the technology and also help me better understand the whole process. But as a beginner, it's also quite challenging. And thanks to Seeedstudio by launching the good hardware and providing the detailed guide which is easier for a beginner to follow, and also thanks to Roboflow by offering all the public datasets, so that I don't need to create my own dataset.

So, let's do it!

# Step 1: install sensecraft on wio terminal to show the data

# Step 1: install sensecraft on wio terminal to show the data

Firstly, download the latest version of Sensecraft to local disk, and then connect the wio terminal to PC and turn it on, then quickly double slide the power switch, then will see a ne external drive, called Arduino, drag the downloaded file to the Arduino drive, and flash it into Wio terminal.

Once it's done, the wio terminal will runs the Sensecraft program. And the data collected via the built-in sensor on wio terminal will directly show on the device.

But since the wio terminal only has built-in accelerometer, light sensor and sound sensor, you would need a extra camera to detect the traffic light, then the grove ai vision sensor comes in, and needs to train it firstly.

# Step 2: Train the AI model with a public dataset.

# Step 2: Train the AI model with a public dataset.

The first step of an object detection is to have a dataset, either download public datasets or create your own dataset. To save the time and improve the efficiency, I decided to search on roboflow to see if there is any suitable dataset and found below dataset which created by Wawan Pradana which included 999 images.

https://universe.roboflow.com/wawan-pradana/cinta_v2

Download the version with YOLO V5 format.

Then start the training on Google Colab.

All the training are done on the prepared Google Colab workspace. directly go through the steps mentioned in the workspace and run the code one by one. Firstly set up an enviroment for training, download a dataset, perform the training and then download the trained model.

PS: since different dataset is used, so the code on step 4 would needed to be updated by the code that we have already obtained when we downloaded the dataset on roboflow as below.

# Step 3, depoly the trained model into Grove Vision AI module, and perfom inference.

Firstly connected the grove vision ai module into the pc via type c cable, then double chick the boot button on the ai module, then a new storage drive called Grove ai would pop up, then directly drag the model-1.uf2 to the drive.

Once it's done, connect the ai module together with wio terminal via Grove cable. Then you would be able to see the data on wio terminal directly, the data shows on the line of "Vision", are the data collected by Grove ai vision sensor, and the first number means what light it is now, “0” means green light, and number “1” means red or yellow light, and the second number comes together means the confidence.

For example, below picture means 78% percentage is red or yellow light and suggest to wait a while.

PS: You can also see the video and picture on a preview window of the camera stream.

After connected the ai module on the pc, then click the connect button, and pair with the device called "Grove AI". Once it's successfully paired and connected, then would be able to see the real time inference results on the window as below:

Even though the results are not accurate enough, and the device is not that convenient, but still, it's my first project, trying to use my power to make something to solve a problem or at least catch some attentions from public to notice this thing. It's a good start~