EDES 301 Touchsreen Sketch Identifier

Video: https://drive.google.com/file/d/1SlGvoLCf3eDAtosOgBkHCp4hNyj7oM5w/view?usp=sharing

Exigence

Nobody understands my artistic prowess. They say my drawings are dumb, funny, and bad. How reductive and ridiculous. I'm the greatest artist of this decade. Proof? I've displayed my finest work, crafted in the span of two whole minutes below.

I got tired of my friends making fun of my clear talent in artistry, so I have taken it upon myself to show them that even a computer can interpret what I am drawing. Unfortunately I failed to account for the possible struggle figuring out how to make said computer do its own figuring out.

Utilization

Upon the device starting up (if systemd set up correctly) or running main.py, a canvas with a "Clear" button and a "Guess" button appears. Tapping the screen outside the range of these two buttons will allow for drawing on the canvas. Due to hardware limitations, the process of drawing is very slow, but you should be able get a shape if you are patient and careful. As of right now, the Guess feature can only distinguish between a "triangle" and a beyond perfect, insanely accurate "circle, " again due to hardware limitations (and time).

Wiring

PocketBeagle Voltages:

3.3V (P1_14) ---> + rail

GND (P1_16) ---> - rail

STMPE610 to PocketBeagle via SPI:

SDA/CS ---> P2_31

SCL ---> P2_29

SDO/A0 ---> P2_27

SOI --> P2_25

MODE ---> + rail

Vin ---> + rail

GND --> - rail

ILI9341 to the PocketBeagle via SPI:

Make sure to bridge IM1/IM2/IM3 by soldering

GND ---> - rail

VIN ---> + rail

CLK ---> CLK (P1_08)

MISO ---> MISO (P1_10)

MOSI ---> MOSI (P1_12)

CS ---> CS (P1_06)

D/C ---> D/C (P1_04)

RST ---> RST (P1_02)

STMPE610 and ILI9341:

Connect +X, -X, +Y, -Y ports on SPI side of ILI9341 to the corresponding ports with the same label on the STMPE610.

Program Setup and Configuration

For this information, check out the GitHub link below!

Future Improvements

Long story short, I want to improve the hardware, which would allow me to actually test and improve drawing models, resulting in the ability to expand the model to predict more objects.

As I could not get numpy installed, rendering my previously trained ML model useless, I had to figure out a way to guess between two simple shapes without using a model or even any numpy operations. This proved more difficult than expected. Machine learning is heavily limited by the hardware, and between the lack of functional libraries and the very slow refresh rate of the drawing canvas with Pillow, my expectation of proper predictions was unreasonable.