Some Clever Coding Turns an ESP32 LED Matrix Into a Low-Res Copy of Your Computer's Desktop
Capturing, downscaling, and mapping a desktop's display to a low-res LED matrix, this clever software transmits via UDP.
Pseudonymous maker "fabe1999" has written a program, which turns a low-resolution LED matrix build around an Espressif ESP32 microcontroller into a mirror of his computer's desktop — transmitted via Wi-Fi.
"I made a program to screencast my PC to the ESP[32] which is controlling my self-made LED matrix," fabe1999 writes of the work. "The screen capture essentially takes a screenshot of the whole desktop and uses the coordinates of the window to reframe the image so it matches the transparent part of the application. This screenshot is than downscaled to a picture with 14×16 pixels. From the rescaled [image] you can simply request the RGB value of each pixel and send it to the matrix."
The project builds on a larger effort to create an application for designing and playing back custom animations, sent to the ESP32-based matrix over a Wi-Fi link. There's a catch, however: keeping up with a rapidly-refreshing video is no easy task.
"Every UDP [packet] contains the RGB values of a line (16 pixels)," fabe199 explains. "The ESP receives [the packet] and decodes the string to the matching LEDs on the matrix and display[s] the received color values. I'm currently sending 16 UDP packages for every Frame and a new frame is created every 100ms.
"But the ESP is too slow and only shows about 85% of the UDP [packets] sent by the program. I tried to add latency to increase the accuracy but that resulted in a very sluggish animation. And because every line of pixels get refreshed every 100ms if a [packet] got ignored the next one will fix it. (That's why there are sometimes lines that are a frame behind.)"
More details are available on fabe1999's Reddit thread, though not the C# source code for the desktop-streaming software. "It's probably better to not look at the code," the maker admits, "it's pretty janky."