[Gamer Assist] Haptic Feedback System for Games Using Esp...

So, in this video we’ll be seeing how to build this haptic feedback device and the ambient light system which reacts to the player's health value inside a PC game! For that we’ll be using the ESP-NOW protocol, about which we learnt in the previous video and also see how to use different RGB LEDs and different libraries to drive them. Finally a little bit of python and open cv to get the data from the game.

Supplies:

These are the list of products which can help you do this project with ease(Affiliate Link)

• Esp12E: https://amzn.to/3h3b37s
• Haptic motor(exact one): https://amzn.to/2J8E06T
• Haptic motor(alternative): https://amzn.to/2J8E06T
• 3.3V regulator (smaller capacity): https://amzn.to/2J8E06T
• 3.3V 800mah (1A) : https://amzn.to/31762F2
• Velcro: https://amzn.to/2J8E06T
• Header Pins: https://amzn.to/2J8E06T
• Angled Header Pins: https://amzn.to/2J8E06T
• Wires: https://amzn.to/2J8E06T
• FTDI: https://amzn.to/2J8E06T
• PCB: https://amzn.to/2J8E06T
• Soldering Gun: https://amzn.to/2J8E06T
• Soldering Lead: https://amzn.to/2J8E06T
• Foam sheets: https://amzn.to/2J8E06T
• screw terminal: https://amzn.to/3qHwVKP
• Resistor : https://amzn.to/2ZCruRu
• Tp4056: https://amzn.to/3vPSCLL
• Boost: https://amzn.to/2OSdHEU
• LED Strip : https://amzn.to/319usO7
• RGB ring : https://amzn.to/3f6Hhkt
• Node MCU : https://amzn.to/3f498Ss
• Breadboard + power supply : https://amzn.to/3lK1HRS
• 7805 : https://amzn.to/2Pp8hRG
• DC jack : https://amzn.to/3d0SCQl

Step 1: Python Gamer Assist Application

This is one of my long-lost least-motivated projects, initially I started in 2018 as soon as I saw this video by Youtuber Michael Reeves "A Robot Shoots Me When I Get Shot in Fortnite". It had everything I loved, from electronics, coding and getting shot by the gun. But I don’t know what happened, it took forever to complete.

Trust me, without this the whole project wouldn’t be possible. Because this is the application that reads the data from the game and sends it to the microcontroller. But I wanted it to be elegant and wanted to extend the functionality of what Michael Reeves had done. And...that’s where my journey of banging my head against the wall started. It was very time intensive to get to this working stage.

Step 2: Working of Gamer Assist

Let me show you the current progress so you know what I'm talking about. To run this you need a configuration csv file, which will contain the location of the health bar in the game and other information for processing the data. So, the first step is to create the configuration file.

Since the program is GUI friendly you can do this for any game, just as an example let me show you how to set up this application for the game apex legend. First take a screenshot of the game when the game is in full screen mode and the health value is clearly visible, and use the shortcut Alt + tab to get to the python application, then click edit config. (Make sure the screenshot is in full screen mode during selection) Draw a rectangle around the health value. Then click grab. In the editor window you can adjust these sliders and check the result until you get the health value on the output section.

Once that’s done. Save the file. Remember this is just a one time process which you might have to do for every game.

Now we can go back to the home page and upload the configuration file.

Just hit the start button and it should read the onscreen data and send it to the hardware through serial communication.

If you want to know how to install and run this python application. You can checkout my Github readme for more details

Step 3: Addressable RGB LED

Sorry if it's boring already, I know most of you are here for electronics so let’s dive into that topic.

The major part of this project is controlling RGB LEDs. There are so many variants and so many ways of using them. But in this video we’ll just focus on addressable RGB LEDs.

Unlike regular LEDs these LED’s can be accessed individually through a microcontroller with just one data pin. If you absorb this Led strip you can see there is an IC placed next to the LEDs. This is what takes the data from the single data pin and drives the RGB LED.

Even these addressable LEDs have many different variants. The Led strip I have drive 5 LEDs together and it’s not possible to address them individually rather they address 5 LEDs at once. Even though the strip looks long, while controlling them through a microcontroller we can consider it as 5 individual LEDs instead of 25 LEDs because the strip contains only 5 driver IC. But the LED ring I have consist of 16 LED's [WS2812] which have integrated driver inside the IC, so we can address all of these 16 LEDs individually.

Step 4: Making the Ambient Light System

To test this out we’ll build a small circuit using NodeMCU, IC 7805, terminal block and dc jack. My led strip works on a 12V supply, so the DC jack will take the 12V and supply it to the led strip through the terminal block, while the 7805 regulator step down this voltage to 5V and sends it to the NodeMCU operation. You can also alternatively build this with esp12E and AMS1117 using the circuit diagram.

For programming these addressable LEDs there are a few libraries that we can use, the popular ones are FastLED and adafurit_Neopixel library. We’ll be using the FastLED library for the led strip

To test them we’ll open the example sketch of the FastLED. Here are 2 important points to remember. First, you need to know what kind of LED drivers the strip is using and the number of LEDs in the strip. In my case it’s GS1903 and I have 5 of them. So I enter the value for the IC and the number of LED’s.

Then just compile and upload the code. You can see the LEDs are working great!

Need to Develop This Project Into a PCB?

Getting a electronics project into production would be nightmare. To ease you into the production world we have developed a platform (PCB CUPID) for PCB enthusiasts and hobbyists to ask and answer questions related to PCB design, fabrication, and assembly.

In addition to the Q&A feature, this website also has a wealth of blog posts and useful resources to help you learn about developing and manufacturing printed circuit boards. Whether you're a beginner looking for a crash course on PCB basics, or an experienced designer looking for tips and tricks, you'll find something of value on the site

So head on over and check it out, and don't forget to participate in the Q&A community to get help and share your own knowledge. Thanks!

Step 5: Making Ambient Light System Wireless

Let’s make this wireless now, so that it’s not necessary for these led strips to be connected to the PC. This is where we use the esp-now protocol to send the data wirelessly.

For the code, go to my github repo and download the project “gamer assist”. Then open the folder arduino code and upload the sketch find_MAC_address. This will give the mac address that we need for our espnow protocol. After uploading the code, open the serial monitor and note down the mac address of the NodeMCU which we’ll be using in the later part of this video. Once you have the mac address you can upload the health_value_ambient sketch, which will control the color of the light depending on the health value. But before uploading make sure you change the name of driver IC and the number of LED’s according to the led strip you have. In my case it is GS1903 and the LED count is 5.

Step 6: Testing Ambient Light System

To test if it’s working we just need another NodeMCU which can receive the serial data from the python application and transfer the data through esp now protocol to our ambient light setup. Open the file Serial to esp_now.ino, In this file enter the mac address of your Nodemcu which we got earlier, then just compile and upload the code. Now we can run the python application. When you play the game the led strip should change its colour from green to red depending on the players health value inside the game.

Let me also show you a another small demo for better understanding.

You can see the Image I have set up the python application to read the data from the notepad and send it through serial communication. If you observe the terminal you can see the data is processed sent to the NodeMCU via serial communication.

Step 7: Building the Haptic Feedback System

Building this was quite simple, First solder the RGB led ring then take appropriate size of the PCB and fit the ring with it. Then follow the circuit diagram and solder the esp12E, the vibrating motor and the driver for haptic on the front. Then solder a small boost converter and tp4056 charging circuit on the back of the PCB. Finally a 3.3v regulator. Initially I soldered the AMS1117 voltage regulator, but that wasn’t working so well because it had a very steep voltage drop when the LEDs and motors were ON, so instead I used a tps73033 regulator because it worked with a very low voltage drop and supplied enough current to run the esp12E. This package of the IC was quite messy to solder because of the size, but at the end of the day this got the job done.

Initially for battery I used a 500mah LIPO battery but it couldn’t power everything at once because it could discharge maximum of only 1 amp so I replaced 500mah battery with a 2C 800mah battery, so that it can run longer and supply enough power to the RGB ring as well as the motors.

Step 8: Programming the Haptic Feedback System

Now, we can upload the code to the esp12E, i’m just going to connect to esp12E through an FTDI module as we used to do in our previous videos. Once the connection is done, upload the find mac address code.Then remove the ground wire from GPIO-0 and restart the esp12E. Now this should output the mac_address on the serial monitor. Once you have the mac_address we can go ahead and upload the ring code from my Github repo. To have a finishing touch, I will be using this velcro so we can wear this while playing the game.

Open the code serial to esp now and change the mac address to your haptic feedback device address and then upload the code to the NodeMCU to convert the serial data to esp now. And that’s it! You're done!

Step 9: Using the Haptic Feedback and Ambient System

Open the config file of the game in the python application and then just hit start. (As described in step - 2)

You can even try games which have a health bar instead of health value, by just selecting the option in the editor window. Just make sure only a single white line is visible when the image is processed, the white line indicated the separation between the health lost and health remaining, then python application use this line information to determine the remaining health.

Step 10: Final Thoughts!

The python application isn't perfect yet. It has some latency for the lights to change colour as well as some bugs needs to be fixed within the application. If you like you can fix the issues or paly with the code to see how it's working (Github).

This brings us to the end of the project, if you would like to read similar tutorial, make sure to follow my profile as well as subscribe to my youtube channel for the videos! If you have any questions or suggestions I would love to hear from you, just drop them in the comment section.