Magic Fingers: Cheat code Macro using ML

Tired of typing cheat codes in an offline game, let's type it using gestures. Here, we will be using Arduino Nano BLE Sense + Tiny Motion Trainer and a small python script to get it done.

Hardware Components

We have got the Arduino BLE Nano from SparkFun Electronics.

Setting up TF4Micro Motion Kit

• Install the board from Tools > Board > Boards Manager and search for Arduino Mbed OS Nano Boards.
• To Install Arduino libraries, Navigate to Tools > Manage Libraries
• Search for and install:
• Arduino_LSM9DS1

• ArduinoBLE

• Arduino_TensorFlowLite

• Open the sketch and flash

Download the latest release hereOpen the arduino/tf4micro-motion-kit and double click on <tf4micro-motion-kit.ino> file. Click the Right arrow in the top left corner to build and upload the sketch.

Training the Data

• Visit Tiny Motion Trainer and click on Connect Device. Select the TF4Micro-Motion Kit to pair with the device.

In the Choose your settings tab set the parameters such as capture threshold, no of samples, and the delay between captures.

Add the labels - circle, double tab, square, triangle and click on Start recording to collect the data. We will be collecting 25 samples for each labels in this case.

Set the Epochs to 100 and click on the Start training button to train the model. The Accuracy and Loss metrics for the Training and validation is plotted.

You can test the trained model and once you are satisfied, click on the Download your model.

Now we had got the Arduino sketch and the model generated on the Tiny Motion Trainer.

Application

• Connect the Arduino Nano BLE Sense to the PC

• You can find the Serial port on the Device manager

• The Gesture label is printed on the Serial monitor. We use the python script to extract the gesture and simulate the keystrokes.

Install the keyboard package using

pip3 install keyboard

• The python script extracts the gesture labels from the Serial data and maps the cheat code to the variable, and types the respective keyword.

Here, I have used the GTA Vice city cheatcodes.

import serial
import time
import keyboard
s = serial.Serial('COM128')  #Choose the COM Port
while True:
res = s.read()
print(res)
if res == "circle": #action
keyboard.write('aspirin') #Type the Cheatcode
if res == "double tap": #action
keyboard.write('comeflywithme') #Type the Cheatcode
if res == "square": #action
keyboard.write('leavemealone') #Type the Cheatcode
if res == "triangle": #action
keyboard.write('thugtools') #Type the Cheatcode

The End

Run the python code and connect the Arduino Nano BLE to the PC. Wear the Arduino on your fingers using velcro.

<Video will be added soon>

If you faced any issues in building this project, feel free to ask me. Please do suggest new projects that you want me to do next.

Give a thumbs up if it really helped you and do follow my channel for interesting projects. :)

Share this video if you like.

Blog - https://rahulthelonelyprogrammer.blogspot.com/

Github - https://github.com/Rahul24-06/

Happy to have you subscribed: https://www.youtube.com/c/rahulkhanna24june?sub_confirmation=1

Thanks for reading!