Punchy the MECH & the Autonomous Fight Club

Introducing Punchy and the Autonomous Fight Club!

Think BattleBots/RobotWars, but with a twist - there's no human in control!

I've been working in the robotics industry for over 20 years, and it's been amazing to see how far we've come. As a kid, I loved watching Transformers, Voltron, GoBots, and reading manga, dreaming that these fantastic pieces of technology would one day become part of everyday life. And now, with the accessibility of technology, we can build autonomous robots for less than $50 at home.

But let's talk about Punchy. Late last year, I had a dream where I saw two small robots fighting with big boxing gloves, like brothers tend to do. They were on wheels, rolling around, swinging and yelling at each other. I woke up laughing and thought, "This needs to exist!" Five days later, on New Year's Eve, the first Autonomous Fight Club happened, where I made two autonomous robots (called MECHs) that battled it out for a group of friends. Since then, many people have joined in and made their own MECHs, tweaking, modding and implementing new code every single day.

Now, you may be wondering if autonomous robots are safe and why we'd allow people, including kids, to make them. But I believe that the more people understand how technology works, the better we can make technology that helps the world. The next phase of technology - artificial intelligence - is ramping up and moving to the physical world, beyond just making pretty pictures and writing poems. The big question we all have now is the ethics for AI and its use. And I believe that we all get to be part of that decision process, not just big companies, and definitely not just one person like Elon. By making technology accessible, fun, and educational, we can all be part of the greater conversation about where this world goes.

So let's begin!

Overview

The original thought around the design was autonomous robots (MECHs) that could find each other using computer vision, move about and fight with no human interaction. Think Battle Bots or Robot Wars minus the Human remote control.

Primary Goals

• Cheap to build (under $50)
• Easy to assemble
• Mobile/Wireless platform
• Vision Guided with Motion Control
• Common components/off the shelf
• Ability to modify/personalise
• Agnostic to ML Model
• Educational (Machine Learning, Robotics, Design, Building, etc)
• Fun!

Stretch Goals

• Easily upgradeable software
• Easily upgradeable hardware
• Scalable in design/functional purpose

Supplies

I wanted to make this project accessible to everyone, so I've put together a list of the source files, tools, and components you'll need to build your own MECH. If you don't have access to a 3D printer or basic tools, no worries! I've included links to places where you can purchase them. And if you're new to coding, don't worry either! I've provided links to the source code to make it easy for you to get started.

This project is always evolving, and if you want to stay up-to-date with the latest developments, I recommend keeping an eye on the Git repo here

MECH

3D Print Files

Microcontroller Code

Battle Server

Server Code

Computer Vision Pre-Trained Weights

Amazon List with all components

Here is a link to an Amazon list with all the components you will need for building 2 MECHs

Components you will need are:

• 3x Micro Servos (continuous/360)
• 1x ESP32-CAM module (AI-Thinker)
• 1x USB Power bank (+2A output)
• 1x Ultrasonic sensor (HC-SR04)
• 8x Hook Jumper Wires (Female/Female 100mm)
• 1x 2x8 Header
• 1x 1x8 Header
• 2x Magnets (8mm Diameter x 3mm )
• 4x M3x12 Socket Head Cap Screws
• 4x M3x8 Socket Head Cap Screws
• 4x M3 Nuts
• 8x M2x8 Self Tapping Screws
• 2x M2x8 Pan Head Self Tapping Screws
• Rubber Band (65mmx6mm)
• FTDI programmer (5V)
• USB Cable
• Assorted Heat shrink
• Velcro/Cable Ties

Tools you will need;

• 3D Printer (or access to one)
• Soldering Iron & Solder
• Pen Knife
• Allen Keys
• Screwdriver
• Wire Cutters
• Wire Strippers
• Drill and Drill Bits
• Computer (Windows, Linux, Mac)
• Internet access

Troubleshooting Tips & Suggestions

Micro Servos - Make sure the servos are 360 degree (continuous) as the design uses them to spin the wheels.

USB Power bank - Make sure to select a power bank with adequate current output. You can check this by looking on the pack to confirm. I suggest >2A @ 5V

FTDI Programmer - Make sure to use a 5V variant of the FTDI Programmer, not the 3.3V.

MECH System Architecture

Architecture

Making all this work required some thought about the system architecture and how everything would come together.

• MECH should be equipped with sensors and effectors
• Control code on the MECH should be highly configurable via remote wireless connection
• Remote computer can provide the computation capability for the ML inference and training
• Simple interface to allow multiple MECHs to be controlled remotely
• A way for a MECH to lose the fight

Taking these basic requirements, I decided to flesh out the system in more detail and into the relevant processes.

Computation locally on the MECH was to be kept to a minimum otherwise a more expensive processor would be required and also a larger power source. This would have resulted in a heavier, more complex and expensive MECH. By having a MECH connect to a local wireless network and for a python script running on a computer to receive, process/inference and send commands back to the MECH would remove the hard work from the MECH and allow it to be cheaper and easier to build.

The system configuration that was decided;

MECH

• AI Thinker (ESP32-CAM) as a microcontroller, wireless communication platform, I/O Interface and Camera source
• Micro Servos 360/Continuous variants to be used for motion control/actions
• Ultrasonic sensor used for range finding/distance
• USB Powerbank as a power source
• Two Wheel Drive
• C as the programming language (Arduino)
• Removable fist/glove that can be knocked off - how you win/lose

Battle Server

• Python as the coding language
• Use of well documented Python libraries
• YOLO used for Computer Vision
• A visual interface to see and understand what the MECH see

In addition to the basic system configuration, the need to train new ML vision models was needed but lucky there is many ways to do this. At the time of writing there are multiple online services that make this process super easy. All you need is a dataset of images and some patience to label and annotate them using these tools. The initial model that was built was done using RoboFlow's annotation tools.

Workflow

With the system configuration in place and the core components selected, the basic workflow was needed. The basics of this are as follows (more detail in a section below).

MECH

• MECH powers up
• Load configuration settings from memory
• Connects to WiFi
• Connects to Battle Server
• Starts Camera WebServer
• Wait for commands from Battle Server (motion, actions, etc)
• Executes command
• Send command completed to Battle Server

Battle Server

• Loads ML Model
• Connects with MECHs (from 1 to n...)
• Runs inference on latest image from MECH camera
• Executes strategy based on information
• Sends command to MECH (motion, actions, etc)
• Receives command completed from MECH

3D Design

I have designed and created multiple MECHs but here is the first one you can just download and 3D print yourself or click here to order the models if you don't have access to a 3D printer.

Components

Below is a basic breakdown of the design and what each component does

• Fist - detachable part of the MECH that can be "knocked off"
• Sliding Arm - connects the fist to the action servo. Translates rotational motion to linear motion
• Base Frame - interfaces for most of the components of the MECH
• Battery/AI Thinker (ESP32) Holder Front - supports the battery, AI Thinker module and Ultrasonic sensor
• Battery Holder Rear - supports the battery and provides cabling guides
• Skid Plates Front and Rear - provides stability and support to the MECH
• Wheels - press fit style that connect directly to the servo splines
• Tyres - provides grip

3D Printing Your MECH

If you are lucky enough to be able to print the parts yourself I believe you would have a good idea on what is required to get good results. I have created multiple MECHs but here is the first one you can just download and 3D print yourself or click here to order the models if you don't have access to a 3D printer.

I found the best results for your first MECH is to print it out of PLA for everything except the tyres which you can print out of TPU as you need a bit of grip and the PLA will just slip/slide.

Suggested Print Settings

• Material: All parts (except tyres) can be printed in whatever material you have (original was in PLA). Tyres to be printed in TPU.
• Supports needed: Yes (touching build plate only)
• Layer height: 0.2mm
• Infill: 50%
• Nozzle: 0.4mm or 0.6mm

Troubleshooting Tips & Suggestions

Print times and finishes can vary a lot based on your print settings.

Electrical

Hookup Diagram

Above is the hookup diagram for the MECH. More information can be found on the Github Repo here.

Components

• Power - 5V

The MECH was designed around 5V power source, I decided to use as many off the shelf components as possible to allow it to be put together with as little effort as possible.

Busbar - Distributing power was done by soldering the short leads of 2x8 header pins together to create a busbar. This made it very easy to have all the power in one spot for distribution.

Power Bank/Battery - It is important to have a power bank that can deliver +2A @ 5V as the ESP32-CAM will keep resetting and you will have not enough power to drive the servos when fights break out. It also provides a regulated 5V output and power indicator. If not using a power bank but using batteries, make sure to regulate the voltage to ~5V.

USB Power Cable - I modified a USB cable by cutting it and soldering two Female Jumper Ends to the positive and negative ends of the cable. The data lines of the cable were left unattached.

• Servos - FEETECH FT90R

Micro Continuous Rotation/360 servos are connected and controlled by PWM signals from the ESP32 module. These little servos are a great choice as they common and easy to interface. More information on these effectors can be found here.

• Ultrasonic Sensor - HC-SR04

The ultrasonic sensor is used to provide range sensing capabilities and also assist in decision making within fights. It requires 5V (there is 3.3V variants available) and two I/O connections (Trigger and Echo). More information on the sensor can be found here.

• Microcontroller - AI Thinker/ESP32-CAM

This module is a great choice for this project as it is small, capable of running off 5V and has enough I/O capabilities for the system. It also comes with a camera module attached which makes for a clean design. The power for the module comes from the 5V busbar. The wiring diagram/hook up guide shows the connections to the sensors and effectors. For more information on this module, you can find it here.

• FTDI Programmer - PL2303TA

This is used to program the AI Thinker/ESP32-CAM module. It must be noted that the FTDI unit be a 5V variant as it has been heard that the AI Thinker/ESP32-CAM module can have issues when using a 3.3V variant.

Troubleshooting Tips & Suggestions

It is worth noting that you can use an inline USB FTDI 5V programming cable as the power cable from the battery to the AI Thinker/ESP32-CAM module and also leave it connected to the programming pins. This would allow you to use the same cable for programming and power. The one limitation to this approach is that most computer USBs do not supply enough power so you will be able to program and do basic debugging with it connected to your computer, the AI Thinker/ESP32-CAM module will reset if it can not get enough power (when the MECH is moving and fighting).

MECH Assembly

Assembly of the Punchy MECH is quite simple if you have all the parts and tools. The steps below are best carried out in order to ensure the quickest and easiest assembly process. Be careful when assembling and make sure to check orientation of each part before securing in place.

Assembly Steps

1. Remove any excess flashing material from the 3D prints with a pen knife.

2. Using one of the M3 socket head screws, thread the holes of the 2x Skid Plates and the single hole on the front of the Base Frame. Screw a M3x8mm Socket Head screw into each of these threaded holes ~3mm deep.

3. Using the same process above, use a M3x8mm bolt to thread a hole into one of the white Servo Disc/Plate the servo was supplied with but leave the M3x8mm bolt screwed in ~2mm deep.

4. With a 3mm drill bit, clear out the other clearance holes in the Base Frame, Battery Holders (front and rear), Skid Plates that will have M3 connections.

5. Press fit one of the 8mm diameter Magnets into the Slider. Do this on a flat surface being careful to keep everything aligned.

6. Similar to above, press fit another 8mm diameter Magnet into the Fist

7. Fit the two Tyres to the Wheel hubs.

8. Using one of the servos, press fit the spline into the wheel. Do this on a flat surface being careful to keep everything aligned. Remove the Servo from the Wheel carefully.

9. Fit the 3x Servos to their relevant positions on the base frame and secure with the M2x8mm self tapping screws.

10. Push the 3 leads of the Servos through the Base Frame holes.

11. Attach the white Servo Disc/Plate to the action Servo. Secure in place with a M2x8mm self tapping screw.

12. Fit the Slider to the Base Frame slot and secure in place with the Slider Plate and a M2x8mm self tapping screw from the underside.

13. Mount the Rear Battery Holder and one of the Skid Plate to the rear of the Base Frame with 2x M3x12mm bolts and nuts.

14. Mount the Front Battery/AI Thinker holder and the other Skid Plate to the front of the Base Frame with 2x M3x12mm bolts and nuts.

15. Fit the rubber band by sliding it between the Slider (hook end) and the Action Servo. On the underside of the Slider there is a small hook that it secures to. With the other end of the rubber band, pull it forward and loop it around the ear/tabs on each side of the Front Battery Holder.

16. Securely mount the AI Thinker into its mount (slide/interference fit).

17. Fit the Ultrasonic sensor to the Front Battery/AI Thinker Holder.

18. Fit the Busbar with the Servos, Ultrasonic Sensor and Power to the AI Thinker board. Make sure the pins match up with the hookup diagram.

19. Fit the Wheels onto the Servos and secure in place with M2x8mm self tapping screws

20. Fit the USB Power Bank/Battery by inserting it through the Rear Battery Holder to the Front Battery Holder. There is a small tab on the Front Battery Holder that you can manually bend forward 45 degrees.

21. Plug the USB cable into the USB Power Bank/Battery and tidy up the wires. Secure in place with a Velcro strap or a Cable Tie

22. Attach the Fist to the Slider

AssemblyTroubleshooting Tips and Suggestions

• If you have a M3 and M2 Tap, use it to thread the holes in Steps 2 & 3
• Take your time when assembling to ensure the correct orientation of each part
• The splines on the servos are very soft so take your time when press fitting the wheels

MECH Software (C code)

The MECH software is quite straight forward and easy to follow if you have some Arduino/C experience. As the project is continuously evolving I will only touch on the core concepts here but for further details, please checkout the latest information on the Github Repo here.

Programming Environment

VS Code - Everyone has their own preference but this is what I suggest ;)

PlatformIO - This is highly advised to use instead of the Arduino IDE. If you are new to microcontrollers, I recommend watching some tutorials on PlatformIO. It has a slightly steeper learning curve than Arduino's IDE but the benefits outweigh this once you want to do more complex things.

Architecture (Software Defined Hardware)

The goal was to make the MECH software an interface to the hardware that resides on it. This way it would be fully configurable and controllable remotely without having to update its code as frequently. By having access to the hardware remotely would allow the user to have greater control and flexibility to how it acts.

Program Structure

Setup

• MECH powers up
• Load configuration settings from memory
• Connects to WiFi
• Connects to Battle Server
• Starts Camera WebServer

Main

• Wait for commands from Battle Server (motion, actions, etc)
• Executes command
• Send command completed to Battle Server

For the latest implementation of the MECH code, please checkout the latest information on the Github Repo here.

Computer Vision (Train & Test)

Computer vision is a very complex field and there is so many tools and frameworks that can be used to perform vision based objectives. I wanted to simplify this to make it less daunting for someone starting in this space so I looked to off the shelf, well documented frameworks with large communities that provide support. At the time of writing this, the vision model used was YOLOv8 by the team at Ultralytics. It is a easy to implement CV model that provides enough flexibility while being simple to implement. More information can be found here.

Note: The easiest way to get started is to pull our latest pre-trained weights and use them as this will allow you to get up and running straight away. Head to the Github Repo to pull our latest models here

Custom Dataset Training

As our MECHs aren't (currently) part of the larger well known public datasets, we needed to create our own. This process is not difficult but it is very time consuming but lucky there are many companies that provide great tools to speed up this process. One such company is Roboflow who provides a lot of free tools/tutorials to create your own custom dataset.

Capturing Images

This process requires is pretty self explanatory. The main thing to note here is the image size in pixels as this is what the model will be trained on. It is recommended to use our MECH Image Capture tool to capture images. The more images you take, the better the results after training will be.

Annotate, Train and Test

For this section I highly recommend following the simple guide the team at Roboflow created here

Troubleshooting Tips and Suggestions

• Having a diverse set of images will help build a good model. Make sure to collect a lot of images at different angles and lighting conditions.
• It is also good to include some images where there is no MECH in the dataset so the model also knows that sometimes it will not see something.
• Image Size is very important when creating a dataset. Most of the models require a set/know image size in the dataset.

The Battle Server - Inference & Fight Strategy

At the heart, actually brain of the system is the Battle Server. This is where all the real processing and application of strategy is done. This was designed to do all the heavy lifting on a computer instead of on the microcontroller to keep costs down and the speed of the system high.

Below is a high level overview of the Battle Server but as with everything created here, it is best to checkout the latest information over on the Github repo here.

Note: At the time of writing, the MECH Strategy is integrated into the Battle Server code base. This is not ideal and is currently under development to have it as a separate Code that can be referenced by the Battle Server. Keep an eye on the Github repo here.

Programming Language

Python was selected for the software language as it is probably one of the most well known and documented.

Programming Environment

VS Code - Everyone has their own preference but this is what I suggest ;)

Program Structure

Setup

• Loads ML Model
• Connects with MECHs (from 1 to n...)

Main

• Runs inference on latest image from MECH camera
• Executes strategy based on information
• Sends command to MECH (motion, actions, etc)
• Receives command completed from MECH

Strategy

The Strategy each MECH has can be modified to suit your needs. When a new image is received from a MECH it performs object detection. The results from the inference are then made available to the Strategy. Inside the Strategy the user can perform decision making based off the data. These decisions generally result in motion and action commands. Some of the current parameters include, velocity and direction control of servos, actions, path planning, searching algorithms.

For further details on the most up to date Battle Server, checkout the latest information over on the Github repo here.

Welcome To Autonomous Fight Club (AFC)

So now you have built your MECH (hopefully you built 2) and the Battle Server is setup and running, you are ready to fight!

Autonomous Fight Club

Are you ready for the ultimate test of engineering and strategy? Introducing the Autonomous Fight Club, where MECHs go head-to-head in an epic battle of strength and wit.

These aren't your ordinary robots - these are machines equipped with advanced sensors and compute modules that allow them to see, move, and understand their opponents and the world around them. And their weapon of choice? A powerful arm with a fist ready to swing at their competition. We call them MECHs!

Here's how it works: each contestant brings their own MECH to the ring, referee will say "fight!" and the MECHs will start searching for each other using their sensors. Once they've locked onto their target, the real action begins - a full-on fist fight until one MECH is "disarmed" and its fist is knocked off.

But this competition isn't just for professionals - it's open to anyone of any age who wants to build, compete and conquer! So if you think you've got what it takes to build the ultimate fighting machine, sign up for the Autonomous Fight Club today and build your very own MECH!

Basic Rules

The rules of Autonomous Fight Club are simple.

1. The first rule of Autonomous Fight Club is that you talk about Autonomous Fight Club.

2. All MECHs must be autonomous, meaning they cannot be remotely controlled by an animal during the competition.

3. MECHs must be equipped with a single arm with a fist that is magnetically coupled.

4. The fight begins when the referee says "fight!" and continues until one MECH is "disarmed" or unable to function. The remaining MECH is declared the winner.

Where to from here?

It seems like a lot of people have really loved building their own MECHs and personalising them in ways I never thought of which is super cool. The goal I have now is to support this project and allow the community build upon it. Below are some of the features we are developing and testing out already. If you want to get involved, please reach out as we would love to see this project encourage more people to have a greater understanding of AI, ML, CV and Robotics.

Here are some of the features that are currently under development.

• ML Strategy
• Mods for your MECH (skins)
• AI referee and Commentator
• Fight dashboard for online viewing/streaming
• Different sized MECHs (cordless power tool MECH, Earth Moving MECHs)
• Different weapons (Punchy, Slappy, Choppy, Flippy)
• Non-fighting MECHs (dog patting MECH)
• Build a better vision model (more data)
• MECH personalities (voices, sounds, intro music)
• More cameras and sensors
• Competitions and rules
• Pre-assembled kits