Software apps and online services | ||||||
 |
| |||||
| ||||||
Every holiday I get together with family and "Game" night is a big part of the festivities. A favorite of the family is "Dominoes" where we play a variation called "Mexican Train" which you can learn about here if you do not know how to play. Basically, you draw out a number of tiles and you match them until you have none left and your opponents all have to count the dots called "Pips" on their tiles they had left in their hand they had not yet played. You keep score by tracking these left-over pips. Unfortunately, counting the pips slows the game down and no one likes counting up the pips. We thought there must be an app for that in the app store.Here is a typical start to a game:
iPhone to the rescue?Afterall, how hard could it be to snap a photo and get the iPhone to count them up? Well, it turned out that it must be pretty hard because no one was offering an app. We searched and had not luck finding one.
I saidguessI will have to build it - and sowe did using OpenCV!
I looked at ChatGPT4 with Vision as certainly a multimillion-dollar GenAI model can count dots. Afterall it's this great big complex neural model and the Pips are always clearly and consistently marked. No issue, right? Nope! It could not do it and I'll show you the proof here on my YouTube video where we go head-to-head with ChatGPT4 Vision with our OpenCV solution, and we destroy it!
Before we get into the code you can our Open CV solution here just as we shared in the video as I have hosted it over on Hugging Face for public use. https://kdb8756-pip-counter.hf.space
Go ahead and check it out if you have a few minutes. There are some sample images to try so you can see it in action. You can even upload your own images if you have some dominoes and would like to try it. It is free and starts on demand.
Please Remember to use this link to get the code from my GitHub so you do not have to type in all the code or try to come up with test images. I also have included all that to help you get started.
Now that you know where to find the code let's take a look at how it works. We will start with a flowchart, and you will see how the OpenCV lets us keep things concise and efficient._______________________________________________________________A quick Visual for the program for reverence:
- Start: The process begins.
- Read Image: An image is uploaded and read.
- Resize Image: The image is resized for consistent processing.
- Is Image None?: Checks if the image is null.
- Yes: If the image is null, the process ends returning None.
- No: If the image is not null, it proceeds to the next step.
- Convert to HSV: The image is converted to HSV color space.
- Apply Hue Threshold: A hue threshold is applied for color filtering.
- Create Mask: A binary mask is created for the specified color range.
- Search Contours: The script searches for contours in the image.
- Draw Contours and Calculate Pips: Contours are drawn, and pips (dots) are counted.
- Put Text on Image: The total pip count is displayed on the image.
- End: Return Processed Image: The processed image with pip count is returned.
_______________________________________________________________
Our Pipcounter Coding AdventureOur coding adventure begins with gathering our tools - the Python libraries that make the magic happen. We have OpenCV, a powerhouse for image processing; PIL for advanced image operations; NumPy for handling numerical data like a pro; and Gradio, the wizard that conjures up a web interface for our application. And, of course, we can't forget warnings - our little helper that ensures our journey is smooth, keeping those pesky warnings at bay.
Like any good story, we need to set the stage. Here, our stage is set with global variables. We define the style of text we'll use on our images, the color palettes (though they're just waiting backstage for now), and the initial hue value - a key player in color filtering.
The search_contours Function
The core of our story revolves around search_contours, a function that's like a treasure hunter, seeking out contours (or edges) in images. It’s where we sift through the image, looking for areas of interest - in our case, the pips on dominos. This function meticulously draws these contours and marks their centers, counting them like a seasoned explorer counts his found treasures.
center_crop_with_padding
Every story needs a twist, and center_crop_with_padding is ours. This clever function takes an image and transforms it, cropping it into a perfect square and then elegantly framing it with a white border. It's like giving the image a new perspective, focusing only on what's truly important.
detect_pips_uploaded
As we reach the climax of our tale, detect_pips_uploaded takes the stage. This is where the real magic happens. We take an image - a moment captured in time - and resize it for consistency. We then shift our view to the HSV color space, a realm that offers a different perspective on colors. By creating a mask within a specific range, we isolate the parts of the image we're most interested in. And then, with a call to our treasure-hunting search_contours, we count the pips, finally revealing the hidden story within the image.
But what’s a story without a way to share it? Enter Gradio, our wizard, creating an interface as if by magic. This interface is our portal, allowing users to upload their images and set the hue threshold, a crucial factor in our pip-counting spell.
As our adventure comes to an end, we launch the Gradio interface. It's like opening the doors to an enchanted castle, inviting users from far and wide to partake in our image-processing feast, to explore the mysteries hidden within their images.
Our journey through the realms of Python, OpenCV, and Gradio has been a tale of discovery, creativity, and technological wizardry. It's a testament to the power of code, the art of image processing, and the joy of sharing knowledge through interactive applications._______________________________________________________________
Github Code Link for Easy access to the code:jjmlovesgit/pipcounterHere is a traditional code review with code list with inline comments for those who prefer the classic review style.# Import necessary librariesimport cv2 # OpenCV for image processingfrom PIL import ImageOps # PIL for image operations like cropping and paddingimport numpy as np # NumPy for numerical operations and array handlingimport gradio as gr # Gradio to create a web interface for the applicationimport warnings # To handle warnings# Suppress user warnings to keep the output clean and readablewarnings.filterwarnings("ignore", category=UserWarning)# Global variables for font style and color palettefont = cv2.FONT_HERSHEY_SIMPLEX # Font style for text in OpenCVfont_scale = 1 # Font scale (size)color_search = np.zeros((200, 200, 3), np.uint8) # Color palette for search (not used in this script)color_selected = np.zeros((200, 200, 3), np.uint8) # Color palette for selected (not used)hue = 0 # Initial hue value (for color filtering)def search_contours(mask, frame, source): """ Find and draw contours on the image. :param mask: Binary mask for the specific color range :param frame: The original image frame :param source: The source of the image (not used in the function) :return: Count of contours (pips) and the frame with contours drawn """ contours, hierarchy = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) pip_count = 0 for contour in contours: area = cv2.contourArea(contour) if 200 < area < 10000: # Filter out contours that are too small or too large cv2.drawContours(frame, [contour], -1, (0, 255, 0), 2) # Draw contour pip_count += 1 # Calculate the center of the contour M = cv2.moments(contour) cX = int(M["m10"] / M["m00"]) if M["m00"] != 0 else 0 cY = int(M["m01"] / M["m00"]) if M["m00"] != 0 else 0 cv2.circle(frame, (cX, cY), 3, (255, 255, 255), -1) # Draw center point cv2.putText(frame, str(pip_count), (cX - 16, cY + 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 1) return pip_count, framedef center_crop_with_padding(image): """ Crop the image to a square and add white padding. :param image: The original image :return: The cropped and padded image """ im = Image.fromarray(image) width, height = im.size new_size = min(width, height) # Crop the image to a square im = im.crop(((width - new_size) // 2, (height - new_size) // 2, (width + new_size) // 2, (height + new_size) // 2)) # Crop further and add a border im = im.crop((25, 25, new_size - 25, new_size - 25)) im_with_border = ImageOps.expand(im, border=3, fill=(255, 255, 255)) return np.array(im_with_border)def detect_pips_uploaded(uploaded_image, hue_threshold): """ Detect and count the number of pips in the uploaded domino image. :param uploaded_image: The image uploaded by the user :param hue_threshold: The threshold for hue filtering :return: Processed image with pip count """ if uploaded_image is None: return None image = uploaded_image[:, :, :3] # Remove alpha channel if present image = cv2.resize(image, (512, 512)) # Resize image for consistent processing hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV) # Convert image to HSV color space # Define the HSV range for color filtering lower_hsv = np.array([max(0, hue - hue_threshold), 50, 20]) upper_hsv = np.array([min(179, hue + hue_threshold), 255, 255]) mask = cv2.inRange(hsv, lower_hsv, upper_hsv) # Create a binary mask for the specified color range count, result_frame = search_contours(mask, image, source="image") # Detect contours (pips) # Display the total pip count on the image cv2.putText(result_frame, f'Total pip count is: {count}', (10, result_frame.shape[0] - 30), font, 1, (0, 255, 255), 2, cv2.LINE_AA) return result_frame# Gradio interface setupiface_uploaded = gr.Interface(detect_pips_uploaded, inputs=[ gr.inputs.Image(type="numpy", label="Upload Image", source="upload"), # Image upload input gr.inputs.Slider(label="Hue Threshold", minimum=0, maximum=500, step=1, default=250) # Slider for hue threshold ], outputs=gr.outputs.Image(type="numpy", label="Result"), # Output display title='<b><font size="4">🁫Image Processing demonstration of the findContours function in OpenCV using Python:🁫</b>', description='<div style="text-align:center;"><b style="font-size:20px;">An application to help you keep score when playing Dominos</b><br>Use your Ipad take photo and upload to calculate a score<br>Note: lighting and camera angles are key factors :)</div>', allow_flagging=False, live=False, theme="dark",)# Launch the Gradio interfaceiface_uploaded.launch()
_____________________________________________________________
Project SummaryWhat you saw today was a project where we created a program written in Python with the OpenCV library to let us count out "Pips" on our dominoes. I had a lot of fun doing this. We used it when we played Dominoes at our last family holiday, and we all had a great time with it. I hope you try it out. Remember the hugging face link will let you play with it any time.The takeaway to remember is that that new is not always better. ChatGPT4 cannot count pips! It's important to use the right tool for the task. Not every problem is an AI/ML problem for a neural network! Do not forget about the excellent libraries like OpenCV that deliver consistent predictable performance across a wide variety of use cases. In our case our tried-and-true partner OpenCV delivered an exceptional near 100% success rate with the task!
Comments