ChessAI - Chinese Chess Game Analyzer with Computer Vision

ChessAI is a groundbreaking tool that brings together computer vision, chess algorithms, and advanced analytics to revolutionize the Chinese Chess analytics landscape. With ChessAI, you don't need expensive electronic boards to analyze your games. Simply use your regular board, set up a camera to capture the position, and let ChessAI do the rest.

Features

• Chessboard Analysis with OpenCV: ChessAI uses cutting-edge computer vision algorithms to analyze a chessboard's position from a simple camera shot. It identifies pieces, their positions, and the overall game state, providing you with a digital snapshot of your game.
• Chess Engine Integration: We've integrated ChessAI with a powerful chess engine. This engine uses complex algorithms and vast databases of previous games to suggest the best possible moves based on the current board position. It's like having a grandmaster at your side, guiding your gameplay.

Implementation

Feature 1: Chessboard Analysis with OpenCV

This project uses computer vision and deep learning to detect chess pieces and chess board positions. The processing flow is as follows:

The image from the camera is pushed through the Board Aligner block, which uses OpenCV ARUCO and the homography alignment algorithm. After cropping and aligning the chessboard area, we use an object detection model based on the YOLOX model to detect the chess pieces.

We obtain the chess board position by comparing the detected chess pieces and reference positions. After that, this position is converted to an FEN string, which is a standard way to present a chessboard.

Feature 2: Chess Engine Integration

The chess engine godogpaw is integrated as the default chess engine to generate move suggestions. After receiving a FEN string of the current chess board, that representation will be used to ask for the next move.

Feature 3: Web UI

We developed a web UI for users to use ChessAI. This web UI uses FastAPI for the backend and React/TailwindJS for the front end.

Challenges and Lessons Learned

• The dataset built from scratch: In order to build a dataset for chess piece detection, we collect ~800 images using Google Images Search. The data collection took a lot of time. Fortunately, the labeling process was sped up by using the Segment Anything mode of the AnyLabeling tool.
• Currently, the chessboard is aligned using the ARUCO codes. However, there are many situations in which the ARUCO codes are occluded by other objects, causing problems for the board alignment step. The solution can be to train a new deep-learning model to detect the chessboard without using ARUCO.

Source code

• The source code is open at https://github.com/vietanhdev/chessai/. Totally free to use and redistribute.

References

• The object detection model (for chess pieces) is based on YOLOX - License: Apache 2.0.
• Chess engine: godogpaw - License: MIT.
• UI components: shadcn-ui - License: MIT.