AI Basketball Referee Detects Traveling

In case you’re not a basketball fan and your parents never forced you to play the sport as a child: traveling in basketball is when a player takes more than two steps without bouncing the ball on the court. This move is illegal to keep players from grabbing the basketball rugby-style and simply running across the court. Unfortunately, basketball referees are just as prone to mistakes as referees of other sports and they sometimes fail to call out traveling. To provide objectivity, YouTuber Ayush Pai created an artificially intelligent basketball referee with the sole job of detecting traveling.

For this system to work, Pai’s artificial intelligence (AI) needed to monitor a player’s steps and every bounce of the basketball. This doesn’t work with multiple players (at least not yet), so the AI only needed to keep track of a single person. It would be possible to expand this system to monitor a real basketball game by only tracking the player with the ball, but that would require several camera angles and step tracking for every player. That step tracking turned out to be a real challenge and Pai’s solution was quite clever.

Pai didn’t want the AI to try to count steps through the camera feed, as that would be unreliable and very resource-intensive. Instead, he wanted to use a pedometer to count the player’s steps. He planned to use his Apple Watch’s built-in pedometer function, but found that it introduced a significant delay between a step and an update to the step count. Pai needed real-time step counting, so he programmed his own Android pedometer app. The player straps the Android smartphone to their leg and the app sends a step event as soon as it occurs to the computer running the AI.

The dribbling detection was more straightforward. It uses simple computer vision software to create a mask of the basketball based on color. With that mask, the software could draw a circle representing the ball. Because a bouncing ball follows a standard parabolic arc, it was easy to determine if the movement was a dribble or just the player moving the ball in their hands. Anytime the ball reaches the bottom of that parabolic arc, it counts as a dribble.

With those pieces in place, detecting traveling was trivial. If the player takes more than two steps without the AI seeing a dribble, that indicates traveling and the system can sound an alarm.