Tennis Smith's Cat Doorbell Uses On-Device Machine Learning to Spot a Cold Cat via Sight and Sound

Maker Tennis Smith has built a Raspberry Pi-powered "cat doorbell," designed to use on-device machine learning (ML) to figure out when a cat wants to come back in from the cold — and to send its humans a text message alert so they can come to its assistance.

"We have a cat that likes to go out on our enclosed patio. When he is ready to come in, he will stand next to the door and meow (yell). We open the door and let him in. No problem," Smith writes by way of background to the project. "But, he frequently stays out long enough for us to forget. More than once he was outside yelling and we were oblivious. That's Problem #1."

"It is also complicated by the fact that he likes to yell anyway," Smith continues. "He will lay on the ground wallowing and yelling just for the fun of it. That causes 'false alarms.' We would think he wants in, but all he was really doing is enjoying life. Frustrating to us (but I think it secretly amuses the cat). That's Problem #2. Somehow, we needed a device that would: alert us when the cat wanted inside (fix Problem #1); ensure that was really his intention (fix Problem #2). Clearly, we needed a Cat Doorbell."

Smith's initial version of the project worked on sound alone, listening out for the cat's miaow and triggering an alert accordingly — but was prone to false alarms. The solution: the addition of a camera, triggering alerts only if the cat is both seen and heard. The secret sauce: a Python-powered program which adopts two TensorFlow example projects to identify the cat by sight and sound using on-device machine learning.

"This is essentially a small state machine," Smith writes. "The Doorbell listens passively for the sound of a cat meowing. When it hears that sound, it enables a camera. If the on-board light sensor detects darkness, an LED strip will be turned on. For 45 seconds the Doorbell uses the camera in an attempt to identify a cat. If no cat is identified, the Doorbell goes back to passively listening."

"If a cat is identified during the 45-second window," Smith continues, "a text message is sent to me. The system then pauses for two minutes to prevent triggering a new alarm. If dark, the LED light stays on until after the two-minute pause is over. The Doorbell then goes back to listening."

Smith's full project write-up is available, alongside the source code under an Apache 2.0 license, on his GitHub repository.