Brownie - Automate Your Home with AI

Need to know a door opens or closed, someone is coming and a trash can is full or not in your house with an AI camera? It's cool!!

When you start to use machine learning especially using deep learning, you might be found that you must have a lot of images to work because of improving generalization performance.

But wait. Is it needed for you such a generalization performance at home? Can you take many pictures to create a model? It's so hard...sigh.

So I made an easy-to-use AI camera app, that needs only one image to work well at home.

How do you do that?

1. Use a deep learning model as a feature extractor. In other words, automatically find nice features in the image with a model that was trained lots of images.

2. Compare with features similarity with a classical and simple machine learning algorithm such as k-Nearest Neighbor, and find the nearest feature.

That's all. It surprisingly works at home.

Here is an example of it.

Setup for M5StickV device

1. Download MakerFaireTaipei2019.zip at https://github.com/ksasao/brownie/releases/tag/v2.0 .

2. Unzip and copy all files under the M5StickV directory into your microSD card.

3. Insert the card into your M5StickV device.

4. Connect your USB C cable with PC and M5StickV device.

If you need to fix your device firmly to a tripod, you can output the spacer below with your 3D printer.

Setup for your PC

1. Copy Windows directory in the zip file above to your Desktop of your pc.

2. Double click a BrownieMonitor.exe to run the app.

3. Input COM port name of your M5StickV connected to.

4. Push the "Connect" button.

How to use

1. To associate an object with the name, prepare a QR code that has the name of it. You can use the QR Code Generator https://www.the-qrcode-generator.com/. You can also use QR.pdf including the zip file above.

2. Present a QR code to the camera to learn a scene or item. When a red frame will appear, you remove the QR code and show a scene or item you want to be learned. To reset that learned, you just show a *reset QR code to the camera.

3. Verify that the target is recognized correctly (Green Frame will appears) and check the name shown on the Brownie Monitor app.

4. Brownie Monitor will kick a command when it receives the name from the camera. Any command will be accepted such as python. Use start to launch default app associated with file suffix (e.g..mp3,.png,...).

Application examples

• Play a welcome sound when a door opens
• Turn off the light when you sleep in bed
• Switch what is displayed on the screen when someone comes behind you :-)

If you find a cool use case, please tell me on the Twitter (@ksasao)!