AI camera with Spresense

What is this project about?

The device uses a camera and a trained neural network model to recognize when I seat next to the lamp and switches it on.

It recognizes patterns, can count number of people and can be trained for specific commands. Besides, this allows to save energy switching of when we are not present.

How it works?

The Spresense Main Board uses the camera and a trained model to detect when the user is present. By strategically placing the Spresense on the room's corner, it can turn lights on and off as a user moves through the room. To do this, the Spresense is connected to a relay that acts as a switch. This allows the user to move around without touching the switches and making a better use of energy. This application requires the Spresense Main Board + Sprense Camera Board.

Prototype

Create dataset

First we need select the interest region. In this case is 56x56 pixels that should be resize to 28x28 using the code below.

Then we need to record at least 600 photos, half with someone seated and the other half just the sofa. Record them with different light conditions. Modify create_dataset.ino to your region of interest.

img.clipAndResizeImageByHW(small, lefttop_x, lefttop_y,
                    rightbottom_x, rightbottom_y, out_height, out_width)

After recording, copy the images from the SD card and classify them in two folders.

Now, in Neural Network Console click on DATASET > Create Dataset > Image Classification and add specify

Train the model

Find the model that works better for your problem. This worked for me.

Then, run trainning, run evaluation, export NNB model and paste it to the SD card. In my case the model is less than 5 KB!

Inference

Modify the following line on inference.ino with your region of interest and run it!

img.clipAndResizeImageByHW(small, lefttop_x, lefttop_y,
                    rightbottom_x, rightbottom_y, out_height, out_width)

To switch the light I use a relay at 3.3V with optocoupler as you see on the next photo.