Missing Item Finder

Overview

Imagine that you have a bad habit of forgetting things easily like your keys, cell phone, shoes, watches, switching off T.V, A.C, fan, lights etc. I am one of those people who easily forget to switch off TV, fan or where I have kept my keys or cell phones in my own room or sometimes my aunt's little son misplaces important things (he is very small and everything is a toy for him) and it is very annoying. Such problems arises everyday at my house. Sometimes we or any small child gets locked in room by mistake.

I am going to build such device that could identify people and objects in their and take snapshots as per program designed and in unusual conditions it would notify (like wastage of electricity, a quick notification along with image will be sent to respective device). The device would comprise of a camera installed in one corner of the house. Existing solution like RFID tags, BLE tags are not very efficient in their works because a person has to buy separate units of those to keep track of his items which could be done through just a single camera where ever he wishes to implement. As we are stepping towards future we are becoming more dependent on smart technologies and electronic gadgets and thus we are losing our ability to remember things so if technology is the cause for this then it could be a solution for this. Moreover the device will track different objects and would store the observed location in different directory.

Technical Overview

Whenever it does strike my mind of my belongings, Tile and other BLE devices comes to my mind. I tried to install it with my books, purse etc., it came out to be of huge amount. So, I tried to brainstorm something to overcome it, got to know about image processing, object tracking and about Matrix Creator. Matrix Creator comes up with 15 inbuilt sensors making it a powerful tool for my project. It includes temperature sensor, humidity sensor, ultraviolet sensor and many others, a perfect beast box for home automation.

So, the main concept we are going to use is tracking different objects and obtaining their end points(like when someone takes the key and puts it in shirt's pocket, end point become the shirt maybe even note the color of the shirt). Since the problem of forgetting their belongings and are unable to keep track on them is mainly in adults who are very much surrounded with technology. It would be like a personnel assistant taking care each and every things. Moreover assistants like alexa or google home or siri all need to be provided with the net connectivity while on the other side snips is an offline assistant so, privacy is maintained making the biggest reason of using it in my projects.

Background

Home Automation

With increasing population more and more people are getting addicted to technology (including me) forget about the surrounding, their belongings, and spends lots of time with cell phones or a pc or a laptop. Device would even say the status of the room using the on board sensors (like temp., humidity etc. ).

How technology is warping our mind.

https://www.huffingtonpost.in/2013/12/11/technology-changes-memory_n_4414778.html

Let's go!

Setting up Raspberry Pi

There are several models of Raspberry Pi, and for most people Raspberry Pi 4 Model B is the one to choose. Raspberry Pi 4 Model B is the newest, fastest, and easiest to use.

Raspberry Pi 4 comes with either 1GB, 2GB, or 4GB of RAM. For most educational purposes and many hobbyist projects, 1GB is enough; for use as a desktop computer, we recommend 2GB.

Raspberry Pi Zero and Zero W are smaller and require less power, so they’re useful for portable projects such as robots. It’s generally easier to start a project with Raspberry Pi 4, and to move to Pi Zero when you have a working prototype that a smaller Pi would be useful for.

If you want to buy a Raspberry Pi, head to rpf.io/products.

A power supply

To connect to a power socket, all Raspberry Pi models have a USB port (the same found on many mobile phones): either USB-C for Raspberry Pi 4, or micro USB for Raspberry Pi 3, 2 and 1.

You need a power supply that provides:

• At least 3.0 amps for Raspberry Pi 4
• At least 2.5 amps for Raspberry Pi 3

i recommend using https://www.raspberrypi.org/products/raspberry-pi-universal-power-supply/

A microSD card

Your Raspberry Pi needs an SD card to store all its files and the Raspbian operating system.

You need a microSD card with a capacity of at least 8 GB.

Many sellers supply SD cards for Raspberry Pi that are already set up with Raspbian and ready to go.

A keyboard and a mouse

To start using your Raspberry Pi, you need a USB keyboard and a USB mouse.

Once you’ve set your Pi up, you can use a Bluetooth keyboard and mouse, but you’ll need a USB keyboard and mouse for the first setup.

A TV or computer screen

To view the Raspbian desktop environment, you need a screen, and a cable to link the screen and the Pi. The screen can be a TV or a computer monitor. If the screen has built-in speakers, the Pi is able to use these to play sound.

HDMI

The Raspberry Pi has a HDMI output port that is compatible with the HDMI port of most modern TVs and computer monitors. Many computer monitors may also have DVI or VGA ports.

Raspberry Pi 4 has two micro HDMI ports, allowing you to connect two separate monitors.

You need either a micro HDMI-to-HDMI cable, or a standard HDMI-to-HDMI cable plus a micro HDMI-to-HDMI adapter, to connect Raspberry Pi 4 to a screen.

Raspberry Pi 1, 2 and 3 have a single full-size HDMI port, so you can connect them to a screen using a standard HDMI-to-HDMI cable.

DVI

If your screen has a DVI port, you can connect the Pi to it using a HDMI-to-DVI cable.

VGA

Some screens only have a VGA port.

To connect your Pi to such a screen, you can use a HDMI-to-VGA adapter.

Headphones or speakers

The large Raspberry Pi models (but not Pi Zero/Zero W) have a standard audio port like the one on your smart phone or MP3 player. If you want to, you can connect your headphones or speakers so that your Raspberry Pi can play sound. If the screen you’re connecting your Raspberry Pi to has built-in speakers, Raspberry Pi can play sound through these.

An Ethernet cable

The large Raspberry Pi models (but not Pi Zero/Zero W) have a standard Ethernet port to connect them to the internet; to connect Pi Zero to the internet, you need a USB-to-Ethernet adaptor. Raspberry Pi 4, 3, and Pi Zero W can also be wirelessly connected to the internet.

Set up your SD card

If you have an SD card that doesn’t have the Raspbian operating system on it yet, or if you want to reset your Raspberry Pi, you can easily install Raspbian yourself. To do so, you need a computer that has an SD card port — most laptop and desktop computers have one.

The Raspbian operating system via NOOBS

Using the NOOBS software is the easiest way to install Raspbian on your SD card.

Download NOOBS

• Visit the Raspberry Pi downloads page.

• You should see a box linking to the NOOBS files. Click on the box.

• The simplest option is to download the zip archive of the files. Make sure to pay attention to where you save the archive, so that you can find it again quickly.

Format the SD card

Anything that’s stored on the SD card will be overwritten during formatting. So if the SD card on which you want to install Raspbian currently has any files on it, e.g. from an older version of Raspbian, you may wish to back these files up first to not lose them permanently.

Visit the SD Association’s website and download SD Formatter for Windows or Mac.

• Visit the SD Association’s website and download SD Formatter for Windows or Mac.

Follow the instructions to install the software.

• Follow the instructions to install the software.

Insert your SD card into the computer or laptop’s SD card slot.

• Insert your SD card into the computer or laptop’s SD card slot.

In SD Formatter, select your SD card, and the format the card.

• In SD Formatter, select your SD card, and the format the card.

Extract NOOBS from the zip archive

Next, you will need to extract the files from the NOOBS zip archive you downloaded from the Raspberry Pi website.

Find the downloaded archive — by default, it should be in your Downloads folder.

• Find the downloaded archive — by default, it should be in your Downloads folder.

Double-click on it to extract the files, and keep the resulting Explorer/Finder window open.

• Double-click on it to extract the files, and keep the resulting Explorer/Finder window open.

Copy the files

Now open another Explorer/Finder window and navigate to the SD card. It’s best to position the two windows side by side.

• Now open another Explorer/Finder window and navigate to the SD card. It’s best to position the two windows side by side.

Select all the files in the NOOBS folder and drag them into the SD card window to copy them to the card.

• Select all the files in the NOOBS folder and drag them into the SD card window to copy them to the card.

After copying the files you can eject the SD card.

Now connect the SD card to Raspberry pi using larger adapter.

The boards require a Raspberry Pi 3 to work, you will also need:

Micro-USB power adapter for Raspberry PI,

• Micro-USB power adapter for Raspberry PI,
• Micro SD Card (minimum 8 GB) with Raspbian Stretch
• External Speaker with 3.5 mm audio cable

Installing the MATRIX Kernel Modules (Raspberry Pi)

Installing the MATRIX Kernel Modules (Raspberry Pi)

Run the following commands in your Raspberry Pi's terminal to add the MATRIX repository & key and update your repository packages.

curl https://apt.matrix.one/doc/apt-key.gpg | sudo apt-key add -

echo "deb https://apt.matrix.one/raspbian $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/matrixlabs.list

sudo apt-get update

sudo apt-get upgrade

curl https://apt.matrix.one/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.matrix.one/raspbian $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/matrixlabs.list
sudo apt-get update
sudo apt-get upgrade

Reboot your device.

sudo reboot

sudo reboot

Install the MATRIX Kernel modules. Aside from a few other changes, this allows the microphones on your MATRIX device to register as an ALSA microphone on your Raspberry Pi.

sudo apt install matrixio-kernel-modules

sudo apt install matrixio-kernel-modules

Reboot your device again

sudo reboot

sudo reboot

These next commands will test your microphones by raising your output volume, creating a 5 second long audio recording, and then playing that recording.

Make sure you have a device that can output audio plugged into your Raspberry Pi's HDMI or Audio Jack.

amixer set PCM 100% && arecord recording.wav -f S16_LE -r 16000 -d 5 && aplay recording.wav

amixer set PCM 100% && arecord recording.wav -f S16_LE -r 16000 -d 5 && aplay recording.wav

Installing Sam CLI Tool

Installing Sam CLI Tool

On your computer, you will be using the SAM CLI tool to create, manage and deploy your Snips assistants, to configure your hardware, to view the logs of your assistants on the Raspberry, as well as to run your application locally for quick prototyping.

On your computer, you'll need to install Node.js as a prerequisite. Once installed, you can install SAM into your computer with the following command.

sudo npm install -g snips-sam

sudo npm install -g snips-sam

Installing Snips Through Sam

Installing Snips Through Sam

Although the installation will be on your Raspberry Pi, Snips will be installed through the SAM CLI tool from your personal computer. Open your personal computer's terminal and input the following commands.

Connect to your Raspberry Pi through one of the following:

Using your Pi's hostname.

• Using your Pi's hostname.

sam connect raspberrypi.local

sam connect raspberrypi.local

Or your Pi's IP address.

• Or your Pi's IP address.

sam connect 192.168.1.182

sam connect 192.168.1.182

If you don't find your device, run sam device or follow this entire guide.

You'll then be prompted to insert a username and password. The default login is as follows:

Username: pi

• Username: pi

Password: raspberry

• Password: raspberry

After you're connected use the init command, from your computer, to install Snips onto your Raspberry Pi.

sam init

sam init

Configuring Snips (Raspberry Pi)

Configuring Snips (Raspberry Pi)

With Snips properly installed, the next step is to edit the snips.toml file for configuring the mics.

From a terminal session on your Raspberry Pi, run the following:

sudo nano /etc/snips.toml

sudo nano /etc/snips.toml

Scroll down to where you see [snips-audio-server] and replace

# mike = "Built-in Microphone"

with the following:

mike = "MATRIXIO-SOUND: - (hw:2,0)"

Press ctrl+x, press y, and then press the enter key to save your changes.

Testing Snips (using Sam)

Testing Snips (using Sam)

Through your personal computer's terminal, you can use SAM to test the speaker and microphone of your Raspberry Pi.

Testing your Raspberry Pi's speakers

• Testing your Raspberry Pi's speakers

sam test speaker

sam test speaker

Testing the microphones on your MATRIX device

• Testing the microphones on your MATRIX device

sam test microphone

sam test microphone

If both commands seem to work, you've properly configured Snips on your Raspberry Pi!

This final command will test if Snips is properly working by installing a demo assistant for the weather.

sam install demo

sam install demo

Once installed, test the assistant by saying

"Hey Snips, what will the weather be like in Paris in two days?"

The response from Snips should be:

"You asked for the weather in Paris in two days.

Setting up your assistant

A voice assistant is what enables you to control a device using voice. It is composed of a set of apps. For instance, a 'Smart House’ assistant may have a ‘Lights’, ‘Thermostat’ and ‘Oven’ apps for instance. On the other hand, an app has the responsibility of understanding the user intention when using voice commands and to trigger the expected behavior, an action, to fulfill the intention of the command.

In order to create a voice assistant you will need to use the Snips console. From there you can add apps(s) to your assistant or create apps from scratch if you wish to.

In the following articles we will see step by step how to create an assistant and how to deploy it your device.

Video Tutorials

Video Tutorials

If you prefer to follow along with a video, we have a prepared a video tutorial covering the most basic steps (Create your Account, Add Apps to your Assistant, Deploy your Assistant) to create an assistant with some apps deployed to your Raspberry Pi.

First of all, you’ll need to create an account on the Snips console. Please go to https://console.snips.ai/signup​

Enter your username, email address, password and click on ‘Sign up’.

Congrats! You’re onboard!

Create an Assistant

Create an Assistant

Once you successfully signed up, you’ll be prompted to create your first Voice Assistant directly.

If it’s not the first time you’re using the Console, go to the Console’s home page and click on ‘Create a New Assistant’.

In this guide, we’ll name the Assistant ‘Smart House’ and will pick ‘English’ as a language.

Congrats! Now that you have an Assistant, let's add apps to it.

Add Apps

To empower your Assistant, you’ll have to add Apps to it. Remember apps are made of Intents (understand the users’ Intention from their voice query) and Actions (trigger the expected behavior from the detected Intent). Let’s do it!

If you just created your first Assistant after signing up, you should automatically be redirected to the app store.

If not, whenever you want to add apps to one of your Assistant, simply select your Assistant from the Console’s home page. In this example, click on ‘Smart House’.

And then click on ‘Add an app’

You will land on the app store. The app store lists all the apps that have been published by users (including Snips) for everyone to add to their Voice Assistant.