- Imagine you are busy soldering something at your table and you need to turn your solder iron OFF/ the lights ON.
- Lets assume that's a basic solder you have. Also, you need to quickly know where you have placed the 220 ohm resistor. Let's assume you have not labelled your inventory.
This is where the Smart Table Assistant come into picture. It helps to manage inventory information and control power to appliances hands-free and through voice. You can ask the assistant to turn off the solder and ask for the place where you have placed the resistor (provided you have not changed it from the place where you had told the assistant to remember)!
IntroductionA voice-powered smart table assistant for makers and hackers to help manage inventory information and control power to appliances like a table lamp, solder iron and more, when their hands are busy developing. Being a developer myself, I am aware of the difficulty of managing inventory items and that too, when you are developing at a rapid pace. In such situations imagine an assistant to whom you can tell,
"Hey, remember that I have taken 220 ohm resistor and a 4 channel 5V DC relay module"
and you can keep adding items to the list whenever you take them from your inventory.
At the end of the session, the assistant will have the list of the items you have taken out and will remind you to place them back at their respective places. Say,
"John you have taken out these items and they are on your table. You need to place the 220 ohm resistor in the white box inside your second table drawer and so on".
Also imagine a scenario where you are soldering something and without taking your hands off, you need to turn ON/ OFF a solder iron you are using or a table lamp. This is one more scenario where the smart table assistant can come in handy and useful when you are the only person available to both develop your project and need to record a video of what you are making. You can say to the assistant,
"Hey, turn the focus lights off"
or
"Hey, turn the monitor* on"
*monitor - may be used for the purpose of showing the camera feed or to display the dialogue script.
It is useful for makers and hackers who are all by their own and who need some extra pair of imaginary hands to help!
PlanAs explained earlier the main features are
- to help makers and hackers manage their inventory and
- to control appliances through voice
when both their hands are busy building something.
The solution will have the Voice Interaction Base Kit as the central brain of the system. User will ask it what he/she needs, to the kit and based on the voice command received it will take the respective action tagged under it.
For controlling appliances it will instruct the wirelessly connected smart relay system to turn on/ off the appliance with respect to the voice commands received. This system consists of the following components,
- Arduino MKR1000 - to receive POST requests from IFTTT through thinger.io and then based on it, instruct the serially connected Arduino UNO to turn ON/ OFF relay respective to the voice command.
- Arduino UNO - it receives serial information from Arduino MKR1000 to turn ON/ OFF relay respective to the voice command.
- Relay Module - it receives signals to close or open the AC power circuit connected to it.
For smart inventory management, the kit will receive information on where the items are placed. This will be stored in Google Sheets which will act as a database which can be retrieved or altered when the users requests so. IFTTT can act as a bridge between the voice interaction base kit and Google Sheets.
Unboxing, Assembling and OOB Demo of the Voice Interaction Base KitIn the video below, I'll show the unboxing of the voice interaction base kit and a timelapse of how to assemble the parts of the kit together. After which I'll show the out of the box (OOB) demo of the kit which has the below conversation,
[Me] - "Hey Snips, what's the currenttemperature?"
[Assistant] - "The current temperature is 31.2 degree"
[Me] - "Hey Snips, what's the humidity?"
[Assistant] - "The current humidity is 66.7 percent"
[Me]
- "Hey Snips, turn on the relay"
[Assistant] -
***Plays a sound and turns the relay on but this is not shown in the video!***
[Me] - "Hey Snips, turn off the relay"
[Assistant] -
***Plays a sound and turns the relay off but this is not shown in the video!***
For a more detailed explanation and code of the out of the box demo project - Temperature & Relay, please visit the snips official github repository link - https://github.com/snipsco/snips-demo-dev-kit
The Grove Relay is a 5V DC relay which will turn ON when we instruct it through voice and it will be indicated by a Red LED (D3). However, it receives, only 3.3V from the Raspberry Pi 3B+ and hence the indicator Red LED will not glow brightly and also it will not properly close/ open the AC circuit connected to it. Therefore connect it to a 5V signal to make it to work.
Product Page - http://wiki.seeedstudio.com/Grove-Relay/
Smart RelayThe process goes like this when you instruct Snips, the instructions are understood by Snips and it calls out the respective applet in IFTTT. Then IFTTT triggers the respective action for the obtained input from Snips. The action would be to pass a POST request to thinger.io with JSON body as below.
{
"in": {
"state": 1
}
}
This has the input variable 'state' passed to the Arduino MKR1000 through its API in thinger.io. Based on the state passed Arduino MKR1000 serially communicates the corresponding relay to be turned ON/ OFF to the Arduino UNOconnected to the Relay module. For instance, 1 - Turns the Relay1 ON and 2 - Turns OFF the Relay1, 3 & 4 for the next relay ON & OFF and 5 & 6 for the third relay. You can connect your home appliances to the extension box and control using voice commands for the respective relays.
I have modeled and 3D printed a case for the Smart Relay system, find below the gallery of images of the finished case containing the system.
Below is the Smart Relay System in action without Snips.ai.
This is a challenge where I have learned so much in a little span of time. I was recently introduced to the snips ai platform through matrix creator. Immediately after which I was selected in this competition for free hardware kit and it was so much fun with the voice interaction base kit. Thanks once again for sponsoring! This is also when I got introduced to 3D modeling and printing. So I was able to 3D print a case and light bulb holder for the smart relay system. However the one part which put me back and where I struggled the most was the code actions and this error,
The session was ended because one of the component didn't respond in a timely manner
There are so many discussions for this error in the Snips forum - https://forum.snips.ai/search?q=The%20session%20was%20ended%20because%20one%20of%20the%20component%20didn%27t%20respond%20in%20a%20timely%20manner
and few useful documentations regarding code actions (official and user blogs) - https://docs.snips.ai/additional-resources/troubleshooting and https://www.smb-tec.com/blog/5-quick-tips-about-snips-skill-actions
However I could not resolve it and progress further. If I had found a video demo or programming documentation of code actions end to end implementation and basics (which is not available atleast in English), this would have been complete by now. But still I will continue to build and try to complete this project and blog here. Stay tuned!
Useful Links1. Challenge page - https://www.hackster.io/contests/building-for-voice
2. Snips console - https://console.snips.ai/
3. Snips documentation - https://docs.snips.ai/the-maker-kit/dev-kit
4. Snips forum - https://forum.snips.ai/
5. Voice Interaction Base Kit product page - http://wiki.seeedstudio.com/Snips_Voice_Interaction_Base_Kit/
6. Snips quick start YouTube playlist - https://www.youtube.com/playlist?list=PLmOf6ALFW9aY_EZOYtTM27TqKSasHqZr_
7. Code action reference - https://www.smb-tec.com/blog/5-quick-tips-about-snips-skill-actions
8. Troubleshooting - https://docs.snips.ai/additional-resources/troubleshooting
9. Snips GitHub page - https://github.com/snipsco
Comments