What Do I Build Next? A TV and Electronics Command Console

After working on updates to my story on 2/22/19 for 2hours plus at 9:30pm my "story, content and pictures have disappeared....and must be re-entered" even though "Autosave was on"....very disappointed with Hackster's server tonight.

What Do I Build Next? A TV and Electronics command console using SNIPS_AI and Seeed Studio Voice Interaction Development Kits.

My project submission: January 26, 2019 at 12:25PM PT

What problem are you going to solve?

I am a geek with many remote controls in livingroom. TV, 2 BlueRay players, Bluetooth Soundbar, Xfinity/FiOS cable box, VHS player.... you name it...i have it. It's Not good to have clutter in the Livingroom so my wife says. SO, to hide those components away and remotes then just speak "Hey_Snips, power On...." and I'm in control.

What are you going to build to solve this problem? How is it different from existing solutions? Why is it useful?

I plan on replacing the single relay switch in this kit with Seeeds 8 relay switch and I'll be on my way. This will allow me to hide the many power bricks and multi-outlet in the wall cavity and hang the Sony TV on the wall. No more livingroom clutter to see, this Snips_AI panel could be hidden behind TV as well... "out of sight, out of mind....makes wife happy...and I'll be happier because I still have my TECH...

How does your solution work? What are the main features? Please specify how you will use the Voice Interaction Base Kit OR Voice Interaction Satellite Kit in your solution.

I'll be using the 4 or 8 relay switch from Seeed to individually control power to each of my hidden components. I'll train "Snips_AI" to power on each component via voice command instead of the remotes.

List the hardware and software you will use to build this.

Seeeds 8 relay I2C module, this Snips_AI base and satellite kit. I HAVE many Raspberry PI flavours on hand to chose from and boxes of components from Adafruit, Amazon, Seeed, DFRobots, etc to build with and prototype before this project moves to full speed. I attended the Snips DevDay online on 1/22/19 so I have a basis and example to build upon. I also, most importantly have the Internet and the eagerness to learn.... and fail...and figure it out. Now to get both kits to start... voicing my commands.

Does your project deliver an interactive user experience using voice?

Yes, it'll repeat and display what it hears via console or display and speaker

Would you like a free Base Kit or Satellite Kit? NOTE: Satellite Kits require a Base Kit or Raspberry Pi 3 B+ in order to operate as intended.

I want the base kit but also need the satellite kit to do complete idea, so I'll be ordering 8 I2C relay from Seeed and additional items for possible lights control as well. This week 1/26/19.

On January 26th, I placed my order with Seeed Studio for both Voice Interaction Development Kits and two Grove- 4-channel SPDT Relay modules. My out of pocket cost is $240.00 including shipping. The Seeed Studio website was very unresponsive due to upgrade issues. Due to the recent Chinese New Year Holiday's and out of stock relay modules, my order was shipped until after February 18th. On February, 21, my order was delivered.

I had tried to 2nd source these items through Amazon and EBAY for faster delivery, but since they are from Seeed Studio I couldn't locate the relays cheaper or faster delivery.

The Project Kit information: (content courtesy of Seeed Studios)

Base Kit:

Description Below from Seeed Studios website:

The Snips Voice Interaction Development Kits empowers users to jump start their development of voice interfaces and easily integrate the Snips AI Voice Platform with hardware. Snips runs on device, meaning all of your voice commands and data are processed locally rather than being sent to the cloud. This on-device processing results in a completely secure offline voice assistant that keeps your personal data private, as it should be.

The Voice Interaction Base Kit allows you to run the entire Snips AI Voice Platform. Powered by a Raspberry Pi 3 Model B+ and the ReSpeaker 2-Mics Pi HAT, it is capable of acting as your home smart speaker. Along with the kit comes a speaker, a Grove - Temperature & Humidity Sensor (SHT31), a Grove - Relay, and a pegboard to hang it on a wall or create a nifty stand.

But that's not all! The Voice Interaction Satellite Kit will extend your base station to reach each room of your house and allow you to interact with the hardware based on where you issue your commands!

Raspberry Pi3 B+, ReSpeaker 2-Mics Pi HAT, Grove – Relay, Grove - Temperature&Humidity Sensor (SHT31), Speaker 6Ohm 2W, 16GB MicroSD Card, Power adapter with Micro USB connector, 2 Grove cables, Acrylic Base panel, Acrylic Protective cover for Relay, Acrylic Table stand

Assembly components

6mm, M2 full thread black hex nylon spacer, 3 x 12mm, M2 full thread black hex nylon spacer, 5 x 25mm, M2 full thread black hex nylon spacer, 3 x 6mm, M2.5 full thread black hex nylon spacer, 4 x 5mm, M2 carbon steel phillips screw, 22 x 5mm, M2.5 nylon phillips screw, 8 x 10mm, M3 carbon steel phillips screw, 4 x 6mm, M2 full thread black hex nylon spacer, 3 x Blue rope, 1 x Screw driver

Satellite Kit:

* All Satellite Kits require a Base Kit or Raspberry Pi 3 B+ in order to operate as intended.

The Voice Interaction Satellite Kit can extend the reach of your base station to each room in your house and enable you to interact with the hardware based on where you issue your commands! You can arrange multiple Satellite Kits throughout your home to add new functionality to the Base Kit or any other smart speaker, extending your voice control across several rooms.

Raspberry Pi Zero Wireless, ReSpeaker 2-Mics Pi HAT, Grove – Relay, Grove - Temperature&Humidity Sensor (SHT31), Speaker 6Ohm 2W, 16GB MicroSD Card, Power adapter with Micro USB connector, 2 Grove cables, Acrylic Base panel, Acrylic Protective cover for Relay, Acrylic Table stand.

Assembly components

6mm, M2 full thread black hex nylon spacer, 3 x 12mm, M2 full thread black hex nylon spacer, 5 x 25mm, M2 full thread black hex nylon spacer, 3 x 6mm, M2.5 full thread black hex nylon spacer, 4 x 5mm, M2 carbon steel phillips screw, 22 x 5mm, M2.5 nylon phillips screw, 8 x 10mm, M3 carbon steel phillips screw, 4 x 6mm, M2 full thread black hex nylon spacer, Blue rope, Screw driver

additional KIT link: https://docs.snips.ai/the-maker-kit/dev-kit

Grove-4-Channel SPDT Relay Modules:

The Grove - 4-Channel SPDT Relay has four single pole - double throw (SPDT) switches. It only requires low-voltage and low current signals to control those switches. Specifically, you can use 5V DC to control max.250V AC or 110V DC.

We use an on-board STM32F030F4P6 to control the channels separately. The control command is transmit via the I2C interface, the on-board STM32F030F4P6 will parse the command, so that you can control the switch you want.

Features

• High temperature resistant plastic shell
• High voltage load
• Low power consumption
• Long lasting
• 128 Optional I2C address

Applications

• Domestic appliance
• office machine
• Remote control TV receiver
• monitor display
• audio equipment high rushing current use application

Pin Map

additional wiki link: http://wiki.seeedstudio.com/Grove-Relay/

ReSpeaker 2-Mics Pi HAT:

additional ReSpeaker_2_Mics_Pi_Hat Link: http://wiki.seeedstudio.com/ReSpeaker_2_Mics_Pi_HAT/

My Project Build Notes:

While I waited for delivery, as usual I researched, gathered content from existing Projects that included Grove sensors, SNIPS_AI, Voice Interaction, etc. I have many Grove sensors and hardware for the Raspberry Pi, Arduino, Linksprite, Dexter, and Intel boards "future What Do I Build Next? Projects"...

I am lucky that I recently attended via the "Virtual Classroom" a Hackster.IO event "Dev Day 2019" on January 22, 2019. It featured the Raspberry Pi and the Matrix Creator Development Board. Being the Geek that I am, I had purchased a Matrix many months ago so I had some exposure "early on in its growing pains" with limited documentation and many crashes. This project appeared to me as a big help in the research and development for this project as both "concepts" used voice commands, SNIPS_AI, and the Raspberry Pi. There was also a good lab manual and online documents available. Content notes very good for setting up SNIPS_AI.

( https://events.hackster.io/armdevday?mkt_tok=eyJpIjoiT1RBek9UWXlOVEJqWVdFMiIsInQiOiJpQnhmVjE3Q3JMb3NpdFZONkdGK0VSNm56UjNvc0tua1RrMyt2RnVqamtjY01pVnFlVDNRalhhNFJEaTNoUGxnMm9QbUpNNzdEWXArQ3hScERibitNeEozK2VZVVcyNzhTZTBpK0JwYlFCOUlIMW1RVEFKN3AzcTBaMlwvUVwvUjE3In0%3D )

( https://github.com/matrix-io/Arm-DevDay-2019?utm_source=arm_devday+registrants&utm_campaign=78c3d18b25-EMAIL_CAMPAIGN_2019_01_19_02_03&utm_medium=email&utm_term=0_59b6d13c9d-78c3d18b25-143445241&mc_cid=78c3d18b25&mc_eid=f27766b829 )

Now for my stuff and experience:

On February 21, my Kit from Seeed Studio was "delivered"... shame on you DHL!!!

Thankfully, contents were well wrapped.

Base Kit Unboxing: familiarize yourself with the kit, inspect for damage

Assembly: Base Kit (Remove protective and annoying film from acrylic plates) those of you who read me previous project notes...know I hate doing this!

Make a backup of included SD Card (corruption happens often):

The included instructions are lacking details. There are two bags of hardware. One bag has short standoffs for the Raspberry Pi mounting. The other bag has hardware to mount the sensors and speakers. There are 3 different sized standoffs but the picture showing location/screw holes and placement "count the holes" does not indicate which sized standoff is used where. ( a little clue, the longest ones are for the Relay shock protection cover, the others are for the sensors and speaker) There will be extra "tiny" screws.

Assembly of the Satellite Kit: (same as base kit except for standoff locations for Raspberry PI W... we're on our own here)

Don't forget, backup the original SD card.

Base and Satellite Complete: Now to read further about proper 1st boot and software setup, localization, and wifi.

To be continued...2/22/19 11:51pm completed 2nd story write up tonight. Almost 5 hours to write this up 2x due to Hackster.IO server issue? Tech support would have attempted to recover later, but I was impatient tonight.t

Well, yesterday was February 28th. The Hardware winner's were t sensor...congrats to all of you who won. I didn't, but I will persist and spend more of my money ($300+USD and counting).

I have learned abit with these kits. The Pi Desktop is not loaded as well and searching the SD card directory structure via the terminal when you don't know where to go is not for the meek.

The Voice/Demo assistant files were loaded from a locally stored file/directory. My initial attempt(s) to find and edit the loaded Assistant on the Snip-sat unit resulted in a non-working kit. I have been fustrated by the lack of documentation on the creation and setup up of these kits and have them work together.

My goal was to connect a Grove- I2C hub in place of the Grove- Temp/RH sensor and actually reconnect that Respeaker port, leaving three available. One would be used for my Grove-4-channel SPDT relay also purchased with my kits. I already had the I2C hubs from previous purchases. I also had many Grove- starter kit sensors to experiment with.

Learning #1: Install pip, npm, I2C-tools, smbus, spidev, and many dependencies (the SD card included does not have a complete Raspbian install) inorder to verify that each kit can "see/talk" to the Grove items being incorporated, but how? (Note this may not be correct order of entry)

"sudo apt-get install pip"

"sudo apt-get install npm"

"sudo pip install spidev"

"sudo apt-get install i2c-tools"

"sudo apt-get install python-smbus"

"sudo npm install -g snips-sam" (needed to talk via terminal between base and satellite kits)

"sudo pip install rpi.gpio"

"sudo apt-get install python3-dev"

*this is just a start**

Learning #2: Python code examples and documentation are sparse. I am trying to use examples from the "DexterIndustry GrovePiZero and GrovePi+ modules" that I have. Unfortunately, the DexterIndustries GrovePiZero software install procedure also points to the use of their own customized "Robot" SD image and the use of the GrovePiZero, so I installed that beneath the Seeed ReSpeaker module. The software didn't install completely but it did clone the repository with code examples onto the SNIPS-SAT SD card.

Learning #3: Where is the documentation? How was this SD card created? Back in 2014, when I got my 1st of many Raspberry Pi's, I discovered that on the SD card images that I was downloading and using and developing with...was a record of "keystrokes and typed in entries".... after booting your Pi up and wait for the prompt to appear. Press the "Down Arrow" and your last typed entry will appear... guess what? keep this up and you'll come across entries that are not your's. I logged over 120 entries before my initial ones....scary!

This project has gone into Internet Research and email mode for almost three weeks. I fear I am over complexing my Control Center Project.... I want besides the basic included TEMP/RH and single Relay; I want to add the 4 Channel Relay for more devices, a PIR-Motion sensor to detect movement in room ( https://www.seeedstudio.com/Grove-PIR-Motion-Sensor-p-802.html ) and Light sensor to detect DAY or Night conditions ( http://wiki.seeedstudio.com/Grove-Digital_Light_Sensor/ ) and a PiCamera ( https://www.amazon.com/Raspberry-Camera-Module-OV5647-Sensor/dp/B078Y63QNG/ref=asc_df_B078Y63QNG/?tag=hyprod-20&linkCode=df0&hvadid=241967399507&hvpos=1o3&hvnetw=g&hvrand=12359166303890286292&hvpone=&hvptwo=&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=9001771&hvtargid=pla-447902284057&psc=1 ) or Grove-Serial Camera ( https://www.seeedstudio.com/Grove-Serial-Camera-p-945.html/ ) or ( http://wiki.seeedstudio.com/Grove-Serial_Camera/ ) for Facial Recognition/User access.

Today is March 16th, recently "hardware awards" were announced.... I wasn't chosen, but I had already invested $200.00 plus (Yes, no sponsors or freebees here, My money!!! I broke my own rule about contests... and judging). Don't invest more money than you want to lose or you'll be stuck with hardware that'll collect dust.

Back to my project. Research and tech support emails to Seeed Studios have been a daily occurance since I recieved my kits and actually as soon as I signed up for this contest. Once again, the lack of complete code examples for the Raspberry Pi using python was lacking. Even the use of SNIPS-AI ws a bit overwhelming to digest.

I learned in the 1st few hours of testing that the "assistants" online once downloaded overwrote "Demo code" that came installed on the included SD cards (thus the importance of making a copy of the SD card before doing anything or at least after "user localization and basic PI setup" was complete. I did both.

Secondly, the online "SNIPS-AI core" assistant examples need further customization and understanding of use with the few "exposed GPIO and I2C" ports. The incliuded RELAY is hardcoded to GPIO pin 12, GPIO pin 13 is present at the connector but the auxillary cable must be modified via a break-out cable. There is only one I2C connector onboard and the TEMP/RH sensor is tied to that. I had seen that in the beginning and planned to use a GROVE-I2C hub to add additional GROVE-sensors.

I know very little about I2C except that many sensors can be daisy-chained on the buss and only requires the two communication wires "SDA and SCL" and the address of the device ( https://learn.sparkfun.com/tutorials/i2c/all ) and ( https://learn.sparkfun.com/tutorials/raspberry-pi-spi-and-i2c-tutorial ) are some basic guides.

Thirdly, test the included hardware and software before making any changes. Although, I had both the "SNIPS-BASE and SNIPS-SAT" kits up and running; and seemingly working. The ReSpeaker board, microphone, speakers and SNIPS was listening, processing, and answering the baic questions about "Hey Snips...What is the Temperature or Humidity and even Turn ON or OFF the relay", I found that my units were responding together. I lacked local environment control.

My commands for the ON/OFF were echo'd to both units, thus if I said "Hey Snips... turn ON/OFF light in livingroom where the Snips-Base unit was, the fan in the bedroom where the Snips-Sat unit was would also turn ON/OFF.

I also found that my Snips-Sat unit had a faulty relay. The "activity LED" would illuminate dimly but not energize the relay. I was initially mis-led by this LED. I thaught my "test 3V DC fan" was defective when it wouldn't function. There is no feedback to the SNIPS module that the action failed.

I was lucky, being that I had multiple GROVE-KITS, in my stockpile to swap-out and continue testing. I had to submit a video(s) to SEEED Studios tech support before they would acknowledge the component failure.

I also discovered that my GROVE-4 Channel SPDT relay modules case had a laser cutting defect (a missing slot in the cover) and the complete assembly for the enclosure was not possible. The SEEED studio Quality Dept. verified their stock and this was overlooked by their auditors as well. New covers would be sent?...... with the failed GROVE-Relay and two other items that I decided I needed ( a GROVE-Camera and a GROVEPiZero module thus I saved $25.00 shipping).

The GROVEPiZero ( https://www.dexterindustries.com/product/grovepizero/ ) or ( https://www.seeedstudio.com/GrovePi-Zero-GrovePi0-p-3011.html ) or GROVE BASE hub ( https://www.seeedstudio.com/Grove-Base-Hat-for-Raspberry-Pi-Zero-p-3187.html ) or ( https://www.seeedstudio.com/Grove-Base-Hat-for-Raspberry-Pi-p-3186.html ) " Tech support indicates that these Grove-Base-Hats are 3.3v compliant so 5v sensors won't work. I had planned to add in between the Raspberry Pi (s) and the ReSpeaker to gain additional i2C, GPIO, Analog and Digital ports. I later found that loading the required drivers/software/libaries did not play well with the "base stretch-lite build" SD image on both kits. Thus I would need further development and documentation on the existing build and its configuration. I would also need to swap out the RaspberryPi W from the SNIPS-Sat unit with a Raspberry Pi 3B+ (or equivalent full RPi) due to processing requirements.

Again, I have hardware available. I had GROVE-Shields for the Arduino, Edison, Galileo, and Raspberry Pi on hand. Although, I could not use any of them, even the GROVEPI+ that was for the Raspberry Pi was not usable as it only had 26 pins and the ReSpeaker required a 40 pin Raspberry Pi. I also had one GROVEPiZero available so I only needed to buy one from SEEED. Amazon, DexterIndustries, and EBAY had them at a permium and with unrealistic shipping or delivery dates. The stock for sale, was actually at SEEED studios in China. So recent WORLD events made for long delivery times.

But, I can move on with development... ya if I could find code examples for the GROVE-4 Channel Relay modules that I bought. No i2C or python code examples are available from SEEED. Emailed Support for my project and hardware questions are slow. Basically, SEEED is leading the horse (me) to the water though (sparse forum and generic code) then it's up to me to drink (develop my own code) from it.

So, after "updating" Raspbian Stretch-Lite to Raspbian Stretch desktop with Pixel "and hopefully ALL features, I have been adding additional libraries for my hardware without "bricking the SNIPS-SAT" SD image. I even went so far as to build a custom image based upon Raspbian FULL Stretch with Desktop with my extra libraries for the GROVEPiZero and GROVE-sensors (Dexter Industries) but have not installed SNIPS-assistants or requirements yet.

I want to develop "local" control examples before adding SNIPS-AI components. I have been searching through similar devices and code examples for something usable or adaptable. I have had partial success with locally run python code examples.

Today is March 27th, Seeed has not sent the replacement Grove Relay for my Satellite station. I have made decision to move GOOD relay to satellite station so I can continue development endeavors. I have also created a revised satellite station SD image using a full Raspbian build using "Dextor Industries Raspbian for Robots" image as I am trying to incorporate additional ports using the GrovePiZero module sandwiched between the Raspberry Pi 3B+ and the ReSpeaker board. I felt that I needed CPU power that the Raspberry Pi3B+ had to utilize the GrovePiZero as well as the Raspberry Pi Camera.

I am waiting for additional items from Seeed Studio as well..."my GROVE SERIAL Camera, another GROVEPIZero, the Covers for my "still unworking 4 channel SPDT i2c relays", and the replacement GROVE Relay. so my project submission is changing its scope...

I also wanted to try Facial Recognition using OpenCV or mmal-Motion to recognize me and take a picture or announce my name or ask me a question like "Hi Jim, what would you like?" then I could ask "Hey Snips... what is the temperature or humidity or turn ON something"....but again one of these video/facial streaming apps actually "mucked" up by Snips-SAT sd card...again...patience I said.

Three weeks left.....'till contest end.

But, now comes next hurdle. I wanted to use a PowerTail switch or an Adafruit IOT Power Switch. Each of these would turn OFF/ON any AC device. I soon learned that these require a DC voltage signal of at least 3.3 vdc and not just a contact closure that the GROVE Relay (included in these kits provided). I wasn't going to connect 120 VAC to this untested GROVE RELAY kit. How could I do this control safely?....

I chose to add a 7.4 vdc battery pack (from a nearby robot) and use the GROVE Relay as a switch and that would power my Adafruit and Power Tail Power Switches on and off.

I was unsuccessful in getting an i2c IR or bluetooth module to control any of my many electronic components. I have everything tied to my XFINITY X1 remote control. Its not IR exclusively...its bluetooth as well...Alot more to decipher and code than allotted for this contest.

So moving along, I wanted to attempt the Facial recognition aspect of this project. I had a mock up Snips-SAT station with the GrovePiZero, ReSpeaker, and a custom SD image built upon the "Raspbian for Robots" SD image. This SD image gave me a Raspbian Desktop with access to the GrovePiZero library and other code examples that worked locally when connected to the GrovePiZero.

I must first find and incorporate a "Facial Recognition example". I first found and installed a project based upon "OPENCV" and "motion" and "mmal-motion"... but wiped out my Dexter desktop and ultimately access to all code on the desktop...

time to start over.... a new SD image "Raspbian for Robots"...etc...

I later chose this one " https://robotzero.one/face-recognition-party-greeter-raspberry-pi/" and utilized " https://gist.github.com/ageitgey/1ac8dbe8572f3f533df6269dab35df65" build steps and code to get me going....