DIY Smart Home Assistant with Raspberry Pi and ReSpeaker

ReSpeaker Mic Array, as the “ear” of Raspberry Pi here, can listen to your speech commands and send them to Raspberry Pi. According to your commands, Raspberry Pi will control Wio Link to do what you want via Wi-Fi.

We have talked the smart home for so many years, but our home is still not so smart enough. And this time, it is really coming, Amazon Echo, Google Home, Apple homekit and so on.

Step 1: What We Need

Raspberry Pi(with network connected) *1

ReSpeaker Mic Array with audio jack *1

SD card with Raspberry Pi image *1

Wio Link with WS2812 LED Strip *1

USB cable *2

PC or Mac

Step 2: Download Our GitHub

First, you should ssh to Raspberry Pi, and download our Github for ReSpeaker Mic Array:

git clone https://github.com/respeaker/mic_array.git

Step 3: Test the Mic Array

1. Run pixel_ring.py to control the pixel ring of the Mic Array through USB HID. After running this, ReSpeaker Mic Array will start to blink.

sudo pip install pyusbsudo python pixel_ring.py

If you don't want to access USB device with sudo, add a udev .rules file to /etc/udev/rules.d:

echo 'SUBSYSTEM=="usb", MODE="0666"' | sudo tee -a /etc/udev/rules.d/60-usb.rules
sudo udevadm control -R  # then re-plug the usb device 

2. Read 8 channels audio from the Mic Array and estimate sound's DOA. Then the LEDs on Mic Array will show the direction of the sounds.

sudo apt-get install python-numpy    # or pip install numpypythonmic_array.py 

3. Do VAD and then estimate DOA. Then the LEDs on Mic Array will show the direction of the sounds, too.

sudo pip install webrtcvad
python vad_doa.py 

4. Do KWS and then estimate DOA.

Get snowboy work and run python kws_doa.py. Then you should say “alexa” to Mic Array to wake it up, if you sound is detected, the LEDs will show the direction of the sounds.

git submodule init
git submodule update
cd snowboy/swig/Python
sudo apt-get install python-dev libatlas-base-dev swig           # requiremetns to compile snowboy
echo 'from snowboydetect import *' >__init__.py     # create__init__.pyfor a python module
cd ../../..                                          # chang to the root directory of the repository
ln -s snowboy/swig/Python snowboydetect
python kws_doa.py

Step 4: Install ReSpeaker Python Library

Install ReSpeaker Python Library to use Bing Speech API

git clone https://github.com/respeaker/respeaker_python_library.git
cd respeaker_python_library python setup.py install 

And you need a Bing key, how to get it?

Step 5: Setup Your Wio Link

Add WS2812 LED Strip to the digital0 port of Wio Link and update the firmware. Then click"view api"and get the access token.

If you are new to Wio Link, please click here for wiki and guide.

Step 6: Add Bing Key and Wio Token to Python Code

Save the code below in your Raspberry Pi, and don't forget to fill in Bing key and Wio token:

from respeaker.bing_speech_api import BingSpeechAPI as Bingimport wave
from mic_array import MicArray
import Queue
from pixel_ring import pixel_ring
import sys
import numpy as np
import collections
from snowboydetect import SnowboyDetect
import time
import json
from urllib import urlencode
from urllib2 import Request, urlopen, URLError, HTTPError# write your Wio token hereWIO_TOKEN = "**************"# write your Bing key hereKEY = "**********"
bing = Bing(key=KEY)RATE = 16000
CHANNELS = 8
KWS_FRAMES = 10     # ms
DOA_FRAMES = 800    # msdetector = SnowboyDetect('snowboy/resources/common.res', 'snowboy/resources/snowboy.umdl')
detector.SetAudioGain(1)
detector.SetSensitivity('0.5')# about 5seconds
q = Queue.Queue(maxsize=768)def gen_queue(q):
    try:
        data = q.get(timeout=1)
        while data:
            yield data
            data = q.get(timeout=1)
    except Queue.Empty:
        passdef controlLED(onoff=0): 
    try:
        if onoff == 1:
            rgb_hex_string = '000080'
        else:
            rgb_hex_string = '000000'
        url = 'https://cn.wio.seeed.io/v1/node/GroveLedWs2812D0/clear/4/{}?access_token={}'.format(rgb_hex_string, WIO_TOKEN)
        request = Request(url, data='')
        response = urlopen(request)
        data = response.read()
        result = json.loads(data)
        if result['result'] == 'ok':
            return True
        else:
            return False
    except Exception as err:
        return Falsedef main():
    history = collections.deque(maxlen=int(DOA_FRAMES / KWS_FRAMES))
    global q    try:
        with MicArray(RATE, CHANNELS, RATE * KWS_FRAMES / 1000)  as mic:
            for chunk in mic.read_chunks():
                history.append(chunk)
                # Detect keyword from channel 0
                ans = detector.RunDetection(chunk[0::CHANNELS].tostring())
                if ans > 0:
                    print("wake up")
                    print("start recording")
                    pixel_ring.arc(12)
                    q.queue.clear()
                    for chunk in mic.read_chunks():
                        q.put(chunk[0::CHANNELS].tostring())
                        if q.full():
                            break
                    print "queue full"
                    pixel_ring.spin()
                    text = bing.recognize(gen_queue(q))   # data can be generator
                    if text:
                        print('{}'.format(text))
                        if 'turn on' in text:
                            controlLED(1)
                        if 'turn off' in text:
                            controlLED(0)
                    pixel_ring.off()    except KeyboardInterrupt:
        pass    pixel_ring.off()
    # except ValueError:
    #     passif __name__ == '__main__':
    main()

Step 7: Run the Demo

Save the Step6 python code in smart_home.py, then run it:

python smart_home.py

Wake it up with saying "snowboy", then say "please turn on the light", or "please turn off the light" to control Wio Link.

Enjoy it!!