Ghost Pepper voice assistant for HA

This is an offline visual voice assistant that connects via MQTT to any domotica system that supports MQTT. I use it to control my Home Assistant system. With over 90 animations and 150 possible commands it will suite most tasks.

Will there are more capable voice assistant those are not working offline and aren't that visual appealing. To overcome this I decided to make my own voice assistant. May I present you this Ghost Pepper voice Assistant.

How does it work?

The heart of this device is a ESP32 and a AI Thinker VC-02 voice and speech module. After a wake word the voice module signals the ESP with a code and that will start an animation. Based on the voice command the animations changes till you are finished. The ESP sends out the selected command via MQTT to Home Assistant (HA) in my case. There an automation will start based on the received code and additional commands are send back to start a different animation. The animations are loaded from a SD card and displayed on a screen. This image is reflected on a transparent sheet of plastic, hence the Ghost pepper in the name. There is an additional touch button that can signal HA without activating the voice assistant.

Everything is packed inside a resin printed case. The size of the housing depended on the size of the drinking glass that I could find in the thrift store and also by the size of all the other components.

All the parts for this housing are available online in Tinkercad.com. Just do a search on Ghost Pepper HA assistant to find them. I made them publicly available.

I've include a zip file containing the Arduino project folder with all the tools and code needed. Just unzip the folder in one place and you are good to go. Make sure you look at the readme file because it will contain important details to work with and how to program the VC-02 module. The config file will show you how to connect the different parts together.

See it in action below

Here are also a few examples of yaml files used in HA

alias: GP assistant Wake up
description: ""
trigger:
  - platform: mqtt
    topic: az_assistant/status
    payload: "48"
condition: []
action:
  - service: mqtt.publish
    data:
      qos: 0
      retain: false
      topic: az_assistant/command
      payload: "70"
mode: single

---------------------------------------

aias: GP assistant Dance
description: Preform a random selected dance
trigger:
  - platform: mqtt
    topic: az_assistant/status
    payload: "170"
condition: []
action:
  - service: mqtt.publish
    data:
      qos: 0
      retain: false
      topic: az_assistant/command
      payload_template: "{{ states('sensor.gp_random_dance') }}"
mode: single

---------------------------------------

alias: GP assistant exit
description: Preform a random selected exit action
trigger:
  - platform: mqtt
    topic: az_assistant/status
    payload: "173"
condition: []
action:
  - service: mqtt.publish
    data:
      qos: 0
      retain: false
      topic: az_assistant/command
      payload_template: "{{ states('sensor.gp_random_exit') }}"
  - delay:
      hours: 0
      minutes: 0
      seconds: 2
      milliseconds: 0
  - service: mqtt.publish
    data:
      qos: 0
      retain: false
      topic: az_assistant/command
      payload: "0"
mode: single