A New Choice in Voice

The smart speaker market experienced a remarkable rise over the past decade, with these devices promising to forever change the way people interact with technology and their living spaces. The journey began in 2014 with the launch of Amazon's Echo, powered by the voice assistant Alexa. Its success sparked a wave of competition, leading to the introduction of Google Home in 2016 and later, Apple's HomePod in 2018. These smart speakers quickly gained popularity due to their seamless integration with smart home devices, voice-controlled functionalities, and the ability to provide personalized information and entertainment to users.

However, after this impressive surge, the smart speaker market began to stall. This slowdown in sales can be attributed to many factors, like the privacy and security concerns that come along with an always-listening device in the home. But perhaps the biggest factor was the lack of compelling use cases beyond basic functionalities. While smart speakers proved useful in managing certain tasks, like controlling lighting and playing music, they struggled to justify their cost for some consumers who saw them as a novelty rather than a necessity. As a result, the market failed to tap into more diverse consumer segments, limiting its growth potential.

Even still, we have seen Star Trek and pine for the types of voice interactions that are portrayed with those computers of the future, so we know that compelling use cases do exist. Not wanting to wait for the perfect product to be commercially produced, engineer Justin Alvey took matters into his own hands and built a custom smart speaker. And it is not just customized to suit Alvey’s tastes, rather it is an open platform that can be programmed with the Arduino IDE.

As any hobbyist knows, hacking together something that gets the job done and building a polished device with style are two completely different things. So, laughing in the face of a voided warranty, Alvey began with a Google Nest Mini as the starting point. This provides the fancy casing, speaker, microphone array, LEDs, capacitive touch sensors, and more. But the Nest Mini is not meant to be modified by the user, so the PCB was removed to make way for a new one.

Alvey designed a new PCB, this one populated with an Espressif ESP32-S3 microcontroller. This chip can be easily programmed with custom software using simple tools, and it has a Wi-Fi radio to enable wireless communications. As such, the reengineered smart speaker can communicate with more powerful computers on your local network that run machine learning algorithms, or for that matter, any other software that you want to interact with. With the control that this hack gives you, the options are virtually endless.

To power the device, Alvey created his own Maubot assistant bot that was integrated with his Beeper account to retrieve personal information. Beeper aggregates conversations from a variety of messaging apps in an encrypted manner using the Matrix communications protocol. A GPT-3.5 model was also leveraged for general conversational and question answering capabilities. The responses are passed to a text-to-speech service, then the audio is played through the Nest Mini’s speaker.

At present, Alvey is working on open-sourcing the PCB design and putting together some build instructions for those that want to create their own custom smart speaker without doing all the hard work of reverse engineering the Nest Mini. He is also planning to release the firmware and source code, so stay tuned for updates.

As a side note, if you want to avoid privacy violations but do not want to gut your smart speaker, check out Speaker Snitch.