An AI Assistant with an Attitude

Over the past few years an AI summer has been heating up like never before, and many new and interesting applications have been unveiled during this time. As is usually the case, quite a lot of hype also came along for the ride. A number of overhyped AI assistants, in particular, have emerged lately that promise to do absolutely everything for you, but end up doing little more than a typical smartphone, all while proving to be frustratingly unreliable.

Hardware hacker Jared Carrillo was fed up with these under-delivering AI assistants and, rather than complaining, decided to try to do something about it. His goal was to build a portable AI assistant that could not only answer questions, but also take some actions, like placing an order for a pizza. He sort of succeeded, but really ended up with an assistant that is about as frustrating as any other product on the market. But at least it is his frustrating AI assistant. And it was also a good opportunity for experimentation, which Carrillo believes will give him the insights he needs to eventually build a better device.

The assistant is called the Intelligent System for Automated AI Computing (ISAAC). The initial prototype was built into a 3D-printed case, but ultimately a nicer metal case was manufactured by PCBWay. Inside the case, there is a Raspberry Pi Compute Module 4 for processing, as well as a Nano Base Board to provide for easy peripheral connections. There is also a 2.8-inch display for interactive features, and a Raspberry Pi camera to capture images. Audio recording and playback are made possible with a RASPIAUDIO sound card. A rechargeable battery provides power to the portable assistant.

ISAAC operates in a similar way to many other voice assistants made by other hardware hackers. After pressing a button on the device, it records five seconds of audio. That audio is then transcribed to text using the OpenAI Whisper API. That text is fed into ChatGPT, and finally the response is converted to audio with the eSpeak speech synthesizer before being played by ISAAC.

This architecture enables the device to respond to spoken questions, and also explain what is present in images captured by the camera. One of the big benefits of building your own device is that you can program it to do whatever you want. In this case, Carrillo built some hooks into the software that made it possible to order a pizza with a simple voice request. He also built in a feature that can get back at a friend that has been bugging him by spamming them with a steady stream of text messages. That is not exactly a standard feature of AI assistants — although that is probably a good thing.

ISAAC is not going to change the world, but the methods used by Carrillo can easily be copied, so it should be pretty simple to clone ISAAC. Be sure to check out the video for all the details if you want to create your own custom AI assistant.