Arvind Sanjeev's SPIN Is a Synth with a Generative AI Twist — and a Turntable, Too

Maker Arvind Sanjeev has built a chunky portable synthesizer that provides a physical interface to the MusicGen language model (LM) so you can collaborate with the artificial intelligence (AI) on musical creations: SPIN.

"SPIN is an AI music synthesizer that allows you to co-create compositions with a language model, MusicGen," Sanjeev explains of his creation. "It is a playful invitation to explore the nuances of algorithmic music, encouraging you to slow down and zoom in on its artifacts. It celebrates the marriage between human and machine creativity through music."

Inside SPIN's eye-catching wooden housing is a Raspberry Pi 4 Model B single-board computer, connected to an Arduino Mega that drives the light-up input pad. Next to this is a Numark PT-01 record player, though there's no ordinary vinyl atop it: instead, it hosts a time-coded "control" record, the spinning of which controls playback. A Behringer audio driver reads the time code coming from the Numark deck, and the resulting audio output is played through built-in stereo speakers.

The music itself is controlled through the input pad, but not in the way you'd expect a normal synth to work. Rather than shaping waveforms or selecting pre-recorded clips for playback, the input pad is labeled with genres, moods, and instrument types — everything from drums and a guitar to "nature" and "water" — that combine to form a textual prompt. This prompt is then fed to MusicGen, which spits out its generated audio track — playback of which is controlled through the vinyl record.

"I wanted SPIN to encourage people to be playful; having a scratch interface served this purpose," Sanjeev writes of the inclusion of the record player. "A DVS adds an extra dimension while listening to the generated compositions. It allows us to slow down these synthetic tunes and listen between the notes. So, I decided to combine a DVS system with the MusicGen API in the form of an old-school synthesizer."

This isn't the first time Sanjeev has imbued an everyday object with the power of artificial intelligence. Back in December 2022 he showed off Ghostwriter, a typewriter with a difference — providing, as it did, a direct line to OpenAI's GPT-3 large language model for two-way conversations. "The Ghostwriter is a project that invites us to mindfully co-create with the AI," he explained at the time, "through a vintage typewriter's tactile and physical form."

"SPIN points to a future where music can be hyper-tailored to people's tastes," Sanjeev suggests. "It shows a glimpse of how AI can generate custom micro-genres that didn't exist before. However, does this come with an ethical cost? As part of my role as an adjunct faculty for AI at CIID [the Copenhagen Institute of Interaction Design], I have also been surveying AI's unintended consequences; especially around ownership.

"SPIN knocks on the door of ethical content creation. As MusicGen is trained on datasets of human-generated music, who really owns the copyright to its output? Ethical questions surrounding ownership, creativity, and potential biases in the algorithms are primary topics for discussion."

The full project write-up is available on Sanjeev's website.