Hardware components | ||||||
| × | 1 | ||||
Software apps and online services | ||||||
 |
| |||||
| ||||||
| ||||||
My smart home instantly anticipates and fulfils my every material need, so what then is this strange emptiness I feel inside?
What if I could somehow express the vastness of this inner void in the form of sound?
What if I could conjure this sound out of thin air with nothing but my hands?
Why do I wake up in the night with a vague memory of a soft voice chanting "Radar ... ... RADAR ... ... RAAAAADAAAAR!"?
This project seeks to answer the above questions and more.
Origins ... 🕳🐇The Theremin was one of the first electronic musical instruments to attain a degree of popularity. The orginal was essentially a malfunctioning radio - by heterodyning two radio frequency oscilators, the audible beat-frequencies responded to the movement of the player's hands and could cover the entire range of pitches hearable by humans (and, depending on the speakers available, possibly whales, dogs and bats as well). Controlled by the player's body's capacitance to earth, the Theremin is also the ancestor of modern capacitive sensing devices.
If a sophisticated personal RADAR system is available to us, what does it let me do that other technologies do not? Can I measure distance, angles, speeds etc. of hands and other objects and use such measurements to control a synthesiser? Could gestures let me control other aspects of the sound, like a conductor rather than a player?
I now feel compelled to find out!
Infineon's PSoC products allow for some very interesting use of capacitance sensing and, as I have shown elsewhere, have tremendous potential for musical applications.
The Plan ...Using the Infineon's PSoC 6 Artificial Intelligence Evaluation Kit, I want to explore its Xensiv RADAR module for musical purposes.
I am not very good at programming in C, so I will try my best to ride on the shoulders of giants as much as possible and keep things simple wherever I can.
I plan to start with a simple RADAR distance-sensing example such as: https://www.infineon.com/dgdl/Infineon-AN667_Distance_estimation_using_XENSIV_KIT_CSK_BGT60TR13C-ApplicationNotes-v01_00-EN.pdf
Once I have the example built and running on CY8CKIT-062S2-AI, I will modify the code to take the example's console output and send it via TCP socket to another computer on the same network.
On the receiving computer, I will use Pure Data to translate the received TCP packets to audio.
Pure Data is a flow-chart-style environment that lets you build up complex interactive sound systems by connecting various fundamental processes together with graphical wires. In the above picture, the netreceive object at top left accepts a network connection from our PSoC board (okay, Matlab in the pictured example) and receives data created by our RADAR sensing subsystem. The print object below it echoes anything it receives to the console.
Practicalities ...From a quick read of the RADAR specs, we can track the hand's distance relative to the sensor and send that data on to another device. I hope to combine this with recognition of gestures and other types of symbolic movement to make other musical changes in the receiving software.
Depending on how far I get, perhaps I will figure out how to track more than one hand and to train a vocabulary of hand signs and gestures that could be mapped to some aspect of the sound.
The board's UART output could be persuaded to send 'classic' serial MIDI data to synthesisers and other devices. The USB port can also be persuaded to send and receive MIDI. There is even a bluetooth MIDI protocol. although I would love to get all MIDI types working, However, to save time, I will probably stick to raw TCP data directly to the other device then use PureData to translate it into sound. I am hoping I can use this code example to help figure out how to connect and send to PureData from the PSoC board. https://github.com/Infineon/mtb-example-wifi-tcp-client#readme
In general, I am aiming for something quick and dirty to begin with, a minimum viable product rather than a perfect fine instrument. We can always come back later and make it more fancy once it's working.
Of course, the board has a range of other cool sensors that could also become musical controllers of some kind, including PSoC6's feted cap-sense abilities for 'real' theremin control. For now though, I'm planning to keep things simple and get just one thing working properly rather than trying for a heap of things that only sort-of work. If I can, for once, follow even that one simple rule, I'll be very happy! :)
Comments