A previous project I worked on involved two people listening to each other's heart beats to encourage connection and reflection. The implementation can be found on GitHub.
Upon seeing the Sony Spreesense board, I thought this project would be a perfect candidate to remake in hardware. I could split the stereo audio output across two channels to allow for synchronizing audio across two people and use two grove ear-clip heart rate sensors as before to capture the real-time heart beats of participants in order to determine how to trigger the heat beat sound.
My original implementation, however, required low-level sound processing that was outside the scope of the project. While I attempted to create a low level EEG waveform and manipulate it on-device in order to create the heart rate sound live, I ended up recording a pulse and playing back that pulse in the timing defined by the captured BPM for each other. It was really easy to get started with the simple use cases of the audio library and I was impressed at the features and ease of use working with both decoding mp3 files and playing back their contents.
This hardware approach that was possible given the library limitations of the Sony Board allows for one person (and with some small modifications to the real-time sound generation, two people) to listen to sound at the speed of their heart. The sound can be any file longer than a few seconds (usually heart rates don’t go below 60 beats per minute -- which is exactly one second), but the implementation of the Spreesense audio buffer does not allow for sound files that are too short since otherwise the buffer under-flows and crashes the library playing the sound file.
First, setup the spreesense board connecting it to the breakout board, being careful to not stress the board-to-board connector and installing the spacing pins and chip light shield included.
Second, Build or buy a simple breakout for the Grove connector to the Spreesense board. Be warned that the grove shield I have had too little clearance for the board and it did not fit. The +5 and GND pins are connected to GND and VIN on the Spreesense (which start one pin below the first connector on the pinout) then the SIG line is connected to pin 3 in the sketch.
Install the Spreesense SDK following the instructions at https://developer.sony.com/develop/spresense/developer-tools/get-started-using-arduino-ide/set-up-the-arduino-ide.
After installing the Spreesense SDK and choosing Spreesense as your target board and flasher, you can now upload the provided arduino sketch file.
Then, create a FAT32-formatted microsd card with a folder BIN uploading the mp3 decoder binary to that folder. The binary mp3 decoder file (MP3DEC) can be downloaded from: https://github.com/sonydevworld/spresense-arduino-compatible/raw/master/Arduino15/packages/SPRESENSE/hardware/spresense/1.0.0/libraries/Audio/examples/dsp_installer/mp3_dec_installer/MP3DEC
Then, create a folder SND and upload a file called Beat.mp3 that is the mp3 file you wish to play that is at least 20 seconds (even though only likely less than a second will be played at a time).
Finally, remove the SD card, insert into the spresense, connect the heart rate grove connector and pinout, then flash the firmware from the arduino sketch. Don't forget to connect headphones as well to listen to the audio :). You should be hearing your heart rate in the beat of the music being played back.
Future PlansIn the future, being able to realize real-time synthesis as opposed to exporting static files from Pure Data or Matlab would be ideal in generating a heart rate that feels realistic and powerful.
Other fun things that can be done is playing different sound files based on the heart rate -- only allowing certain playlists to be played if the heart rate is lower than 80 BPM, or playing high-energy music only when the heart rate is higher than 90 BPM. This project could easily be modified to make a heart-rate based seeking MP3 player.
Comments