I wanted to create an IoT device that can allow for streaming arbitrary mp3s, through the Sony Spresense device to take advantage of it's onboard hi-res audio output capabilities.
This opens up the potential to provide capabilities in numerous areas, for example:
- Allow playback of dynamically generated audio to assist the blind
- Provide a constant stream of audio for Air Traffic Control or Maritime Navigation systems
- Drive a home automation system with audio output for entertainment and for playing back information on news/weather and notifications from connected devices.
Now, the Sony Spresense is marketed as "Professional IoT (Internet of Things) development made easy". Unfortunately, the Spresense device itself, does not contain any on-board WIFI, GSM, or Ethernet capabilities thats where the esp8266 wifi module comes to play, this links the spresense board to the internet.
What Is I2S and How to Use It?
I2S or Inter-IC sound, is a protocol that allows for communicating PCM audio data between integrated circuits in an electronic device. This is accomplished with a minimum of three data lines. One is the Binary Clock or BCLK which operates as a continuous serial clock, the second is the WS or LRCK, specifies whether Left or Right audio channel data is currently being sent over serial, and a multiplexed serial data line DIN which contains the actual audio data.
All we need to do is wire up the ESP8266 device to connect the appropriate outputs to the proper inputs on the Sony Spresense to achieve I2S throughput. Normally, this would be pretty straightforward, but I found that I had to connect the I2S DOUT pin of the Sony Spresense to the VBUS on the Adafaruit Huzzah board otherwise the audio output would be very distorted. In addition, once this connection was made, it would fix the audio but disable serial communication to the Sony Spresense. For this reason, I employed a 3-pin switch to allow me to route I2S DOUT to GND when needing serial and for routing I2S DOUT to VBUS when I wanted clean audio playback.
Programming the Spresense to Enable I2S -> Speaker Throughput
Once you have everything wired up correctly, we are ready to start flashing code to devices. We will start by enabling the I2S-> Speaker throughput on the Sony Spresense.
You will want to take a look at the "Getting Started with the Sony Spresense Development SDK". This will require a Linux-like environment for proceeding. Following the guide will require that you clone the Sony Spresense Github repo.
Assuming you have followed the guide linked above and are able to flash the hello example, you are now ready to begin.
We will re-leverage the configuration for the "audio_through" example provided int the Sony Spresense SDK, but will change up a few things.
Start by overwriting spresense/examples/audio_through/audio_through_main.cxx with this modified code which enables I2S-Speaker throughput:
Once you have overwritten the file, execute the following on the terminal to build:
cd spresense/sdktools/config.py examples/audio_throughmake
Next, flash to the Spresense (change ttyUSB0 to that of your device) with:
tools/flash.sh -c /dev/ttyUSB0 nuttx.spk
Finally, connect to your device with minicom, as was used to execute the hello example, and execute:
audio_through
Your device should now be ready to pass audio data through I2S->Speaker.
Programming the ESP8266 with the Web Radio CodeNow that we have our devices connected properly and awaiting input, we just need to get our ESP8266 to output some data over I2S.
Start by cloning the ESP8266Audio and ESP8266SAM repos into your Arduino libraries directory.
cd <ArduinoInstallPath>/librariesgit clone https://github.com/earlephilhower/ESP8266Audio.gitgit clone https://github.com/earlephilhower/ESP8266SAM.git
Next, open the Arduino editor and select "Examples/ESP8266Audio/WebRadio", then "Save As" to save into a new directory for modification. Make sure you are in the newly opened project and overwrite WebRadio.ino with the following:
/* WebRadio Example Very simple HTML app to control web streaming Copyright (C) 2017 Earle F. Philhower, III This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.*/#include <Arduino.h>#ifdef ESP32 #include <WiFi.h>#else #include <ESP8266WiFi.h>#endif#include "AudioFileSourceICYStream.h"#include "AudioFileSourceBuffer.h"#include "AudioGeneratorMP3.h"#include "AudioOutputI2S.h"#include <EEPROM.h>#include <ESP8266SAM.h>// Custom web server that doesn't need much RAM#include "web.h"// To run, set your ESP8266 build to 160MHz, update the SSID info, and upload.// Enter your WiFi setup here:const char *SSID = "<YOUR SSID>";const char *PASSWORD = "<YOUR PASSWORD>";WiFiServer server(80);AudioGenerator *decoder = NULL;AudioFileSourceICYStream *file = NULL;AudioFileSourceBuffer *buff = NULL;AudioOutputI2S *out = NULL;int volume = 100;char url[256];char status[24];bool newUrl = false;int retryms = 0;typedef struct { char url[256]; int16_t volume; int16_t checksum;} Settings;// C++11 multiline string constants are neato...static const char HEAD[] PROGMEM = R"KEWL(<head><title>Sony Spresense Web Radio</title></head>)KEWL";static const char BODY[] PROGMEM = R"KEWL(<body><h3> Sony Spresense Web Radio </h3></br><img src="https:\\www.sony-semicon.co.jp\products_en\spresense\images\spresense.jpg"><hr>Volume: <input type="range" name="vol" min="1" max="150" steps="10" value="%d" onchange="showValue(this.value)"/> <span id="volspan">%d</span>%%<hr>Status: <span id="statusspan">%s</span><hr><form action="changeurl" method="GET">Current URL: %s<br>Change URL: <input type="text" name="url"><select name="type"><option value="mp3">MP3</option></select><input type="submit" value="Change"></form><form action="stop" method="POST"><input type="submit" value="Stop"></form></body>)KEWL";void HandleIndex(WiFiClient *client){ char buff[sizeof(BODY) + sizeof(status) + sizeof(url) + 3*2]; Serial.printf_P(PSTR("Sending INDEX...Free mem=%d\n"), ESP.getFreeHeap()); WebHeaders(client, NULL); WebPrintf(client, DOCTYPE); client->write_P( PSTR("<html>"), 6 ); client->write_P( HEAD, strlen_P(HEAD) ); sprintf_P(buff, BODY, volume, volume, status, url); client->write(buff, strlen(buff) ); client->write_P( PSTR("</html>"), 7 ); Serial.printf_P(PSTR("Sent INDEX...Free mem=%d\n"), ESP.getFreeHeap());}void HandleStatus(WiFiClient *client){ WebHeaders(client, NULL); client->write(status, strlen(status));}void HandleVolume(WiFiClient *client, char *params){ char *namePtr; char *valPtr; while (ParseParam(¶ms, &namePtr, &valPtr)) { ParamInt("vol", volume); } Serial.printf_P(PSTR("Set volume: %d\n"), volume); out->SetGain(((float)volume)/100.0); RedirectToIndex(client);}void HandleChangeURL(WiFiClient *client, char *params){ char *namePtr; char *valPtr; char newURL[sizeof(url)]; char newType[4]; newURL[0] = 0; newType[0] = 0; while (ParseParam(¶ms, &namePtr, &valPtr)) { ParamText("url", newURL); ParamText("type", newType); } if (newURL[0] && newType[0]) { newUrl = true; strncpy(url, newURL, sizeof(url)-1); url[sizeof(url)-1] = 0; strcpy_P(status, PSTR("Changing URL...")); Serial.printf_P(PSTR("Changed URL to: %s(%s)\n"), url, newType); RedirectToIndex(client); } else { WebError(client, 404, NULL, false); }}void RedirectToIndex(WiFiClient *client){ WebError(client, 301, PSTR("Location: /\r\n"), true);}void StopPlaying(){ if (decoder) { decoder->stop(); delete decoder; decoder = NULL; } if (buff) { buff->close(); delete buff; buff = NULL; } if (file) { file->close(); delete file; file = NULL; } strcpy_P(status, PSTR("Stopped"));}void HandleStop(WiFiClient *client){ Serial.printf_P(PSTR("HandleStop()\n")); StopPlaying(); RedirectToIndex(client);}// Called when a metadata event occurs (i.e. an ID3 tag, an ICY block, etc.void MDCallback(void *cbData, const char *type, bool isUnicode, const char *string){ const char *ptr = reinterpret_cast<const char *>(cbData); (void) isUnicode; // Punt this ball for now // Note that the type and string may be in PROGMEM, so copy them to RAM for printf char s1[32], s2[64]; strncpy_P(s1, type, sizeof(s1)); s1[sizeof(s1)-1]=0; strncpy_P(s2, string, sizeof(s2)); s2[sizeof(s2)-1]=0; Serial.printf("METADATA(%s) '%s' = '%s'\n", ptr, s1, s2); Serial.flush();}void StatusCallback(void *cbData, int code, const char *string){ const char *ptr = reinterpret_cast<const char *>(cbData); (void) code; (void) ptr; strncpy_P(status, string, sizeof(status)-1); status[sizeof(status)-1] = 0;}#ifdef ESP8266const int preallocateBufferSize = 5*1024;const int preallocateCodecSize = 29192; // MP3 codec max mem needed#elseconst int preallocateBufferSize = 16*1024;const int preallocateCodecSize = 85332; // AAC+SBR codec max mem needed#endifvoid *preallocateBuffer = NULL;void *preallocateCodec = NULL;void setup(){ // First, preallocate all the memory needed for the buffering and codecs, never to be freed preallocateBuffer = malloc(preallocateBufferSize); preallocateCodec = malloc(preallocateCodecSize); if (!preallocateBuffer || !preallocateCodec) { Serial.begin(115200); Serial.printf_P(PSTR("FATAL ERROR: Unable to preallocate %d bytes for app\n"), preallocateBufferSize+preallocateCodecSize); while (1) delay(1000); // Infinite halt } Serial.begin(115200); delay(1000); Serial.printf_P(PSTR("Connecting to WiFi\n")); WiFi.disconnect(); WiFi.softAPdisconnect(true); WiFi.mode(WIFI_STA); WiFi.begin(SSID, PASSWORD); // Try forever while (WiFi.status() != WL_CONNECTED) { Serial.printf_P(PSTR("...Connecting to WiFi\n")); delay(1000); }Serial.printf_P(PSTR("Connected\n")); Serial.printf_P(PSTR("Go to http://")); Serial.print(WiFi.localIP()); Serial.printf_P(PSTR("/ to control the web radio.\n")); server.begin(); strcpy_P(url, PSTR("none")); strcpy_P(status, PSTR("OK")); file = NULL; buff = NULL; out = new AudioOutputI2S(); decoder = NULL; ESP8266SAM *sam = new ESP8266SAM; sam->SetVoice(sam->SAMVoice::VOICE_SAM); sam->Say(out, "Welcome to Internet Radio powered by So knee Ess Presents!"); delay(500); sam->Say(out, "Let's make some noise!"); delay(500); sam->Say(out, "Connect to me at: "); char ip[16]; sprintf(ip, "%d.%d.%d.%d", WiFi.localIP()[0], WiFi.localIP()[1], WiFi.localIP()[2], WiFi.localIP()[3] ); sam->Say(out, ip); delete sam; delete ip; LoadSettings();}void StartNewURL(){ Serial.printf_P(PSTR("Changing URL to: %s, vol=%d\n"), url, volume); newUrl = false; // Stop and free existing ones Serial.printf_P(PSTR("Before stop...Free mem=%d\n"), ESP.getFreeHeap()); StopPlaying(); Serial.printf_P(PSTR("After stop...Free mem=%d\n"), ESP.getFreeHeap()); SaveSettings(); Serial.printf_P(PSTR("Saved settings\n")); file = new AudioFileSourceICYStream(url); Serial.printf_P(PSTR("created icystream\n")); file->RegisterMetadataCB(MDCallback, NULL); buff = new AudioFileSourceBuffer(file, preallocateBuffer, preallocateBufferSize); Serial.printf_P(PSTR("created buffer\n")); buff->RegisterStatusCB(StatusCallback, NULL); decoder = new AudioGeneratorMP3(preallocateCodec, preallocateCodecSize); Serial.printf_P(PSTR("created decoder\n")); decoder->RegisterStatusCB(StatusCallback, NULL); Serial.printf_P("Decoder start...\n"); out->SetGain(((float)volume)/100.0); decoder->begin(buff, out); if (!decoder->isRunning()) { Serial.printf_P(PSTR("Can't connect to URL")); StopPlaying(); strcpy_P(status, PSTR("Unable to connect to URL")); retryms = millis() + 2000; } Serial.printf_P("Done start new URL\n");}void LoadSettings(){ // Restore from EEPROM, check the checksum matches Settings s; uint8_t *ptr = reinterpret_cast<uint8_t *>(&s); EEPROM.begin(sizeof(s)); for (size_t i=0; i<sizeof(s); i++) { ptr[i] = EEPROM.read(i); } EEPROM.end(); int16_t sum = 0x1234; for (size_t i=0; i<sizeof(url); i++) sum += s.url[i]; sum += s.volume; if (s.checksum == sum) { strcpy(url, s.url); volume = s.volume; Serial.printf_P(PSTR("Resuming stream from EEPROM: %s, type=%s, vol=%d\n"), url, "MP3", volume); newUrl = true; }}void SaveSettings(){ // Store in "EEPROM" to restart automatically Settings s; memset(&s, 0, sizeof(s)); strcpy(s.url, url); s.volume = volume; s.checksum = 0x1234; for (size_t i=0; i<sizeof(url); i++) s.checksum += s.url[i]; s.checksum += s.volume; uint8_t *ptr = reinterpret_cast<uint8_t *>(&s); EEPROM.begin(sizeof(s)); for (size_t i=0; i<sizeof(s); i++) { EEPROM.write(i, ptr[i]); } EEPROM.commit(); EEPROM.end();}void PumpDecoder(){ if (decoder && decoder->isRunning()) { strcpy_P(status, PSTR("Playing")); // By default we're OK unless the decoder says otherwise if (!decoder->loop()) { Serial.printf_P(PSTR("Stopping decoder\n")); StopPlaying(); retryms = millis() + 2000; }}}void loop(){ static int lastms = 0; if (millis()-lastms > 1000) { lastms = millis(); Serial.printf_P(PSTR("Running for %d seconds%c...Free mem=%d\n"), lastms/1000, !decoder?' ':(decoder->isRunning()?'*':' '), ESP.getFreeHeap()); } if (retryms && millis()-retryms>0) { retryms = 0; newUrl = true; } if (newUrl) { StartNewURL(); } PumpDecoder(); char *reqUrl; char *params; WiFiClient client = server.available(); PumpDecoder(); char reqBuff[384]; if (client && WebReadRequest(&client, reqBuff, 384, &reqUrl, ¶ms)) { PumpDecoder(); if (IsIndexHTML(reqUrl)) { HandleIndex(&client); } else if (!strcmp_P(reqUrl, PSTR("stop"))) { HandleStop(&client); } else if (!strcmp_P(reqUrl, PSTR("status"))) { HandleStatus(&client); } else if (!strcmp_P(reqUrl, PSTR("setvol"))) { HandleVolume(&client, params); } else if (!strcmp_P(reqUrl, PSTR("changeurl"))) { HandleChangeURL(&client, params); } else { WebError(&client, 404, NULL, false); } } PumpDecoder(); if (client) { client.flush(); client.stop(); }}
This code removes AAC support and adds an ability to speak the connection information needed to connect to the web portal running on the ESP8266. These modifications leave just enough memory to allow for decoding low-bitrate mp3s on the ESP8266 itself.
Flash the code to your ESP8266 device.
Once flashed, you can connect to the serial port to retrieve the address to connect to the web portal or if your Spresense is plugged into a speaker, you will hear it "speak" the ip address out loud.
Once you connect to the portal via the ip address, you can configure the URL to use for MP3 playback.
Once an mp3 has been successfully set, the device will store that URL in the EEPROM of the ESP8266 and will begin playing it on successive restarts of the ESP8266 device. If this ever causes issues, you can always reflash the device to erase all EEPROM contents and start over fresh.
It is important to note that due to limitations of the ESP8266 processing power and memory availability, it may not be possible to playback mp3s encoded at certain bitrates, Using some assistance from our ESP8266 device, we were able to bring the Sony Spresence device to life with connectivity to the internet to allow for streaming mp3 content from arbitrary sources.
Comments