Talking Web Radio with ESP8266 and Sony Spresense

Demo

Introduction

I was very excited to learn that Sony is making headway into the realm of IoT with their recently announced Sony Spresense hardware. This board is very interesting in that it contains a multi-core CXD5602 microcontroller (ARM® Cortex®-M4F × 6 cores @ 156 MHz) and integrated GPS. What caught my attention though, is it's support for hi-res audio output using a built-in Digital Audio Converter.

Hackster.io recently partnered with Sony for the "Make it Better" contest which provides contestants with free Sony Spresense hardware on the basis that an idea is submitted and a project is created on Hakcster which utilizes the hardware. This contest runs through February 4, 2019 and offers a variety of really cool prizes including a Sony Playstation 4 VR Bundle for the top submission in four unique categories!

During the beginning of the contest, contestants could submit ideas that were vetted and approved upfront before hardware was shipped. My idea involved using an IoT device to playback music using Youtube as the content source. While I did not quite build out to my proposed spec, I did end up demonstrating that mp3s can be streamed through the Sony Spresense board using a configurable web portal which enables numerous scenarios. This project documents that process in detail and demonstrates some really cool capabilities on both the ESP8266 and Sony Spresense devices.

The Problem

I want to create an IoT device that can allow for streaming arbitrary mp3s, through the Sony Spresense device to take advantage of it's onboard hi-res audio output capabilities.

This opens up the potential to provide capabilities in numerous areas, for example:

• Allow playback of dynamically generated audio to assist the blind
• Provide a constant stream of audio for Air Traffic Control or Maritime Navigation systems
• Drive a home automation system with audio output for entertainment and for playing back information on news/weather and notifications from connected devices.

Now, the Sony Spresense is marketed as "Professional IoT (Internet of Things) development made easy". Unfortunately, the Spresense device itself, does not contain any on-board WIFI, GSM, or Ethernet capabilities. So this creates a bit of a problem right from the start.

How can you even begin to stream content from an online source to an IoT device if it does not have internet?

The Solution

There are probably a few ways to solve this issue, but my initial hunch was to grab the least expensive WIFI controller I could find and get it connected via serial, I2C, or some other protocol (I2S) to enable network communication to the Sony Spresense.

I initially opted for an ESP8266 based solution. For purposes of this project, you can use any ESP8266 device with the exception of the ESP8266-01. The reason, being, as we will discuss later on, is that the I2S outputs are not fully exposed on that revision.

The ESP8266 is great because it is cheap, and it also has a wonderful ESP8266Audio library available that is maintained by Earle F. Philhower, III. This library allows for outputting a variety of audio formats (Amiga Mod files, Nokia Ring Tones, Midi with custom fonts, AAC, FLAC, and MP3). The best part, is that the library decodes the format on the ESP8266 itself and is able to pass the digital audio signal over I2S, which is perfect because the Sony Spresense has capabilities of I2S input and pass-through.

I opted to use an ESP8266 exposed through an ESP-12 on an Adafruit Feather Huzzah, mostly because I had it on hand and did not have time to wait for additional hardware to be shipped. The instructions should translate well to any ESP-12 device and can likely be re-leveraged without much modification to the more powerful ESP-32.

What Is I2S and How to Use It?

I2S or Inter-IC sound, is a protocol that allows for communicating PCM audio data between integrated circuits in an electronic device. This is accomplished with a minimum of three data lines. One is the Binary Clock or BCLK which operates as a continuous serial clock, the second is the WS or LRCK, specifies whether Left or Right audio channel data is currently being sent over serial, and a multiplexed serial data line DIN which contains the actual audio data.

All we need to do is wire up the ESP8266 device to connect the appropriate outputs to the proper inputs on the Sony Spresense to achieve I2S throughput. Normally, this would be pretty straightforward, but I found that I had to connect the I2S DOUT pin of the Sony Spresense to the VBUS on the Adafaruit Huzzah board otherwise the audio output would be very distorted. In addition, once this connection was made, it would fix the audio but disable serial communication to the Sony Spresense. For this reason, I employed a 3-pin switch to allow me to route I2S DOUT to GND when needing serial and for routing I2S DOUT to VBUS when I wanted clean audio playback.

Here is a circuit diagram showing these connections:

Programming the Spresense to Enable I2S -> Speaker Throughput

Once you have everything wired up correctly, we are ready to start flashing code to devices. We will start by enabling the I2S-> Speaker throughput on the Sony Spresense.

You will want to take a look at the "Getting Started with the Sony Spresense Development SDK". This will require a Linux-like environment for proceeding. Following the guide will require that you clone the Sony Spresense Github repo.

Assuming you have followed the guide linked above and are able to flash the hello example, you are now ready to begin.

We will re-leverage the configuration for the "audio_through" example provided int the Sony Spresense SDK, but will change up a few things.

Start by overwriting spresense/examples/audio_through/audio_through_main.cxx with this modified code which enables I2S-Speaker throughput:

/****************************************************************************
* audio_through/audio_through_main.cxx
*
*   Copyright 2018 Sony Semiconductor Solutions Corporation
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* 1. Redistributions of source code must retain the above copyright
*    notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
*    notice, this list of conditions and the following disclaimer in
*    the documentation and/or other materials provided with the
*    distribution.
* 3. Neither the name of Sony Semiconductor Solutions Corporation nor
*    the names of its contributors may be used to endorse or promote
*    products derived from this software without specific prior written
*    permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
* FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
* COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
* BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
* OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
* AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
* ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*
****************************************************************************/
/****************************************************************************
* Included Files
****************************************************************************/
#include <sdk/config.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <dirent.h>
#include <fcntl.h>
#include <errno.h>
#include <arch/board/board.h>
#include <asmp/mpshm.h>
#include <arch/chip/pm.h>
#include <sys/stat.h>
#include "memutils/os_utils/chateau_osal.h"
#include "audio/audio_high_level_api.h"
#include "memutils/message/Message.h"
#include "include/msgq_id.h"
#include "include/msgq_pool.h"
/****************************************************************************
* Pre-processor Definitions
****************************************************************************/
/* Default Volume. -20dB */
#define DEF_VOLUME -20
/****************************************************************************
* Private Types
****************************************************************************/
/****************************************************************************
* Public Type Declarations
****************************************************************************/
/****************************************************************************
* Public Function Prototypes
****************************************************************************/
/****************************************************************************
* Public Data
****************************************************************************/
/****************************************************************************
* Private Data
****************************************************************************/
/* For share memory. */
static mpshm_t s_shm;
/****************************************************************************
* Private Functions
****************************************************************************/
static bool printAudCmdResult(uint8_t command_code, AudioResult& result)
{
 if (AUDRLT_ERRORRESPONSE == result.header.result_code)
   {
     printf("Command code(0x%x): AUDRLT_ERRORRESPONSE:"
            "Module id(0x%x): Error code(0x%x)\n",
             command_code,
             result.error_response_param.module_id,
             result.error_response_param.error_code);
     return false;
   }
 else if (AUDRLT_ERRORATTENTION == result.header.result_code)
   {
     printf("Command code(0x%x): AUDRLT_ERRORATTENTION\n", command_code);
     return false;
   }
 return true;
}
static void app_attention_callback(const ErrorAttentionParam *attparam)
{
 printf("Attention!! %s L%d ecode %d subcode %d\n",
         attparam->error_filename,
         attparam->line_number,
         attparam->error_code,
         attparam->error_att_sub_code);
}
static bool app_create_audio_sub_system(void)
{
 /* Create manager of AudioSubSystem. */
 AudioSubSystemIDs ids;
 ids.app         = MSGQ_AUD_APP;
 ids.mng         = MSGQ_AUD_MGR;
 ids.player_main = 0xFF;
 ids.player_sub  = 0xFF;
 ids.mixer       = 0xFF;
 ids.recorder    = 0xFF;
 ids.effector    = 0xFF;
 ids.recognizer  = 0xFF;
 AS_CreateAudioManager(ids, app_attention_callback);
 /* Set callback function of attention message */
 return true;
}
static void app_deact_audio_sub_system(void)
{
 AS_DeleteAudioManager();
}
static bool app_power_on(void)
{
 AudioCommand command;
 command.header.packet_length = LENGTH_POWERON;
 command.header.command_code  = AUDCMD_POWERON;
 command.header.sub_code      = 0x00;
 command.power_on_param.enable_sound_effect = AS_DISABLE_SOUNDEFFECT;
 AS_SendAudioCommand(&command);
 AudioResult result;
 AS_ReceiveAudioResult(&result);
 return printAudCmdResult(command.header.command_code, result);
}
static bool app_power_off(void)
{
 AudioCommand command;
 command.header.packet_length = LENGTH_SET_POWEROFF_STATUS;
 command.header.command_code  = AUDCMD_SETPOWEROFFSTATUS;
 command.header.sub_code      = 0x00;
 AS_SendAudioCommand(&command);
 AudioResult result;
 AS_ReceiveAudioResult(&result);
 return printAudCmdResult(command.header.command_code, result);
}
static bool app_set_ready(void)
{
 AudioCommand command;
 command.header.packet_length = LENGTH_SET_READY_STATUS;
 command.header.command_code  = AUDCMD_SETREADYSTATUS;
 command.header.sub_code      = 0x00;
 AS_SendAudioCommand(&command);
 AudioResult result;
 AS_ReceiveAudioResult(&result);
 return printAudCmdResult(command.header.command_code, result);
}
static bool app_set_volume(int master_db)
{
 AudioCommand command;
 command.header.packet_length = LENGTH_SETVOLUME;
 command.header.command_code  = AUDCMD_SETVOLUME;
 command.header.sub_code      = 0;
 command.set_volume_param.input1_db = 0;
 command.set_volume_param.input2_db = AS_VOLUME_MUTE;
 command.set_volume_param.master_db = master_db;
 AS_SendAudioCommand(&command);
 AudioResult result;
 AS_ReceiveAudioResult(&result);
 return printAudCmdResult(command.header.command_code, result);
}
static bool app_set_mute()
{
 AudioCommand command;
 command.header.packet_length = LENGTH_SETVOLUME;
 command.header.command_code  = AUDCMD_SETVOLUME;
 command.header.sub_code      = 0;
 command.set_volume_param.input1_db = 0;
 command.set_volume_param.input2_db = AS_VOLUME_MUTE;
 command.set_volume_param.master_db = AS_VOLUME_MUTE;
 AS_SendAudioCommand(&command);
 AudioResult result;
 AS_ReceiveAudioResult(&result);
 return printAudCmdResult(command.header.command_code, result);
}
static bool app_set_through_status(void)
{
 AudioCommand command;
 command.header.packet_length = LENGTH_SET_THROUGH_STATUS;
 command.header.command_code = AUDCMD_SETTHROUGHSTATUS;
 command.header.sub_code = 0x00;
 AS_SendAudioCommand(&command);
 AudioResult result;
 AS_ReceiveAudioResult(&result);
 return printAudCmdResult(command.header.command_code, result);
}
static bool app_set_through_path()
{
 AudioResult result;
 AudioCommand command;
 command.header.packet_length = LENGTH_SET_THROUGH_PATH;
 command.header.command_code = AUDCMD_SETTHROUGHPATH;
 command.header.sub_code = 0x00;
 /* i2s in -> sp out */
 command.set_through_path.path1.en  = true;
 command.set_through_path.path1.in  = AS_THROUGH_PATH_IN_I2S1;
 command.set_through_path.path1.out = AS_THROUGH_PATH_OUT_MIXER1;
 command.set_through_path.path2.en = false;
 AS_SendAudioCommand(&command);
 AS_ReceiveAudioResult(&result);
 return printAudCmdResult(command.header.command_code, result);
}
static bool app_init_output_select(void)
{
 AudioCommand command;
 command.header.packet_length = LENGTH_INITOUTPUTSELECT;
 command.header.command_code  = AUDCMD_INITOUTPUTSELECT;
 command.header.sub_code      = 0;
 command.init_output_select_param.output_device_sel = AS_OUT_SP;
 AS_SendAudioCommand(&command);
 AudioResult result;
 AS_ReceiveAudioResult(&result);
 return printAudCmdResult(command.header.command_code, result);
}
static bool app_init_mic_gain(void)
{
 AudioCommand command;
 command.header.packet_length = LENGTH_INITMICGAIN;
 command.header.command_code  = AUDCMD_INITMICGAIN;
 command.header.sub_code      = 0;
 command.init_mic_gain_param.mic_gain[0] = 210;
 command.init_mic_gain_param.mic_gain[1] = 210;
 command.init_mic_gain_param.mic_gain[2] = AS_MICGAIN_HOLD;
 command.init_mic_gain_param.mic_gain[3] = AS_MICGAIN_HOLD;
 command.init_mic_gain_param.mic_gain[4] = AS_MICGAIN_HOLD;
 command.init_mic_gain_param.mic_gain[5] = AS_MICGAIN_HOLD;
 command.init_mic_gain_param.mic_gain[6] = AS_MICGAIN_HOLD;
 command.init_mic_gain_param.mic_gain[7] = AS_MICGAIN_HOLD;
 AS_SendAudioCommand(&command);
 AudioResult result;
 AS_ReceiveAudioResult(&result);
 return printAudCmdResult(command.header.command_code, result);
}
static bool app_init_i2s_param(void)
{
 AudioCommand command;
 command.header.packet_length  = LENGTH_INITI2SPARAM;
 command.header.command_code   = AUDCMD_INITI2SPARAM;
 command.header.sub_code       = 0;
 command.init_i2s_param.i2s_id = AS_I2S1;
 command.init_i2s_param.rate   = AS_SAMPLINGRATE_44100;
 command.init_i2s_param.bypass_mode_en = AS_I2S_BYPASS_MODE_DISABLE;
 AS_SendAudioCommand(&command);
 AudioResult result;
 AS_ReceiveAudioResult(&result);
 return printAudCmdResult(command.header.command_code, result);
}
static bool app_init_libraries(void)
{
 int ret;
 uint32_t addr = MSGQ_TOP_DRM;
 /* Initialize shared memory.*/
 ret = mpshm_init(&s_shm, 1, 1024 * 128);
 if (ret < 0)
   {
     printf("Error: mpshm_init() failure. %d\n", ret);
     return false;
   }
 ret = mpshm_remap(&s_shm, (void *)addr);
 if (ret < 0)
   {
     printf("Error: mpshm_remap() failure. %d\n", ret);
     return false;
   }
 /* Initalize MessageLib. */
 err_t err = MsgLib::initFirst(NUM_MSGQ_POOLS, MSGQ_TOP_DRM);
 if (err != ERR_OK)
   {
     printf("Error: MsgLib::initFirst() failure. 0x%x\n", err);
     return false;
   }
 err = MsgLib::initPerCpu();
 if (err != ERR_OK)
   {
     printf("Error: MsgLib::initPerCpu() failure. 0x%x\n", err);
     return false;
   }
 return true;
}
static bool app_finalize_libraries(void)
{
 /* Finalize MessageLib. */
 MsgLib::finalize();
 /* Destroy shared memory. */
 int ret;
 ret = mpshm_detach(&s_shm);
 if (ret < 0)
   {
     printf("Error: mpshm_detach() failure. %d\n", ret);
     return false;
   }
 ret = mpshm_destroy(&s_shm);
 if (ret < 0)
   {
     printf("Error: mpshm_destroy() failure. %d\n", ret);
     return false;
   }
 return true;
}
/****************************************************************************
* Public Functions
****************************************************************************/
#ifdef CONFIG_BUILD_KERNEL
extern "C" int main(int argc, FAR char *argv[])
#else
extern "C" int audio_through_main(int argc, char *argv[])
#endif
{
 printf("Start AudioThrough example\n");
 /* First, initialize the shared memory and memory utility used by AudioSubSystem. */
 if (!app_init_libraries())
   {
     printf("Error: init_libraries() failure.\n");
     return 1;
   }
 /* Next, Create the features used by AudioSubSystem. */
 if (!app_create_audio_sub_system())
   {
     printf("Error: act_audiosubsystem() failure.\n");
     return 1;
   }
 /* On and after this point, AudioSubSystem must be active.
  * Register the callback function to be notified when a problem occurs.
  */
 /* Change AudioSubsystem to Ready state so that I/O parameters can be changed. */
 if (!app_power_on())
   {
     printf("Error: app_power_on() failure.\n");
     return 1;
   }
 /* Initialize Speaker Out */
   if (!app_init_output_select())
   {
     printf("Error: app_init_output_select() failure.\n");
     return 1;
   }
 /* Set through operation mode. */
 if (!app_set_through_status())
   {
     printf("Error: app_set_through_status() failure.\n");
     return 1;
   }
 /* Initialize I2S parameter */
 if (!app_init_i2s_param())
   {
     printf("Error: app_init_mic_gain() failure.\n");
     return 1;
   }
 /* Initialize Mic gain */
 if (!app_init_mic_gain())
   {
     printf("Error: app_init_mic_gain() failure.\n");
     return 1;
   }
 /* Start through operation. */
      /* Set output mute. */
      if (board_external_amp_mute_control(true) != OK)
        {
          printf("Error: board_external_amp_mute_control(true) failuer.\n");
          return 1;
        }
     if (!app_set_through_path())
       {
         printf("Error: app_set_through_path() failure.\n");
         return 1;
       }
     /* If output speaker, cancel mute. */
/*
     if (!(type == TEST_PATH_IN_MIC_OUT_I2S ||
           type == TEST_PATH_IN_I2S_OUT_I2S))
*/
       {
       /* Cancel mute. */
         if (!app_set_volume(DEF_VOLUME))
           {
             printf("Error: app_set_volume() failure.\n");
             return 1;
           }
         if (board_external_amp_mute_control(false) != OK)
           {
             printf("Error: board_external_amp_mute_control(false) failuer.\n");
             return 1;
           }
       }
     /* Running... */
       while(true)
       {
         sleep(1);
       }
     /* Set mute. */
     app_set_mute();
 /* Set output mute. */
 if (board_external_amp_mute_control(true) != OK)
   {
     printf("Error: board_external_amp_mute_control(true) failuer.\n");
     return 1;
   }
 /* Return the state of AudioSubSystem before voice_call operation. */
 if (!app_set_ready())
   {
     printf("Error: app_set_ready() failure.\n");
     return 1;
   }
 /* Change AudioSubsystem to PowerOff state. */
 if (!app_power_off())
   {
     printf("Error: app_power_off() failure.\n");
     return 1;
   }
 /* Deactivate the features used by AudioSubSystem. */
 app_deact_audio_sub_system();
 /* finalize the shared memory and memory utility used by AudioSubSystem. */
 if (!app_finalize_libraries())
   {
     printf("Error: finalize_libraries() failure.\n");
     return 1;
   }
 printf("Exit AudioThrough example\n");
 return 0;
}

Once you have overwritten the file, execute the following on the terminal to build:

cd spresense/sdk
tools/config.py examples/audio_through
make

Next, flash to the Spresense (change ttyUSB0 to that of your device) with:

tools/flash.sh -c /dev/ttyUSB0 nuttx.spk

Finally, connect to your device with minicom, as was used to execute the hello example, and execute:

audio_through

Your device should now be ready to pass audio data through I2S->Speaker.

Programming the ESP8266 with the Web Radio Code

Now that we have our devices connected properly and awaiting input, we just need to get our ESP8266 to output some data over I2S.

Start by cloning the ESP8266Audio and ESP8266SAM repos into your Arduino libraries directory.

cd <ArduinoInstallPath>/libraries
git clone https://github.com/earlephilhower/ESP8266Audio.git
git clone https://github.com/earlephilhower/ESP8266SAM.git

Next, open the Arduino editor and select "Examples/ESP8266Audio/WebRadio", then "Save As" to save into a new directory for modification. Make sure you are in the newly opened project and overwrite WebRadio.ino with the following:

/*
 WebRadio Example
 Very simple HTML app to control web streaming
 Copyright (C) 2017  Earle F. Philhower, III
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
 the Free Software Foundation, either version 3 of the License, or
 (at your option) any later version.
 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU General Public License for more details.
 You should have received a copy of the GNU General Public License
 along with this program.  If not, see <http://www.gnu.org/licenses/>.
*/
#include <Arduino.h>
#ifdef ESP32
   #include <WiFi.h>
#else
   #include <ESP8266WiFi.h>
#endif
#include "AudioFileSourceICYStream.h"
#include "AudioFileSourceBuffer.h"
#include "AudioGeneratorMP3.h"
#include "AudioOutputI2S.h"
#include <EEPROM.h>
#include <ESP8266SAM.h>
// Custom web server that doesn't need much RAM
#include "web.h"
// To run, set your ESP8266 build to 160MHz, update the SSID info, and upload.
// Enter your WiFi setup here:
const char *SSID = "<YOUR SSID>";
const char *PASSWORD = "<YOUR PASSWORD>";
WiFiServer server(80);
AudioGenerator *decoder = NULL;
AudioFileSourceICYStream *file = NULL;
AudioFileSourceBuffer *buff = NULL;
AudioOutputI2S *out = NULL;
int volume = 100;
char url[256];
char status[24];
bool newUrl = false;
int retryms = 0;
typedef struct {
 char url[256];
 int16_t volume;
 int16_t checksum;
} Settings;
// C++11 multiline string constants are neato...
static const char HEAD[] PROGMEM = R"KEWL(
<head>
<title>Sony Spresense Web Radio</title>
</head>)KEWL";
static const char BODY[] PROGMEM = R"KEWL(
<body>
<h3> Sony Spresense Web Radio </h3>
</br>
<img src="https:\\www.sony-semicon.co.jp\products_en\spresense\images\spresense.jpg">
<hr>
Volume: <input type="range" name="vol" min="1" max="150" steps="10" value="%d" onchange="showValue(this.value)"/> <span id="volspan">%d</span>%%
<hr>
Status: <span id="statusspan">%s</span>
<hr>
<form action="changeurl" method="GET">
Current URL: %s<br>
Change URL: <input type="text" name="url">
<select name="type"><option value="mp3">MP3</option></select>
<input type="submit" value="Change"></form>
<form action="stop" method="POST"><input type="submit" value="Stop"></form>
</body>)KEWL";
void HandleIndex(WiFiClient *client)
{
 char buff[sizeof(BODY) + sizeof(status) + sizeof(url) + 3*2];
 Serial.printf_P(PSTR("Sending INDEX...Free mem=%d\n"), ESP.getFreeHeap());
 WebHeaders(client, NULL);
 WebPrintf(client, DOCTYPE);
 client->write_P( PSTR("<html>"), 6 );
 client->write_P( HEAD, strlen_P(HEAD) );
 sprintf_P(buff, BODY, volume, volume, status, url);
 client->write(buff, strlen(buff) );
 client->write_P( PSTR("</html>"), 7 );
 Serial.printf_P(PSTR("Sent INDEX...Free mem=%d\n"), ESP.getFreeHeap());
}
void HandleStatus(WiFiClient *client)
{
 WebHeaders(client, NULL);
 client->write(status, strlen(status));
}
void HandleVolume(WiFiClient *client, char *params)
{
 char *namePtr;
 char *valPtr;
 while (ParseParam(&params, &namePtr, &valPtr)) {
   ParamInt("vol", volume);
 }
 Serial.printf_P(PSTR("Set volume: %d\n"), volume);
 out->SetGain(((float)volume)/100.0);
 RedirectToIndex(client);
}
void HandleChangeURL(WiFiClient *client, char *params)
{
 char *namePtr;
 char *valPtr;
 char newURL[sizeof(url)];
 char newType[4];
 newURL[0] = 0;
 newType[0] = 0;
 while (ParseParam(&params, &namePtr, &valPtr)) {
   ParamText("url", newURL);
   ParamText("type", newType);
 }
 if (newURL[0] && newType[0]) {
   newUrl = true;
   strncpy(url, newURL, sizeof(url)-1);
   url[sizeof(url)-1] = 0;
   strcpy_P(status, PSTR("Changing URL..."));
   Serial.printf_P(PSTR("Changed URL to: %s(%s)\n"), url, newType);
   RedirectToIndex(client);
 } else {
   WebError(client, 404, NULL, false);
 }
}
void RedirectToIndex(WiFiClient *client)
{
 WebError(client, 301, PSTR("Location: /\r\n"), true);
}
void StopPlaying()
{
 if (decoder) {
   decoder->stop();
   delete decoder;
   decoder = NULL;
 }
 if (buff) {
   buff->close();
   delete buff;
   buff = NULL;
 }
 if (file) {
   file->close();
   delete file;
   file = NULL;
 }
 strcpy_P(status, PSTR("Stopped"));
}
void HandleStop(WiFiClient *client)
{
 Serial.printf_P(PSTR("HandleStop()\n"));
 StopPlaying();
 RedirectToIndex(client);
}
// Called when a metadata event occurs (i.e. an ID3 tag, an ICY block, etc.
void MDCallback(void *cbData, const char *type, bool isUnicode, const char *string)
{
 const char *ptr = reinterpret_cast<const char *>(cbData);
 (void) isUnicode; // Punt this ball for now
 // Note that the type and string may be in PROGMEM, so copy them to RAM for printf
 char s1[32], s2[64];
 strncpy_P(s1, type, sizeof(s1));
 s1[sizeof(s1)-1]=0;
 strncpy_P(s2, string, sizeof(s2));
 s2[sizeof(s2)-1]=0;
 Serial.printf("METADATA(%s) '%s' = '%s'\n", ptr, s1, s2);
 Serial.flush();
}
void StatusCallback(void *cbData, int code, const char *string)
{
 const char *ptr = reinterpret_cast<const char *>(cbData);
 (void) code;
 (void) ptr;
 strncpy_P(status, string, sizeof(status)-1);
 status[sizeof(status)-1] = 0;
}
#ifdef ESP8266
const int preallocateBufferSize = 5*1024;
const int preallocateCodecSize = 29192; // MP3 codec max mem needed
#else
const int preallocateBufferSize = 16*1024;
const int preallocateCodecSize = 85332; // AAC+SBR codec max mem needed
#endif
void *preallocateBuffer = NULL;
void *preallocateCodec = NULL;
void setup()
{
 // First, preallocate all the memory needed for the buffering and codecs, never to be freed
 preallocateBuffer = malloc(preallocateBufferSize);
 preallocateCodec = malloc(preallocateCodecSize);
 if (!preallocateBuffer || !preallocateCodec) {
   Serial.begin(115200);
   Serial.printf_P(PSTR("FATAL ERROR:  Unable to preallocate %d bytes for app\n"), preallocateBufferSize+preallocateCodecSize);
   while (1) delay(1000); // Infinite halt
 }
 Serial.begin(115200);
 delay(1000);
 Serial.printf_P(PSTR("Connecting to WiFi\n"));
 WiFi.disconnect();
 WiFi.softAPdisconnect(true);
 WiFi.mode(WIFI_STA);
 WiFi.begin(SSID, PASSWORD);
 // Try forever
 while (WiFi.status() != WL_CONNECTED) {
   Serial.printf_P(PSTR("...Connecting to WiFi\n"));
   delay(1000);
 }
Serial.printf_P(PSTR("Connected\n"));
 Serial.printf_P(PSTR("Go to http://"));
 Serial.print(WiFi.localIP());
 Serial.printf_P(PSTR("/ to control the web radio.\n"));
 server.begin();
 strcpy_P(url, PSTR("none"));
 strcpy_P(status, PSTR("OK"));
 file = NULL;
 buff = NULL;
 out = new AudioOutputI2S();
 decoder = NULL;
 ESP8266SAM *sam = new ESP8266SAM;
 sam->SetVoice(sam->SAMVoice::VOICE_SAM);
 sam->Say(out, "Welcome to Internet Radio powered by So knee Ess Presents!");
 delay(500);
 sam->Say(out, "Let's make some noise!");
 delay(500);
 sam->Say(out, "Connect to me at: ");
 char ip[16];
 sprintf(ip, "%d.%d.%d.%d", WiFi.localIP()[0], WiFi.localIP()[1], WiFi.localIP()[2], WiFi.localIP()[3] );
 sam->Say(out, ip);
 delete sam;    
 delete ip;
 LoadSettings();
}
void StartNewURL()
{
 Serial.printf_P(PSTR("Changing URL to: %s, vol=%d\n"), url, volume);
 newUrl = false;
 // Stop and free existing ones
 Serial.printf_P(PSTR("Before stop...Free mem=%d\n"), ESP.getFreeHeap());
 StopPlaying();
 Serial.printf_P(PSTR("After stop...Free mem=%d\n"), ESP.getFreeHeap());
 SaveSettings();
 Serial.printf_P(PSTR("Saved settings\n"));
 file = new AudioFileSourceICYStream(url);
 Serial.printf_P(PSTR("created icystream\n"));
 file->RegisterMetadataCB(MDCallback, NULL);
 buff = new AudioFileSourceBuffer(file, preallocateBuffer, preallocateBufferSize);
 Serial.printf_P(PSTR("created buffer\n"));
 buff->RegisterStatusCB(StatusCallback, NULL);
 decoder = new AudioGeneratorMP3(preallocateCodec, preallocateCodecSize);
 Serial.printf_P(PSTR("created decoder\n"));
 decoder->RegisterStatusCB(StatusCallback, NULL);
 Serial.printf_P("Decoder start...\n");
 out->SetGain(((float)volume)/100.0);
 decoder->begin(buff, out);
 if (!decoder->isRunning()) {
   Serial.printf_P(PSTR("Can't connect to URL"));
   StopPlaying();
   strcpy_P(status, PSTR("Unable to connect to URL"));
   retryms = millis() + 2000;
 }
 Serial.printf_P("Done start new URL\n");
}
void LoadSettings()
{
 // Restore from EEPROM, check the checksum matches
 Settings s;
 uint8_t *ptr = reinterpret_cast<uint8_t *>(&s);
 EEPROM.begin(sizeof(s));
 for (size_t i=0; i<sizeof(s); i++) {
   ptr[i] = EEPROM.read(i);
 }
 EEPROM.end();
 int16_t sum = 0x1234;
 for (size_t i=0; i<sizeof(url); i++) sum += s.url[i];
 sum += s.volume;
 if (s.checksum == sum) {
   strcpy(url, s.url);
   volume = s.volume;
   Serial.printf_P(PSTR("Resuming stream from EEPROM: %s, type=%s, vol=%d\n"), url, "MP3", volume);
   newUrl = true;
 }
}
void SaveSettings()
{
 // Store in "EEPROM" to restart automatically
 Settings s;
 memset(&s, 0, sizeof(s));
 strcpy(s.url, url);
 s.volume = volume;
 s.checksum = 0x1234;
 for (size_t i=0; i<sizeof(url); i++) s.checksum += s.url[i];
 s.checksum += s.volume;
 uint8_t *ptr = reinterpret_cast<uint8_t *>(&s);
 EEPROM.begin(sizeof(s));
 for (size_t i=0; i<sizeof(s); i++) {
   EEPROM.write(i, ptr[i]);
 }
 EEPROM.commit();
 EEPROM.end();
}
void PumpDecoder()
{
 if (decoder && decoder->isRunning()) {
   strcpy_P(status, PSTR("Playing")); // By default we're OK unless the decoder says otherwise
   if (!decoder->loop()) {
     Serial.printf_P(PSTR("Stopping decoder\n"));
     StopPlaying();
     retryms = millis() + 2000;
   }
}
}
void loop()
{
 static int lastms = 0;
 if (millis()-lastms > 1000) {
   lastms = millis();
   Serial.printf_P(PSTR("Running for %d seconds%c...Free mem=%d\n"), lastms/1000, !decoder?' ':(decoder->isRunning()?'*':' '), ESP.getFreeHeap());
 }
 if (retryms && millis()-retryms>0) {
   retryms = 0;
   newUrl = true;
 }
 if (newUrl) {
   StartNewURL();
 }
 PumpDecoder();
 char *reqUrl;
 char *params;
 WiFiClient client = server.available();
 PumpDecoder();
 char reqBuff[384];
 if (client && WebReadRequest(&client, reqBuff, 384, &reqUrl, &params)) {
   PumpDecoder();
   if (IsIndexHTML(reqUrl)) {
     HandleIndex(&client);
   } else if (!strcmp_P(reqUrl, PSTR("stop"))) {
     HandleStop(&client);
   } else if (!strcmp_P(reqUrl, PSTR("status"))) {
     HandleStatus(&client);
   } else if (!strcmp_P(reqUrl, PSTR("setvol"))) {
     HandleVolume(&client, params);
   } else if (!strcmp_P(reqUrl, PSTR("changeurl"))) {
     HandleChangeURL(&client, params);
   } else {
     WebError(&client, 404, NULL, false);
   }
 }
 PumpDecoder();
 if (client) {
   client.flush();
   client.stop();
 }
}

This code removes AAC support and adds an ability to speak the connection information needed to connect to the web portal running on the ESP8266. These modifications leave just enough memory to allow for decoding low-bitrate mp3s on the ESP8266 itself.

Flash the code to your ESP8266 device using these settings:

It is very important that these settings are used as other combinations may result in failure to playback or serve the web portal.

Once flashed, you can connect to the serial port to retrieve the address to connect to the web portal or if your Spresense is plugged into a speaker, you will hear it "speak" the ip address out loud.

Once you connect to the portal via the ip address, you can configure the URL to use for MP3 playback.

Once an mp3 has been successfully set, the device will store that URL in the EEPROM of the ESP8266 and will begin playing it on successive restarts of the ESP8266 device. If this ever causes issues, you can always reflash the device to erase all EEPROM contents and start over fresh.

It is important to note that due to limitations of the ESP8266 processing power and memory availability, it may not be possible to playback mp3s encoded at certain bitrates. However, here are a few that you can try which should work:

• http://meuk.spritesserver.nl/Ii.Romanzeandante.mp3
• http://www.hochmuth.com/mp3/Haydn_Cello_Concerto_D-1.mp3
• http://www.hochmuth.com/mp3/Tchaikovsky_Rococo_Var_orch.mp3
• http://www.hochmuth.com/mp3/Vivaldi_Sonata_eminor_.mp3
• http://www.hochmuth.com/mp3/Tchaikovsky_Nocturne__orch.mp3
• http://www.hochmuth.com/mp3/Haydn_Adagio.mp3
• http://www.hochmuth.com/mp3/Boccherini_Concerto_478-1.mp3
• http://www.hochmuth.com/mp3/Bloch_Prayer.mp3
• http://www.hochmuth.com/mp3/Beethoven_12_Variation.mp3

Conclusion

Using some assistance from our ESP8266 device, we were able to bring the Sony Spresence device to life with connectivity to the internet to allow for streaming mp3 content from arbitrary sources. This could have been solved in a variety of ways, and could certainly be improved upon. For example, instead of decoding the mp3s on the ESP8266, we could free up some CPU and memory by sending the encoded mp3 data over I2C to be decoded in-stream by the Sony Spresense. That said, we have shown that using serial communication and other protocols, we can easily assist non-internet capable devices.

We also demonstrated some cool features like adding speech synthesis using the ESP8266SAM library. This could be re-leveraged for a variety of projects, for example you could speak out sensor readings or weather information from text received from a web request. While the voice isn't exactly hi-fidelity, it does the job well enough and is notable when considering the limited resources required to do it. It is also impressive to note that after supplying libraries to decode mp3s and enable speech synthesis that we still have room to provide a rudimentary web server for configuring the ESP8266 at runtime.

While the project itself is cool on it's own, I felt that I learned a lot of new skills during this project that can certainly lend use to others. If you want to make something better by adding internet connectivity, I2S communication, dynamic speech, or a light-weight configurable server, the techniques demonstrated in this project are robust enough to allow for improvements to many IoT projects.

Hopefully, you are inspired to take some of this information and recreate it as-is or take the pieces you like to make your projects do something you didn't think of before. Let me know in the comments if you have any ideas of examples of using these techniques in other areas!

Until next time,

Happy Hacking!