Voice Effects Controlled with Gesture Motions, Ada on STM32

Introduction

Hello everyone, welcome to my entry to the Make with ADA 2019 contest. I will be sharing all the story of how I got started using Ada (this is my first project using Ada), how I thought about the parts and how I developed some examples of the modules used. If you want to read directly about the project, I suggest you going to the Final Project Description section.

There it will be written the whole structure of the project, and how this project can be used as an example to implement generic audio effects manager using ADA. In the Ada Implementation Highlights, it will be explained things I thought are great features of Ada to design embedded systems.

At the end of the post, some videos will be shown, with the complete project, conclusions about the whole project achievements, possible improvements, and some thoughts about the future of Ada in embedded development. Hope you enjoy this journey!

Motivation

At this point of my career, I've been wondering how to give "the next step", as a firmware engineer working as a freelancer for embedded custom solutions, I usually find my own projects, I work using C, C++ or Python. Most of the times I work in "little gigs", and my focus is usually getting a prototype working as soon as possible. I decided I wanted to start learning about RTOS, to get started with more complex projects, maybe to start working for companies in more interesting and bigger things. It curiously happened that by that time, the make with Ada contest appeared on Hackster, I saw the brief description about Ada, and the fact that it "includes somehow" a mini RTOS, that is not machine dependent really caught my attention. I thought to myself, " if all these features are true, then Ada would be the dream for embedded software development. Some people that know me might want to tell you that I joined the competition to try winning 5k$, or to try meeting interesting people working in embedded firmware in France, but don't hear them, I am doing this for the fun ;)

About the project selection, I saw the description of the STM32F407 Disco board, and I wanted to work in something related to audio (because the board has nice audio peripherals already). I decided to work on a voice changer because there aren't too much of these around. They exist, however, most of these are implemented for a specific platform, or for a specific family of chips. I thought that using a language with abstraction of low level features might be a good place to work in an audio effects library. I did my best so that the code presented here can be easily ported to other platforms with Ada with ease and hopefully, this project someday becomes the foundation to something as cool as the teensy audio library (but usable on a lot of different processors).

Development history

Getting started with Ada

So, where to get started with Ada? The Getting Started page of the contest provides a lot of documentation to start learning Ada. I like learning by reading and then practicing using examples. As a complete beginner, I thought the https://learn.adacore.com/ explained well the features, but I wasn't able to learn from there. I used this old site to learn the basics of Ada: Ada 95 tutorial. It only exposes Ada 95 features, but I thought it had a lot of examples which helped me learn. Once I learned the basics of the language, the vision of the packages, and how generics worked I thought the Learn Adacore site it is quite well explained and started browsing the Gnat + Ada_Drivers_Library examples by checking the learn site every time I had doubts about the language. I also liked the AdaCore U videos, but I didn't use them much as I don't like learning videos. I didn't check the "Ada for C++ developers" guides and similar, because I think it was going to teach me how to program in C using Ada, and I wanted to give it a shot taking the time to learn Ada (if you followed the approach of reading these guides, please let me know if it was a good approach, learning Ada from scratch took me some time).

Getting started with Gnat + STM32

I checked the libraries before learning how to program using Ada, and it was almost impossible to follow the examples. However, after understanding the language the implementation of everything looked somehow natural. This post also helped me getting started with the library: Getting started with the Ada drivers library

A good selection of the names, it makes unnecessary writing comments (in most of the cases), and the Gnat Community Editor works really well, it finds libraries in your project way easier than any other IDE I had tested before. It's probably related to the fact that you define your own types when writing libraries, so the project can search and decide which procedure you are trying to use under different contexts easier. Luckily there was an audio example with the board I had for the development of the project, so at least getting started with the board was somehow easier than learning the language.

I decided I was writing some example projects, to help others with problems getting started. I browsed some of the projects of previous years, but the projects were presented with the complete code, and it was difficult to understand the parts separated without getting a little involved with the whole project. The path to my "examples" projects can be found in this link: Ada peripheral examples.

I started writing the ILI9341 example. I used the ILI9341 library and extended it adding some procedures to print strings. I used SPI to communicate with the screen without LTDC features. After understanding how the SetPixel function works, it was easier to extend the other functions.

Then I started acquiring microphone signals. I wanted to use the onboard microphone, but the information is not directly available in PCM but requires a lot of filtering first. As this was a core element of the project, I didn't want to invest a lot of time reading about the filter and decided to use an ADC based microphone. The adc_to_dac, mic_to_dac, adc_to_dac and simple_mic_to_dac example will help anyone going with a similar project. The way I envisioned the project, I required to get the samples using a DMA, with ADC interrupts. So the final simple_mic_to_dac function exposes this feature in simple terms.

As the last feature, I implemented the gesture recognition module. This was initially optional, but I got enough time to do it. The gesture test example can be checked for a simple barebones of this module being used. I implemented it by combining the Sparkfun and Adafruit libraries used for apds9960 modules (I found out the Sparkfun initialization part was better, and the Adafruit library gesture recognition algorithm was way simpler).

Final project description

The final project is the one possible implementation for an "Effects Manager": One task that, based on user inputs, selects among different audio effects and applies them to the audio input coming from a microphone. Here it is a diagram of the architecture of the system:

The system runs two tasks, one to handle the audio processing, and one to handle the user input. The only type of communication required is the selection of the effect module, the main task uses the effect selected by the gesture manager. This effect manager is able of pausing/playing the audio (via DAC control, or disabling ADCacquisition), it controls the screen, the volume of the audio, and user gestures. It also enables/disable the gesture module, because using it under normal circumstances introduces a lot of noise in the microphone input.

The project includes the following features:

• TFT pretty GUI, with volume status, the name of the effect, and status of the gesture module (enabled/disabled). Also nice transitions between status, swiping left starts a transition that draws to the screen from right to left, and swiping right starts a transition that draws to the screen from left to right.
• Gesture module recognition: Up gesture increases volume, Down gesture decreases volume, Left gesture transitions to the previous effect, right gesture transitions to the next effect.
• STM32F4 Disco user button is used to enable/disable the gestures module.
• STM32F4 Disco LEDs are turned on based on the gestures used.
• Audio acquisition with generic sized blocks. Enabling changing with ease the size of the block to be processed, based on the type of effects, and user preferences.
• Audio output using DMA, freeing the main task to perform complex effects.

Hardware details

I built a little breaboard to have all the modules in a single place, however, the only connections made are between modules, there is not a special schematics involved. I used an additional 5v to 3v voltage converter to use it as a power supply of the modules to avoid any problem with the main Discovery board.

Here is a link to the list of the connections used:

And here a couple of photos of the breadboard prototype:

Some ADA implementation highlights I found

I wanted to share some of the things that I liked about working with Ada, that I found useful that I didn't found in similar C based applications.

Tasking methods

Working using C applications for embedded devices, requires you to have significant knowledge and understanding of the host platform to be able of implementing things like processes, interrupts, and memory based peripherals. With Ada, I was able to set up a tasking project in about a couple of hours: just add another task, reference it in the main file, and you are good to go. For more complex systems, it is always required some sort of synchronization semaphores/mutex, but for my project, I thought that tasking was supposed to be easy to implement... and gratefully, it was! Just a couple of "Suspension Objects", shown in the interrupt examples, and I was good to go. I can bet that trying to do this same project using C, would require more time to be achieved without problems.

Generic types and packages

The concept of having generic types and elements allows changing several parts of the project without changing just a couple of things. For example, by changing a "size_buffer" variable, all the effects will be applied to a different sized array. I was amazed by the implementation of the echo effect, that required to shift the array elements after every block. That part of the code goes like this:

     -- Erase old values from samples buffer
     This.saved_samples(1 .. This.saved_samples'Last-Samples_To_Process'Last) := This.saved_samples(Samples_To_Process'Last+1 .. This.saved_samples'Last);
     -- Save new samples
     This.saved_samples(This.saved_samples'Last -Samples_To_Process'Last +1 .. This.saved_samples'Last) := data_to_save;

That part of the code works for any size of block, and I didn't have to define a lot of macros or variables tracking the size of the arrays to get that going. The generics part of the language is great, as this code structure can be used for time-based effects, as FFT implementations, changing the size of the buffered blocks as needed without much effort.

Enforcing professionally written librarieswith packages and contracts

Some may think of this as a feature, some may think about as a con. Take a look at this part of the code of the Sparkfun Apds9960 library:

 if( !wireReadDataByte(APDS9960_ID, id) ) {
       return false;
   }
   if( !(id == APDS9960_ID_1 || id == APDS9960_ID_2) ) {
       return false;
   }
   /* Set ENABLE register to 0 (disable all features) */
   if( !setMode(ALL, OFF) ) {
       return false;
   }
   /* Set default values for ambient light and proximity registers */
   if( !wireWriteDataByte(APDS9960_ATIME, DEFAULT_ATIME) ) {
       return false;
   }
   if( !wireWriteDataByte(APDS9960_WTIME, DEFAULT_WTIME) ) {
       return false;
   }

Basically, in a lot of continuous lines of the code, the result of an I2C operation is checked, to stop the operation of the gesture initialization. It is a lot of code, that does the same thing, and what if it fails? the programmer does not have any way to know where it failed, or how. Consider now, my implementation of the I2C write function in Ada:

  procedure I2C_Write(This  : in out APDS9960_Device;
                      Reg   : UInt8;
                      Value : UInt8;
                      Status : out I2C_Status)  is
  begin
     This.Port.Mem_Write
       (Addr          => Apds9960_I2C_Addr,
        Mem_Addr      => UInt16 (Reg),
        Mem_Addr_Size => Memory_Size_8b,
        Data          => (1 => Value),
        Status        => Status);
     if Status /= Ok then
        raise Program_Error with "I2C write error:" & Status'Img;
     end if;
  end I2C_Write;

With this function, an error is raised, and you get to know where the program was caused based on your exception routines. Then the main code is clearer because it does not have a lot of conditions checking the I2C result. Some may think that taking some time writing in terms of contracts, either by specifying it using specific type variables, or writing pre-post conditions for the software should be a must if we ever want to have a world where "reset your device that is working oddly" is not something normal.

Current status of the project

I am updating my code, with all the core features working: effect selection using gestures, enable/disable gesture mechanisms using a button, audio effect processing and screen control. I am uploading the code with 4 effects:

• Almost raw: removes the theoretical offset of the signal, and applies a gain.
• Only voice: applies a dc blocker filter, a biquad low pass filter of 5khz, and applies a gain.
• Echo: applies a dc blocker filter, and applies two echoes to your voice, one after half a second, and one after a second.
• Robot: Your voice sounds like a robot :D
• Pitch shifter: your voice sounds like a chipmunk.

At the moment, the pitch shifter effect is not working, and the Only voice effect sometimes causes oscillations based on the input. Pitch shifter effect is included in the project, but in the output is replaced with Almost Raw effect.

Other than that, the system works quite great. I noticed that the gesture sensor adds a lot of noise to the output, so I added the button function to enable it when it is required to control the effect to be used, and then disable it when it is configured as desired.

Currently, when the last_chance_handler function is called, there is a System.Address shown onthe screen. I wanted to follow this procedure: Stack Overflow Link to output the complete error message but got short of time before deadline.

The cool thing about the system, is that you can write your own effect routines, just need a reset function (for internal variables of your effect), and a process_block function to write your own effects, and add them to the system. Hopefully is someone is interested in this project, it can be continued. I saw some synthesizers using Ada but did not find any voice changer so this might be a good time to start.

Media of the features and results

Standard Effect screen:

• 1: Effect name
• 2: Volume Bar
• 3: Gesture reception, G-ON is enabled, G-OFF is disabled
• 4: RDY message. If RDY appears, system is processing effects

Demo of features

Board start introduction message

Gesture recognition

Reducing noise turning off gesture sensor

Voice changer: Echo

Voice changer: Robot

Last chance handler (starting the board without gesture sensor connected)

Conclusions

I had a great time working in this project, I learned Ada, and tested its potential to be used in embedded platforms. The Gnat Community IDE is really well developed, and it actually helps you find the libraries you want to use, and provides informative error notifications in most of the cases.

I think that actually writing firmware is easier in Ada than in C. Sadly, as it is not the current "standard" then you may need to expend additional time writing firmware libraries not developed yet in Ada. However, the expectations I was not disappointed by the language expectations I had, is actually well thought for embedded development.

The project developed has modular and reusable packages, and it is published with examples of these using complete projects that work. So anyone could be testing the modules to be included in their projects before using them.

It is presented a generic structure to apply audio effects to generic sized blocks of information. This structure can be replicated for any kind of effects, and even be used for different type of analysis of ADC generic signals. Licensed using GPL3, complete code on GitHub, is expected to be expanded by other users of Ada, or even integrated to Ada_Drivers_Library some day.

Here are some great sources that can be useful for extending the effects including in this version of the project:

DSP for Audio Applications

DSP Labs, audio effects (took the robot idea from here)

Hoping you enjoyed this project, please leave me any type of feedback. I'd be glad to answer any questions you have about my project, and helping you use the software of it, if you want to integrate it in your own applications.