Published December 26, 2020 © CC BY-NC-SA

Self-Playing Melodica 🤖🎹

Turn your melodica into a compressed-air melodica controlled by Max and Arduino. It tries to follow the melodic contour it listens to!

AdvancedFull instructions provided16 hours5,103

Things used in this project

Hardware components

Arduino Nano R3

Resistor 1k ohm

SparkFun Dual H-Bridge motor drivers L298

Pushbutton Switch, Momentary

Digilent 60W PCIe 12V 5A Power Supply

Voltage regulator, step-down, 12V in, 3.6V out

Bistable electrovalve (solenoid)

DC Motor, 12 V

Software apps and online services

Arduino IDE

Cycling '74 MAX

Story

Disclaimer: this project aims at building an inaccurate self-playing melodica. If you want to build an accurate one, use a DC motor with encoder (best option) or a stepper motor. You will at least be able to make it play diatonic (white keys) melodies the right way.

For my project "AI've just started to learn to play" I built a robot melodica which listens to fragments taken from YouTube melodica tutorials and tries to reproduce the melody in real time. The robot was active at Incó_ntemporanea Festival in Piacenza, Palazzo Ghizzoni Nasalli, for 24 hours (October 11th, 2020). Four musicians from the local Collettivo_21 were present for the whole 24 hours and jamming/reacting to the robot. Few audience members were allowed in special times of the day, while the whole event was streamed live on the estival's website and on Twitch. In the following video you can see the robot jamming with the musicians as well as some excerpt from the Twitch streaming:

Collettivo_21: Giulia Pastorino / Daniele Nava / Lorenzo Moretto / Davide Bussoleni

How does it work?

Technically the robot consists of a DC motor savaged from an old printer (also keeping the original scaffolding structure), mounted over a melodica and controlled via Arduino. Arduino also controls a bistable electro-valve, which is letting through, when necessary, the air flow coming from a compressor.The Arduino sketch makes it possible, for the robot, to receive a control value from the computer. The control value describes a combination of control states: air valve open or closed, and DC motor still, moving upwards or moving downwards. Two end-stop switches throw back the DC motor to a center position when it reaches the keyboard edges.

Build!

First I salvaged a DC motor from a friend's old printer. I expected a stepper but I got a DC with encoder. I did not want to mess with an encoder and I wanted the result to be quite "wrong", so at the end I decided to go with the DC. (That's also why I decided to use two wheels, resulting in a more cacophonic effect due to the minor seconds).

Two wheels are better than one. ("Ciù uìls ar megl che uàn")

After that, I mounted the motor with all its plastic scaffolding on a wooden structure, so that I could adjust the height and tune it to the melodica dimensions

The two wheels were screwed to the moving part of the printer scaffolding. They were common cart wheels you can buy when you make your own furniture.

1 / 2 • Wheels screwed to the plastic structure.

The wooden structure was made so that the instrument would fit and stay still during the action.

More or less final shape.

Some recycled padding material was glued to the bottom for a final touch.

For a softer bottom.

I prepared the circuit with the Arduino microcontroller, driving the L298. There are several tutorials on how to control a DC moto via L298, for example this one. At the beginning, the basic circuit was meant to drive the motor right and left, as a test. After that, I added two endstop switches on the right and left edges of the wooden structure.

Endstop switch.

This is how the robot worked at this point. I was still using my lungs to play it, but soon I would add an electrovalve controlling the air flow from a compressor. In the video you can also compare the original sound of the tutorial with the wrong-and-lovely rendition of the robot pupil.

The electrovalve dilemma

The last feature I added was an electrovalve. It was used to control the airflow that would reach the melodica, producing the sound. The air source was an air compressor.

I had to use a different L298 for the electrovalve because the one I found was a 3.6V one, so I preferred to be safe and separate the DC motor board and the electrovalve board. Initially, the first electrovalve I used started to melt pretty soon, as solenoids tend to reach very high temperatures when powered for long times (the electovalve needed to be continually powered to be open).

Common electrovalve, 12V, with melting glue due to high temperature.

Then I decided to find a bistable electrovalve. These valves only need a short impulse in order to switch from open to close or vice-versa, so they don't need to be powered for long times. This electrovalve was working at only 3.6V, and fortunately I had a stepdown module I could use to go from the 12V I was already using to power the DC motor to those 3.6V of the electrovalve. If you find a bistable electrovalve which runs at the same voltage of your DC motor, you can of course run them on one only L298, being the L298 a double board.

The valve who solved it all.

Once connected to the L298, this valve opens with HIGH and a LOW signal on the input pins of the board; and closes with the opposite. These values should be sent for at least 30 milliseconds to operate the valve, as you can read in the specs:

Working mode: positive pulse opens valve, negative pulse off.Pulse width: 30 ms.Working pressure: 0.02-1 MPa.Medium temperature: 1-80℃Lifespan: 300, 000 timesSize: 5.5 x 5 cm / 2.17 x 1.97 inches.Weight: 130 g.Wiring method: red: "+", black: "-"

You can set the Arduino to send both LOW signals in order to stop the current flow.

Software

Apart from the Arduino sketch, which you find attached, the listening and controlling software is written in Max and uses the sigmund~ object. The audio analysis of a library of video-tutorials yields the melodic contour (which controls the movement of the DC motor) and the timing for note ons and offs (which control the electro-valve).

It would be impossible to explain the functioning of the Max patch, which is very complex and articulated (it also contains some video processing, streaming control, voting system and much more). But in any case you can send via serial different commands (see below) to control the robot. The patch worked this way:

Video with monodic sound → sigmund~ → differential in pitch and silence detection → time length for DC motor movement and electrovalve control

You are encouraged to try to program it yourself and find your way to solve the many small problems you will have to face.

Basically the Max patch was sending via serial different numbers, as it's clear from the last section of the Arduino sketch, where consecutive if statements are parsing the instructions.

YouTube tutorial (on the right) and the control interface written in Max, showing "1" as current command.

They were the combination of three possible states of the DC motors: steady (1), move left (0) or move right (2). When 100 was added to these numbers, it also meant to close the valve. When 200 was added, it meant that the valve needed to be open. So, for example, message 200 meant 200+0: move the motor left (0) and open the valve (200). Message 101 meant: keep the motor steady (1) and close the valve (100).

* * *

This robot was conceived and realized in the frame of "Ars Cyber != Dystopian", event in collaboration with Claudio Panariello and Luca Guidarini. With the support of Konstnärsnämnden, Ernst von Siemens Foundation, Camera di Commercio Piacenza, Comune di Piacenza, Fondazione di Piacenza e Vigevano.

Code

Final code for the Arduino

const int ledPin = 13;
#define enA 9
#define in1 6
#define in2 5
#define endStopR A0
#define endStopL A1

#define air1  10
#define air2  11



int stateR = 0;
int stateL = 0;

void setup() {
  Serial.begin(9600); // open a serial connection to your computer
  pinMode(ledPin, OUTPUT);
  pinMode(enA, OUTPUT);
  pinMode(in1, OUTPUT);
  pinMode(in2, OUTPUT);

  pinMode(endStopR, INPUT);
  pinMode(endStopL, INPUT);

  digitalWrite(in1, LOW);
  digitalWrite(in2, LOW);
  digitalWrite(enA, HIGH);
}

void loop() {

  stateR = digitalRead (endStopR);
  stateL = digitalRead (endStopL);

  if (stateR == HIGH) {
    Serial.print("Endstop activated:");
    Serial.print("\t");
    Serial.println("RIGHT");

    digitalWrite(in1, HIGH);
    digitalWrite(in2, LOW);

    stateR = LOW;

    delay(500);

    digitalWrite(in1, LOW);
  }

  if (stateL == HIGH) {
    Serial.print("Endstop activated:");
    Serial.print("\t");
    Serial.println("LEFT");

    digitalWrite(in1, LOW);
    digitalWrite(in2, HIGH);

    stateL = LOW;

    delay(500);

    digitalWrite(in2, LOW);
  }


  if (Serial.available()) {

    byte    airMov = Serial.read();  //airMov che è 1, 0 o 2 ma ha anche +100 se airflow è attivo
    Serial.print("command");
    Serial.print("\t");
    Serial.println(airMov);
    byte mov = 0;

    if (airMov > 150) {
      mov = airMov - 200;
      digitalWrite(air1, HIGH);
      digitalWrite(air2, LOW);
    }

    if (airMov > 50 && airMov < 150) {
      mov = airMov - 100;
      digitalWrite(air1, LOW);
      digitalWrite(air2, HIGH);
    }


    if (airMov < 50) {
      mov = airMov;
      digitalWrite(air1, LOW);
      digitalWrite(air2, LOW);
    }





    if (mov == 0) {
      digitalWrite(in1, HIGH);
      digitalWrite(in2, LOW);
      digitalWrite(ledPin, LOW);
    }
    else



      if (mov == 2) {
        digitalWrite(in1, LOW);
        digitalWrite(in2, HIGH);
        digitalWrite(ledPin, HIGH);
      }

      else if (mov == 1) {
        digitalWrite(in1, LOW);
        digitalWrite(in2, LOW);
        digitalWrite(ledPin, HIGH);
      }
  }


}