Hardware components | ||||||
 |
| × | 1 | |||
| × | 1 | ||||
Software apps and online services | ||||||
| ||||||
| ||||||
_4YUDWziWQ8.png?auto=compress%2Cformat&w=48&h=48&fit=fill&bg=ffffff) |
| |||||
This project aims at using the WIZ750SR Serial to Ethernet module from Wiznet for an IoT application of connected musical instruments with MIDI capability.
As a successful illustration, a digital piano (Casio CDP100) could be successfully controlled and polyphonic streams could be sent to the device using simple IP socket and threads implemented in Python, from both a PC as well as an android smartphone. A simple voice controlled application has also been designed and successfully tested with the device.
Note - I have updated all the videos with a voice over and also added a couple of videos to show the complete steps of programming the device and creating the python project from scratch on a PC using IntelliJ IDEA.
Hardware DesignAs part of this work, a basic hardware has been built (prototyped manually) as the design was simple to assemble and test manually.
However, a PCB module has also been designed in KiCAD and the Schematic, Layout and Gerber files have been shared as part of this project for prospective users if they would want to build a professional module as part of their designs.
Below is a snapshot of the manual prototype used in this project.
The snapshot of the schematic and the basic blocks of the design are described below.
- MIDI Interface – Opto-isolator, and DIN connectors. For a 3V system, per the latest MIDI v1.1 specification, only the resistor values need to be updated for maintaining the 5mA loop current of the opto-isolator.
- WIZ750SR TTL module interface headers.
- A Lithium Polymer Battery and charging circuit has been designed in to facilitate re-charging and using this as a modular design in external applications. But this has not been loaded/tested in this design as a 2XAA battery socket has been used for ease of testing.
- A mechanical enclosure design has not been provided as users could buy off-the-shelf enclosure, and customize, or design their own enclosure.
A 3D snapshot of both the top and bottom sides of the PCB is shown below.
The Firmware design for this application is pretty simple and needed only a minor update to the firmware source provided by Wiznet to add the UART baud rate of 31250 to the existing design. MIDI operates at the non-standard baud rate of 31250 but the system clock (8 MHz) of WIZ750SR module provides an exact integer division (8000000/(16*31250) = 16) for easily generating 31250 baud rate.
The complete software and hex files can be downloaded from the link provided below.
The FW can be upgraded by using the steps provided by Wiznet on the Wiki link below.
https://wizwiki.net/wiki/doku.php?id=products:wiz750sr:developers:start
Important notes
- After programming, it is also imperative to configure the WIZ750SR module to an IP address that is within the range of the network that it is being connected to, and the device must also be set with the same subnet mask and the DNS server address as the rest of the devices that are connected to the network. The IP address should preferably be a static one as it makes it easier to identify the device consistently, and to label multiple WIZ750SR modules with successive IP addresses for ease of identification and control.They could be labelled with a unique Label ID too, if required.
- The users can use the WizConfig tool for configuring the IP address of the module. On the WizConfig tool, for the data UART the baud rate must be selected as 38400 (as I have added 31250 as an additional baud rate value ahead of 38400 in the structure for baud rate in the FW header file "uartHandler.h") as shown below. On the Wiznet config file, when we select 38400 as our baud rate it actually updates the baud rate as item 10 in the list which is 31250. This is the same reason for the default baud rate value of 115200 to be showing up as 57600 on the serial terminal window. 115200 will actually be skipped or not set. Since there is not option to edit the Wizconfig tool I had to use this back entry method.
enum baud {baud_300 = 0,
baud_600 = 1,
baud_1200 = 2,
baud_1800 = 3,
baud_2400 = 4,
baud_4800 = 5,
baud_9600 = 6,
baud_14400 = 7,
baud_19200 = 8,
baud_28800 = 9,
baud_31250 = 10,
baud_38400 = 11,
baud_57600 = 12,
baud_115200 = 13,
baud_230400 = 14,
baud_460800 = 15
};
I have provided the FW bin file (MIDI baud rate supported revised Firmware) in the attachment section to program the module directly using the WizISP tool.
Here is a demonstration video on how to program the WIZ750SR module with the updated FW bin file, and set the desired IP address, port no and baud rate for the device.
Application Software DesignSoftware Dependencies:
- Python for PC - IDE (IntelliJ IDEA or PYCharm),
- Python for Android - Pydroid 3 IDE
- Python modules - "KiVy", "SpeechRecognition", "pyaudio", "playsound". Install these from the command prompt or terminal using the command "pip install <Module Name>"
A GUI control as well as a voice control based application has been designed and built in Python demonstrating the capabilities of the module to connect to an instrument (Casio Digital Piano, in this instance) and polyphonic streams have been successfully sent using simultaneous Python threads and played by the instrument over the connected network. The application has been tested successfully using both a personal computer as well as an android smartphone.
A two-pronged approach has been taken at the start of the software development and complete source files for both the design approaches have been provided in the link below.
https://github.com/sriharisrinivasan/WIZ750SR_MIDI_Control
I have created Python threads to send multiple channel streams to MIDI device. The code automatically creates a thread for polyphony based upon the no of channels selected. A snippet of the code in the parser python file "parse_music.py" is provided below.
if Sys_Variables.channel_sel == 0:
# Generate a thread for first channel only
thread0 = threading.Thread(target=send_note_on, args=(0,))
elif Sys_Variables.channel_sel == 1:
# Generate a thread for second channel only
thread1 = threading.Thread(target=send_note_on, args=(1,))
elif Sys_Variables.channel_sel == 2:
# Generate a thread for third channel only
thread2 = threading.Thread(target=send_note_on, args=(2,))
elif Sys_Variables.channel_sel == 3:
# Generate a thread for fourth channel only
thread3 = threading.Thread(target=send_note_on, args=(3,))
else:
# Generate threads for all 4 channels
thread0 = threading.Thread(target=send_note_on, args=(0,))
thread1 = threading.Thread(target=send_note_on, args=(1,))
thread2 = threading.Thread(target=send_note_on, args=(2,))
thread3 = threading.Thread(target=send_note_on, args=(3,))
# Start the individual or simultaneous threads for the 4 channels
if Sys_Variables.channel_sel == 0:
thread0.start()
elif Sys_Variables.channel_sel == 1:
thread1.start()
elif Sys_Variables.channel_sel == 2:
thread2.start()
elif Sys_Variables.channel_sel == 3:
thread3.start()
else:
thread0.start()
thread1.start()
thread2.start()
thread3.start()
# Join the individual or simultaneous threads for the 4 channels
if Sys_Variables.channel_sel == 0:
thread0.join()
elif Sys_Variables.channel_sel == 1:
thread1.join()
elif Sys_Variables.channel_sel == 2:
thread2.join()
elif Sys_Variables.channel_sel == 3:
thread3.join()
else:
thread0.join()
thread1.join()
thread2.join()
thread3.join()
The following is a "Sys_Variables.py" python file which contains all the global variables such as IP address & Port of the module, no of channels, list of instruments, list of scale supported etc. and users can edit this file (as well as the other python files) to customize this code for supporting any MIDI instruments that they may have.
#Change os to "Android" if you want to run this from mobile
# os = "Android"
#Change os to "Windows" if you want to run this from Windows PC
os = "Windows"
# You can change this Android path as per your mobile storage
Android_Path = '/storage/emulated/0/MIDI_Test'
scale_list = ["Select Scale", "Major Scale", "Minor Scale", "Chromatic Scale"]
pitch_list = ["Select Pitch","C", "Cs", "Df", "D", "Ds", "Ef", "E", "F", "Fs", "Gf", "G", "Gs", "Af", "A", "As", "Bf", "B"]
instrument_list = ["Select Instrument", "Piano 1", "Piano 2", "Electric Piano", "Harpsichord", "Strings" ]
scale = "MAJOR"
pitch = "C"
instrument = "Piano 1"
chan0_inst = "Piano 1"
chan1_inst = "Piano 1"
chan2_inst = "Piano 1"
chan3_inst = "Piano 1"
IP_Address = "192.168.0.81"
Port = 5000
music_filename = ""
channel = 0
note = 60
chord_type = 'MAJ'
octave = 4
duration = 1
velocity = 100
tempo = 88
beatspermeasure = 4
measure = 4
no_of_tracks = 4
channel_sel = 0
Create a new Python Project using IntelliJ IDEA tool and the source files from the Github link provided
Method 1 (Applicable to both PC & Android Smartphone) – An interactive GUI based MIDI control applicationThe GUI has the following features. A detailed video demonstration showing the usage is also presented below.
- Provision to enter the IP address and port for the WizNet module to be connected through an IP socket. Alternatively, the values can be edited in the “Settings.py” file provided among the source files.Please refer to the variables “IP Address” and “Port” inside the mentioned file.
- Scale Selection – Major, Minor & Chromatic
- Pitch Selection – All pitches on the piano
- Instrument Selection – CDP100 currently supports 5 voices or instruments. All the instruments are available via selection using the GUI.
- Option to load and play a custom text based music input file directly on the MIDI device with option to select same or different voices for the individual tracks for polyphonic music.
- A custom text based music input file format and .a parser has been developed to allow sending polyphonic tones to the device as simultaneous threads in Python. This is considered as an interim level file format interface for facilitating HW/SW interfacing between an available and well developed Open Source application like Lilypond and the MIDI device (interface using WIZ750SR module). This has been described in detail in the next section.
A video demonstrating the setup and use of the GUI based control from both PC and Android mobile phone is provided below.
Method 2 (Applicable to only PC) – Voice based MIDI Control using the WIZ750SR moduleUsing the Google speech recognition API in python, a module was developed to send voice commands and control the Digital Piano. Only simple commands for playing scales has been developed so far.
The user needs to utter the keyword "MIDI Master" for the application to respond. The user should then ask "Check for Devices" and the application would recognize it and search for the WIZ750SR module in the specified IP address and port and if the device is available and could be successfully connected to, the application will ask for further actions. A video demonstrating the setup and use of the voice based control is provided below. This demo is using an old Python application which has only 2 channel support. However, it does not change anything related to the voice command control between the 2 projects.
An attempt was made to run the same module in Android too but it could not be successful so far due to the unsuccesful installation of “PyAudio” module in PyDroid3 android app. Since installing Audio modules for controlling audio devices like microphone and speaker was also unsuccessful, it was decided not to proceed with this approach for Android (for now).
MUSTiL – Music Text Interface Language - a text based music input file formatA text based Music Interface language has been developed to decode and stream multiple tracks to lists and send them in parallel to the WIZ750SR module from Python application using threading.
The commands available in MUSTiL have been provided in detail below.
a) Commenting:
The comments must be preceded by a ‘#’ at the beginning of the line.
Example: # A Prelude in G-Minor by Johann Sebastian Bach
b) TimeSignature:
The TimeSignature command must be in the following format.
TS(<beats per measure>, <base noteduration for BPM>)
Example: TS(4, 4) refers to quarter note as base note duration for BPM, and 4 beats per measure -the typical 4/4 time signature
c) Tempo:
The Tempo command must be in the following format.
TMP(<tempo value>)
Example: TMP(88) - 88 beats per minute
d) Polyphonic:
The Polyphonic command must be in the following format. A file can only have POLY or MONO command and not both
POLY(<no of tracks>)
Example: POLY(2) indicates there are 2 tracks in the file.
e) Monophonic:
The Monophonic command must be in the following format. A file can only have POLY or MONO command and not both
MONO
Example:MONO.
f) Clef:
The Clef command must be in the following format.
CLEF(Type)
Example: CLEF(TREBLE) indicates the track preceding this command is in the Treble clef. 4 different clef types supported – Treble, Alto, Tenor and Bass
g) Track Start:
A “{“ signifies the beginning of the track or a clef
Example:
{
SN(E2, 8)
h) Track Stop:
A “}“ signifies signify the ending of the track or a clef
Example:
SN(C5, 16)
SNV(Fs4, 8, 98)
}
i) Velocity:
The Velocity command must be in the following format.
VEL(Velocity value)
Example: VEL(100) indicates that the note preceding this command must be played with a intensity of 100 in a scale of 0 to 255.
j) Send Note with Velocity:
The Send Note with Velocity command must be in the following format.
SNV(<NoteNo>, <Duration>, <Velocity>)
Example:
SNV(C4, 4, 115)
Send Note -C4, Duration - Quarter Note, Velocity - 115 and the velocity is applicable to all notes after this command unless over ridden by another SNV or VEL command
k) Send Note:
The Send Note command must be in the following format.
SN(<Note No>, <Duration>)
Example:
SN(D3, 8)
Send Note -D3, Duration - Eighth Note, Velocity - previously set velocity (from either VELor SNV commands whichever is the latest)
l) Send Rest:
The Send Rest command must be in the following format.
SR(<Duration>)
Example:
SR(1) indicates a Rest for Duration of Whole Note
m) Send Multiple Notes (for sending chords):
The Send Multiple Notes command must be in the following format.
SMN(<No of Notes ‘n’>, <Note 1>, <Note2>, …, <Note ‘n’>, <Duration>)
Example:
SMN(3, C2, E2, G2, 8)
Send the 3 notes C2, E2 and G2 or the C Major Chord triad for a duration of the eighth note
n) Stop:
The STOP command must be in the following format.
STOP – This signifies the end of the file
ConclusionThis project opens up a lot of interesting applications particularly in automating music production. Port forwarding has not been tested and hence, once it is successful, we could control the MIDI instrument using our mobile or tab from any part of the world.
The interesting applications for such a use case will be like anti-theft system (creating an impression that someone is present by playing a music over the internet on the instrument), composing and testing the music over air etc.
I have not tested the MIDI output from the device, and using the MIDI output (feedback) from the device, we could run a script on the remote computer to generate sheet music from a live performance on connected instruments. We could do music performance evaluation by feedback from MIDI. The potential applications are endless.
Comments