Johan_Ha
Published © CC BY

Cimini

Chimes that everyone can operate using their own smartphone.

IntermediateFull instructions providedOver 1 day1,583
Cimini

Things used in this project

Hardware components

Arduino MKR WiFi 1010
Arduino MKR WiFi 1010
×1
Pan-tilt device with 9G servos
×1
Various wood and plywood pieces
×1
Copper tube
×2
Copper wire
×1

Software apps and online services

Arduino IDE
Arduino IDE
Generic Web Hosting with PHP and MySQL
ProtoCentral Electronics Tuner app on your phone

Hand tools and fabrication machines

Tube cutter
Drill / Driver, Cordless
Drill / Driver, Cordless
Miter saw
Electric sander

Story

Read more

Schematics

The connections

This is as simple as it gets. There's the power supply for the Arduino. There are two servos to be connected. The servos use same 5 V source as the Arduino. The Arduino can be powered with a 5 V mobile phone power bank, as long as it stays activated by the Arduino power consumption. Or the Arduino could be powered with a phone charger connected to the wall. In my connection, the servos take the power from the +5V pin of the Arduino, while the Arduino takes power from the USB connector.

Code

fetch.php

PHP
This script reads the table in the database and finds the swipe with the lowest ID and outputs the coordinates and the timestamps. If the table is empty, a single letter a is outputted. The event output starts with a single letter b and ends with a single letter c. All the output is in readable html format for debugging purposes. One line contains comma separated data with the x-coordinate, the y-coordinate and the timestamp of one event. A line break is created with <br /> instead of <p></p> to avoid an annoying double line feed, which I got every time I wanted to paste the data from the page to a spreadsheet for analysis.
The MySQL query in 20-22 should include sorting of the data according to the event index number! But my code simply relies on the data being written and read in the same order as the event got created in the first place. It has worked so far.
<html>
 <head>
  <title>List of coordinates</title>
 </head>
 <body>
 <?php 
    

    $servername = "localhost";
    $username = "my_secret_username";
    $password = "my_secret_password";
    $dbname = "my_database_name";

    $conn = mysqli_connect($servername, $username, $password, $dbname);

    $rowSQL = mysqli_query( $conn, "SELECT MIN( id ) AS min FROM `coords`;" );
    $row = mysqli_fetch_array( $rowSQL );
    $smallestId = $row['min'];

    $sql = "SELECT x,y,timestamp FROM coords WHERE id='".$smallestId."' ";

    $result = mysqli_query($conn, $sql);
    
    if (mysqli_num_rows($result) > 0) {
        echo "b<br />\n";
        while($row = mysqli_fetch_assoc($result)) {
            echo $row["x"] . ", " . $row["y"] . ", " . $row["timestamp"] . "<br />\n";
        }
        echo "c<br />\n";
        // Delete the rows just shown!
        $sql = "DELETE FROM coords WHERE id='".$smallestId."' ";
        $result = mysqli_query($conn, $sql);
      
    }
    else
    {
        echo "a<br />\n";
    }


 ?> 
 </body>
</html>

cimini.cpp

C/C++
This is the scetch the Arduino 1010 runs to operate the Cimini. It relies on the software created by Tom Igoe and Federico Vanzati dealing with the WiFi and Internet stuff, as well as on the Servo library.
I've left a lot of lines with serial printing, as well as MyPlot, for debug purposes. They were very handy. I'm happy to discuss them further with anyone.
/*
 *    Cimini, a chimes instrument controlled via 
 *    a smartphone and a website
 */

/*

 This sketch connects to a a web server and makes a request
 using a WiFi equipped Arduino board.

 created 23 April 2012
 modified 31 May 2012
 by Tom Igoe
 modified 13 Jan 2014
 by Federico Vanzati

 http://www.arduino.cc/en/Tutorial/WifiWebClientRepeating
 This code is in the public domain.
 */

/*
 *    TESTING COORDS
 *    xmin = 40
 *    xmax = 940
 *    leftmost bar = 160
 *    rightmost bar = 850
 *    leftbottom = 435
 *    rightbottom = 210
 *    highestpoint = 130
 * 
 */
 
#include <SPI.h>
#include <WiFiNINA.h>
#include <utility/wifi_drv.h>
#include <MegunoLink.h>
#include <Servo.h>

#include "arduino_secrets.h" 
///////please enter your sensitive data in the Secret tab/arduino_secrets.h
char ssid[] = SECRET_SSID;        // your network SSID (name)
char pass[] = SECRET_PASS;    // your network password (use for WPA, or use as key for WEP)
int keyIndex = 0;            // your network key Index number (needed only for WEP)

int status = WL_IDLE_STATUS;
//using namespace std;
// Initialize the Wifi client library
WiFiClient client;
XYPlot MyPlot;
Servo horis, vertis;

// server address:
char server[] = "johanhalmen.xyz";
//IPAddress server(64,131,82,241);

unsigned long lastConnectionTime = 0;            // last time you connected to the server, in milliseconds
unsigned long postingInterval = 5L * 1000L; // delay between updates, in milliseconds

// class for one touch event
class triplet
{
  public:
    unsigned short x;
    unsigned short y;
    unsigned int timeStamp;
    triplet(unsigned short xin, unsigned short yin, unsigned int ts); 
    triplet *next;
    
};

triplet::triplet(unsigned short xin, unsigned short yin, unsigned int ts)
: x(xin), y(yin), timeStamp(ts)
{
  
}

// class to hold one swipe
class batonqueue
{
  public:
    batonqueue(void);
    triplet *queue, *last;  
    void add_instance(unsigned short x, unsigned short y, unsigned int timeStamp);
    void perform(void);
    void servo_action(unsigned short x, unsigned short y);

};

batonqueue::batonqueue(void)
{
  queue = NULL;
  last = NULL;
}

// The RGB LED of the Arduino Mkr 1010 Wifi
#define r_pin 26
#define g_pin 25
#define b_pin 27

batonqueue myQueue;

void failout(void)
{
  // In error cases, blink the RGB LED until power is off
  while (true)
  {
    if (millis() % 1000 > 500)
    {
      WiFiDrv::digitalWrite(r_pin, 1);
      WiFiDrv::digitalWrite(g_pin, 0);
      WiFiDrv::digitalWrite(b_pin, 0);
    }
    else
    {
      WiFiDrv::digitalWrite(r_pin, 0);
      WiFiDrv::digitalWrite(g_pin, 1);
      WiFiDrv::digitalWrite(b_pin, 1);
    }
  }
}


// Add one event to the queue
void batonqueue::add_instance(unsigned short x, unsigned short y, unsigned int timeStamp)
{
  triplet *myNew;
  myNew = new triplet(x, y, timeStamp);
  if (myNew == NULL)
    failout(); // oops, memory is full!
  myNew->next = NULL;
  if (queue == NULL)
    queue = myNew;
  else
    last->next = myNew;
  last = myNew;
  
}

// Start performing the action held in queue
void batonqueue::perform(void)
{
  unsigned int last_stamp, wait_time;
  static unsigned short last_x = 0, last_y = 0;
  unsigned short divisor;
  float dx = 2.5;
  float dy = 2.5;
  int servo_delay = 5;
  float t;
  triplet *i, *j;
  // sweep from some resting middle point
  unsigned long start_time;
  float sqlength, spead, tn, timeadd;

  unsigned long accu_time = 0;
  
  // Adjust the speed. Must not exceed 1 pixel per millisecond.
  // I'm assuming the unit used in the event coordinates is one pixel.
  for(i = queue; i != NULL; i = i->next)
  {
    if (i->next != NULL)
    {
      // calculate length to next point
      sqlength = sqrt(float((i->x - i->next->x) * (i->x - i->next->x) + /// Fixed this!
                 (i->y - i->next->y) * (i->y - i->next->y)));
      // calculate speed to next point, should be 1.
      spead = sqlength / (i->next->timeStamp - i->timeStamp);
      if (spead > 1.)
      {
        // set the speed to 1 by adding time to all the remaining events
        tn = spead * (i->next->timeStamp - i->timeStamp);
        timeadd = tn - (i->next->timeStamp - i->timeStamp);
        for (j = i->next; j != NULL; j = j->next)
          j->timeStamp += (int)timeadd;
      }
    }
  }
  // Now the time stamps have been stretched where needed, to
  // limit the speed to corresponding to 1 pixel per millisecond.
  // THE MAX SPEED IS HARD CODED TO 1 PIXEL PER MILLISECOND. 
  // CHANGE CODE TO ALLOW EASY ADJUST OF MAX SPEED.

  // Move from last position to new start, first horiz, then verti

  if (queue->x > last_x)
  {
    for (float i = last_x; i < queue->x; i += dx)
    {
      servo_action(int(i), 460);
      delay(servo_delay);
    }
  }
  else
  {
    for (float i = last_x; i > queue->x; i -= dx)
    {
      servo_action(int(i), 460);
      delay(servo_delay);
    }
  }
  
  for (float i = 460; i > queue->y; i -= dy)
  {
    servo_action(queue->x, int(i));
    delay(servo_delay);
  }   
  start_time = millis();
  last_stamp = 0;

  // START PERFORMING
  for(i = queue; i != NULL; i = i->next)
  {
    if (i->timeStamp - last_stamp > 40)
    { // divide the steps to 20 ms long steps
      divisor = (i->timeStamp - last_stamp) / 20; // integer division rounds down (hopefully)
      for (int ii = 1; ii <= divisor; ii++)
      {
        t = float(ii) / divisor;
        servo_action(t*i->x + (1.-t)*last_x, t*i->y + (1.-t)*last_y);
        wait_time = t*i->timeStamp + (1.-t)*last_stamp;
        while (millis() < start_time + wait_time);
      }
    }    
    else
    { // just follow the original timesteps, they are small enough
      servo_action(i->x, i->y);
      while (millis() < start_time + i->timeStamp);
    }
    last_stamp = i->timeStamp;
    last_x = i->x;
    last_y = i->y;
    
  }
  // Move down
  for (float i = last_y; i <= 460; i += dy)
  {
    servo_action(last_x, int(i));
    delay(servo_delay);
  }   
  

  // Ok, we're finished with the action. Delete the queue.
  i = queue;
  do
  {
    j = i->next;
    delete i;
    
    i = j;
  }
  while (i != NULL);
  queue = NULL;
}

// This function activates both servos for one coordinate
// This function converts the coordinate data to angle data for the servos
void batonqueue::servo_action(unsigned short x, unsigned short y)
{
/*
 *    TESTING COORDS
 *    xmin = 40
 *    xmax = 940
 *    leftmost bar = 160
 *    rightmost bar = 850
 *    leftbottom = 435
 *    rightbottom = 210
 *    highestpoint = 130
 * 
 */
  if (x < 40) x = 40;
  if (x > 940) x = 940;
  if (y < 130) y = 130;
  if (y > 460) y = 460;

    horis.write(map(x, 40, 940, 180, 10));
    vertis.write(map(y, 130, 460, 45, 3));
}

void setup() {
  WiFiDrv::pinMode(g_pin, OUTPUT);  //GREEN
  WiFiDrv::pinMode(r_pin, OUTPUT);  //RED
  WiFiDrv::pinMode(b_pin, OUTPUT);  //BLUE

  horis.write(90); // First set the start position...
  vertis.write(1);
  horis.attach(4); // ...then start. Servo turns to its start position.
  vertis.attach(5);
  
  //Initialize //Serial and wait for port to open:
  //Serial.begin(9600);
  //while (!Serial) {
    //; // wait for serial port to connect. Needed for native USB port only
  //}

  // check for the WiFi module:
  if (WiFi.status() == WL_NO_MODULE) {
    MyPlot.SetTitle("Communication with WiFi module failed!");
    // don't continue
    while (true) {};
  }

  String fv = WiFi.firmwareVersion();
  if (fv < WIFI_FIRMWARE_LATEST_VERSION) {
    ;
    MyPlot.SetTitle("Please upgrade the firmware");
  }

  // attempt to connect to Wifi network:
  while (status != WL_CONNECTED) {
    MyPlot.SetTitle("Attempting to connect to SSID: ");
    // Connect to WPA/WPA2 network. Change this line if using open or WEP network:
    status = WiFi.begin(ssid, pass);

    // wait 10 seconds for connection:
    delay(10000);
  }
  // you're connected now, so print out the status:
  printWifiStatus();

  MyPlot.SetTitle("My Analog Measurement");
  MyPlot.SetXlabel("Channel 0");
  MyPlot.SetYlabel("Channel 1");
  MyPlot.SetSeriesProperties("ADCValue", Plot::Magenta, Plot::Solid, 2, Plot::Square);
  MyPlot.SetXRange(0, 900);
  MyPlot.SetYRange(0, 900);
  

}

// Get one line from the feed from the fetch.php call
String get_next_line(void)
{
  String hlp = "";
  char c;
  do
  {
    if (client.available())
    {
      c = client.read();
    
      if (c >= 32)
        hlp += c;
    }
  }
  while (c != 10 && c != 13 && client.connected());
  return hlp;
}

// Get the crucial data from one line (x, y, timestamp)
// and add to the queue
void add_to_queue(String line)
{
  int second, first;
  int x, y, timeStamp;
  String hlp;
  
  if (line.endsWith("<br />"))
  {
    second = line.lastIndexOf(',');
    if (second == -1)
      return;
    hlp = line.substring(second+1);
    timeStamp = hlp.toInt();
    hlp = line.substring(0, second);
    first = hlp.lastIndexOf(',');
    if (first == -1)
      return;
    hlp = hlp.substring(first+1);
    y = hlp.toInt();
    hlp = line.substring(0, first);
    x = hlp.toInt();
    myQueue.add_instance(x, y, timeStamp);
    
  }
  return;
}

void perform_the_queue(void)
{
  MyPlot.Clear();
  myQueue.perform();
}

void loop() {
  String line;
  char mode = 0;  // 0: read line
                  //    if a<br />
                  //      make a new request (mind waiting)
                  //    if b<br />
                  //      set mode to 1, reset queue
                  // 1: read lines until c<br />
                  //    if line has two commas and end with <br />,
                  //      add to queue
                  //    if c<br />, set mode to 2
                  // 2: perform the queue
                  //    make a new request
                  //    set mode to 0
  WiFiDrv::digitalWrite(r_pin, 1);
  WiFiDrv::digitalWrite(g_pin, 0);
  WiFiDrv::digitalWrite(b_pin, 0);

  while (true)  // This is the loop of the scetch, not loop()!
                // The loop runs in three modes, according to
                // which stage the system is in (see comments in line 353-)
  {
    if (!client.available())
    {
      WiFiDrv::digitalWrite(r_pin, 1);
      WiFiDrv::digitalWrite(g_pin, 0);
      WiFiDrv::digitalWrite(b_pin, 0);
      while (millis() - lastConnectionTime < postingInterval); // Wait 10 s before next request
      httpRequest();
      mode = 0;
      // reset queue
    }
    switch(mode)
    {
      case 0 : // waiting for a new list to be read
        line = get_next_line();
        //Serial.println(line.c_str());
        if (line.equals(" a<br />"))  // Had accidentally a space there
        { 
          postingInterval += 250;
          if (postingInterval > 10000)
            postingInterval = 10000;
          while (millis() - lastConnectionTime < postingInterval); // Wait 10 s before next request
          httpRequest();
        }
        if (line.equals(" b<br />"))  // Had accidentally a space there
        {
          mode = 1;
          //Serial.println("mode 1");
          WiFiDrv::digitalWrite(r_pin, 0);
          WiFiDrv::digitalWrite(g_pin, 0);
          WiFiDrv::digitalWrite(b_pin, 1);
        }
        delay(500);
        break;
      case 1 : // reads lines from the list and adds to the queue
        line = get_next_line();
        if (line.equals("c<br />"))
        {
          mode = 2;
          //Serial.println("mode 2");
          WiFiDrv::digitalWrite(r_pin, 0);
          WiFiDrv::digitalWrite(g_pin, 1);
          WiFiDrv::digitalWrite(b_pin, 0);
          delay(500);
        }
        else
        {
          add_to_queue(line); // the called function does validity check
        }
        break;
      case 2 : // queue is complete and ready to be performed
        perform_the_queue(); // this keeps the Arduino occupied, no problem with that
        WiFiDrv::digitalWrite(r_pin, 1);
        WiFiDrv::digitalWrite(g_pin, 0);
        WiFiDrv::digitalWrite(b_pin, 0);
        postingInterval = 500;
        while (millis() - lastConnectionTime < postingInterval); // Wait before next request
        httpRequest();
        
        mode = 0;
        
        break;
    }
  }


}



// this method makes a HTTP connection to the server:
void httpRequest() {
  // close any connection before send a new request.
  // This will free the socket on the Nina module
  client.stop();

  // if there's a successful connection:
  if (client.connect(server, 80)) {
    //Serial.println("connecting...");
    // send the HTTP PUT request:
    client.println("GET /cimini/fetchfi.php HTTP/1.1");
    client.println("Host: johanhalmen.xyz");
    client.println("User-Agent: ArduinoWiFi/1.1");
    client.println("Connection: close");
    client.println();

    // note the time that the connection was made:
    lastConnectionTime = millis();
    //Serial.println("connected");
    
  } else {
    // if you couldn't make a connection:
    //Serial.println("connection failed");
  }
}


void printWifiStatus() {
  IPAddress ip = WiFi.localIP();
}

index.html

HTML
This is the interface page, which the user gets by scanning a QR code. The code contains HTML and Javascript.

Lines 4-8
Avoid image getting read from cache. The image is dynamically created, which the browser might not recognise.

Lines 23-29
Defines the small textbox with info. If device is held in landscape orientation, this box won't be seen, but it doesn't contain any crucially important information.

Touch start, lines 61-
Reset everything for recording a new swipe.

Touch moved, lines 81-
Add each event to the array of coordinates and timestamps (punkter[]).

Touch end, lines 91-
Create a binary array to be sent to the receive.php script. When the POST is performed, redraw the user interface image of the chimes with the swipe path added (graphics.php). Line 130 adds a timestamp to the URL, the only purpose of which is to avoid the browser from loading a cached image.

Line 131 and 139-142
At some point I had the swipe path shown only for 3 seconds. After that, the whole page was reloaded.
<!DOCTYPE html>
<html>
<head>
<meta Http-Equiv="Cache-Control" Content="no-cache">
<meta Http-Equiv="Pragma" Content="no-cache">
<meta Http-Equiv="Expires" Content="0">
<meta Http-Equiv="Pragma-directive: no-cache">
<meta Http-Equiv="Cache-directive: no-cache">    
<style>
body
{
    background-color:                   #582815;
    background-repeat:                  no-repeat;
    background-attachment:              fixed;
    height:                             1220px;
    width:                              100%;
    background-size: 100%;
    overflow:                           hidden;
    position:                           fixed;
    top:                                0px;
}

div.relative {
  position: fixed;
  top: 900px;
  border: 3px solid #784835;
  color: #ffdd80;
  font-family: Arial, Helvetica, sans-serif;
  font-size: 50px;
}

</style>

</head>
<body ontouchstart="myFunction(event)" ontouchmove="myFunction(event)"  ontouchend="myEndFunction(event)" overflow="hidden">
<img id="theimage" src="cimini.jpg?cachepreventer=10783662" width="100%" style="border:0px" />
<div class="relative" id="info">
    Drag your finger through the bars.
    
</div>

<script>

var punkter = new Array(5000);
var xy = new Array(3);
var n = 0;
var start_time = 0;
var breakflag = false;


function myFunction(event) {
  var x = Math.floor(event.touches[0].clientX);
  var y = Math.floor(event.touches[0].clientY);
  xy[0] = x;
  xy[1] = y;
  xy[2] = event.timeStamp - start_time;
  
  var now = event.timeStamp;
  
if (event.type == "touchstart") 
  {
    document.getElementById("theimage").src= "cimini.jpg";
    if (y < 800)
    {
        start_time = event.timeStamp;
        xy[2] = 0;
        punkter.length = 0;
        n = 0;
        punkter[n] = xy.slice(0, 3);
        document.getElementById("info").innerHTML = "touched\n";
        breakflag = false;
    }
    else
    {
        document.getElementById("info").innerHTML = "Please, start the dragging inside the image\n";
        breakflag = true;
    }
  }
  else 
  {
    if (event.type == "touchmove" && n < 4999 && breakflag == false && breakflag == false) 
    {
      n++;
      punkter[n] = xy.slice(0, 3);
      document.getElementById("info").innerHTML = "moving\n";
    }

  }
}

function myEndFunction(event) {
    if (breakflag == true)
    {
        document.getElementById("info").innerHTML = "breaked\n";
        return;
    }
        
    


  xy[0] = punkter[n][0];
  xy[1] = punkter[n][1];
  xy[2] = event.timeStamp - start_time;
  n++;
  punkter[n] = xy.slice(0, 3);
 
  document.getElementById("info").innerHTML = "stopped\n";    
    // send the binary data
    
    var myArray = new ArrayBuffer(6*n + 10);
    var longInt16View = new Uint16Array(myArray);
    
    // generate some data
    longInt16View[0] = 1; // endian check
    longInt16View[1] = n;
    for (i = 0; i <= n; i++) 
    {
      longInt16View[3*i + 2] = punkter[i][0];
      longInt16View[3*i + 3] = punkter[i][1];
      longInt16View[3*i + 4] = punkter[i][2];
    }
    
    
    var xhr = new XMLHttpRequest;
    xhr.open("POST", "receive.php", true);
    xhr.onload = function () {
        // Uploaded.
        var d = new Date();
        document.getElementById("info").innerHTML = "succeeded in sending\n";
        document.getElementById("theimage").src="graphics.php?nocache="+d.getTime();
        //setTimeout(resetImage, 3000);
    };
    xhr.send(longInt16View);
    

    
}

function resetImage() {
  
  document.location.reload();
}


</script>

</body>
</html>

receive.php

PHP
This script receives the binary data chunk sent from index.html after a swipe. It saves it in a MySQL database table. The table contains the following data:
$largestNdx - a unique number for each event in the whole table
$largestId - an ID number for all events belonging to one swipe
$x, $y, $timestamp - coordinates and timestamp for one event

Line 66-
A function that gets one 16 bit integer from the received $rawdata chunk.

The $rawdata chunk has the number 1 stored in the first word. I used this number at some point to check the endianness of the data created. After I found out that the endianness was the same in all tested systems, I deleted code related to endianness, but this word I left, because it was a part of the raw data structure everywhere. The second word contains the number of events (max 5000). After that, everything is event data.
<html>
 <head>
  <title>PHP Test</title>
 </head>
 <body>
 <?php 
    
    echo "<h1>Hello2</h1>";
    $rawdata =  file_get_contents("php://input");
    
    $servername = "localhost";
    $username = "my_secret_username";
    $password = "my_secret_password";
    $dbname = "my_database_name";

    $conn = mysqli_connect($servername, $username, $password, $dbname);

    // Get the highest ID of all swipes currently in the database
    $rowSQL = mysqli_query( $conn, "SELECT MAX( id ) AS max FROM `coords`" );
    if ($rowSQL != NULL)
    {
        $row = mysqli_fetch_array( $rowSQL );
        $largestId = $row['max'] + 1;
        $rowSQL = mysqli_query( $conn, "SELECT MAX( ndx ) AS max FROM `coords`" );
        $row = mysqli_fetch_array( $rowSQL );
        $largestNdx = $row['max'] + 1;
    }
    else
    {
        $largestId = 0;
        $largestNdx = 0;
    }
    // Now $largestID holds the ID number of the new swipe to be stored

    $pointer = 4;
    $nofpoints = fetchUint16(2);
    // Now we know how many points the swipe contains
    for ($i = 0; $i <= $nofpoints; $i++)
    {
        $x = fetchUint16($pointer);
        $pointer += 2;
        $y = fetchUint16($pointer);
        $pointer += 2;
        $timestamp = fetchUint16($pointer);
        $pointer += 2;
        
        $sql = "INSERT INTO coords (ndx,id,x,y,timestamp) VALUES ('$largestNdx','$largestId','$x','$y','$timestamp')";
        
      // All echos in the codes are for debugging purposes and don't 
      // show anywhere, when Cimini is in action
    	if (mysqli_query($conn, $sql)) 
    	{
		    echo "New record created successfully !";
	    } 
	    else 
	    {
		    echo "Error: " . $sql . "" . mysqli_error($conn);
	    }
	    $largestNdx++;
    }
    
    
    	
    mysqli_close($conn);
    
    function fetchUint16($offs)
    {
        global $rawdata;
        
        $array = unpack("v", $rawdata, $offs);
        return $array[1];
    }
    
 ?> 
 </body>
</html>

Credits

Johan_Ha

Johan_Ha

15 projects • 24 followers
Music teacher Composer Coding for fun
Thanks to Tom Igoe, Federico Vanzati.

Comments

Related channels and tags
  • Arduino
  • Home Automation
  • Music