•

Published August 28, 2020 © GPL3+

Tangible Light

This is a conceptual app platform designed for tangible user interfaces, and the hardware is a specific example to show the implementation.

AdvancedShowcase (no instructions)2,296

Things used in this project

Hardware components

Arduino UNO

Raspberry Pi 4 Model B

SG90 Micro-servo motor

100

Adafruit PCA9685

Connector Adapter, DC Power - 2.1mm

5v 10A switching power supply

Male/Female Jumper Wires

Software apps and online services

Arduino IDE

We use Arduino to receive data from Processing to perform 3D display on this tangible user interface through servo motor matrics.

Unity

We create a virtual model in Unity as the user interface for people who are not experienced programmers, and the matrics of the display signals generated in the backend will be written in the text file, then send them to RPi through TCP, so that Processing can read them.

The Processing Foundation Processing

We use Processing to read the text file of the matrics of the display signals, and then send these matrics to Arduino through the serial port.

Autodesk Inventor

Hand tools and fabrication machines

Laser cutter (generic)

Soldering iron (generic)

Story

Description

This is a conceptual app platform designed for tangible user interfaces, and we want to make the app easy to use, so even people who don't have deep experience in programming can design, publish their own applications on this software.

On the home page of the software, there are two main sections here. The section above is an online platform just like the App Store on mobile phones, where developers, designers can publish their applications. The other section is the toolsets that you can use them to design applications. It looks like Unity's typography, having many SDK, API, and other stuff embedded in the software. The software itself is connected to the wireless hardware, thus makes it possible to test your application design in real-time through sending matrics of the display signals.

The hardware we used is a simplified version of the well-known tangible user interface from MIT, inFORM. However, the one we built is not licensed from MIT, but our personal built. The main purpose is to present our software ideas. To know more about the original one, please click the link here.

This is a demo for what the software suppossed to look like.

Full scene of the hardware (In real case, RPi will replace the laptop.)

How we Build

There are software and hardware in this project, and I'll go through the detail in each of them.

Firstly, we use Unity to build a virtual inFORM and various buttons as our user interface, and the data will be sent from the Unity backend to the connected computer as a text file. Next, Processing can read the text file and send the data through the serial port. Finally, Arduino can perform the synchronous operation as the virtual one by reading data on the serial port.

On the other hand, for the hardware, there are three layers in this device:

1. Top Pins: We cut the styrofoam into the cuboid pieces, and glue them with the magnet wire at the end of the linkage.

2. Linkage: The linkage, a magnet wire inside a plastic housing(hollow nylon rods), is used to link each pin with an actuator, thus enable a dense pin arrangement independent of actuator size. Besides, it can transmit bi-directional force from the servo motor.

Linkage

3. Servos Matrix: 100 servo motors are arranged in a matrix form to actuate the linear movement of the pins, and are F through PCA9685 servo drivers.

The matrix is composed of 50 servo units with 2 servos in each unit.

Each unit is composed of two racks, gears, and servo motors.

Each servo driver connects to 10 servo motors.

After the assembly, the device can be controlled by Arduino IDE, and perform specific tests. In this demo, we triggered a sine wave on the first row of the matrix.

Hardware demo

Future Work

The demo shows the fast response and steady movement. However, there are some deficiencies due to our budget and engineering problems, causing the device to lack of further functionality.

1. The power system problem: We had spent so much time fixing the power issue which causes the pins not to move simultaneously. Whenever each row of pins was tested separately, it performs as well as the one in the demo, but not in the case of testing them all together. We are still looking for a solution.

2. The pin constraints problem: The pins are placed in the plastic slots to make sure the movement would always stay in one direction. As a consequence of the loose constraints, the pins sometimes tilted slightly when they extend, while tight constraints would increase friction. Therefore, the preferred solution may be increasing the length of the pins, so the ratio of the exposed portion to the hidden portion will be reduced, leading to less tilting condition.

3. The servo unit structure problem: The structure of the unit is not the optimized version, because the racks are not well constrained and occasionally separated from the gear. The problem would not occur most of the time, but pretty annoying when it happens. We are still looking for a solution.

4. No haptic feedback problem: In the original version of the device, the pins can not only perform visual effects but also haptic feedback. Yet, we don't have enough budget to purchase related instruments, so the functionality we made is not very similar to the original one. After all, our main purpose is not to recreate the hardware but to demonstrate the software idea, so this may not be a big issue.

Thanks for reading! I hope you enjoyed it.

Schematics

Code

Display Shape - Sine

#include <Wire.h>
#include <Adafruit_PWMServoDriver.h>
#include <Math.h>

// Address
Adafruit_PWMServoDriver pwm = Adafruit_PWMServoDriver(0x40);
Adafruit_PWMServoDriver pwm1 = Adafruit_PWMServoDriver(0x41);
Adafruit_PWMServoDriver pwm2 = Adafruit_PWMServoDriver(0x42);
Adafruit_PWMServoDriver pwm3 = Adafruit_PWMServoDriver(0x43);
Adafruit_PWMServoDriver pwm4 = Adafruit_PWMServoDriver(0x44);
Adafruit_PWMServoDriver pwm5 = Adafruit_PWMServoDriver(0x45);
Adafruit_PWMServoDriver pwm6 = Adafruit_PWMServoDriver(0x46);
Adafruit_PWMServoDriver pwm7 = Adafruit_PWMServoDriver(0x47);
Adafruit_PWMServoDriver pwm8 = Adafruit_PWMServoDriver(0x48);
Adafruit_PWMServoDriver pwm9 = Adafruit_PWMServoDriver(0x49);

#define SERVOMIN  110 // this is the 'minimum' pulse length count (out of 4096)
#define SERVOMAX  400 // this is the 'maximum' pulse length count (out of 4096)
int t = 0;
int offset = (SERVOMAX + SERVOMIN) / 2;
int ampl = (SERVOMAX - SERVOMIN) / 2;
int freq = 1;

void setup() {
  Serial.begin(115200);
  Serial.setTimeout(10);
  Serial.println("16 channel Servo test!");

  pwm.begin();
  pwm1.begin();
  pwm2.begin();
  pwm3.begin();
  pwm4.begin();
  pwm5.begin();
  pwm6.begin();
  pwm7.begin();
  pwm8.begin();
  pwm9.begin();
// Analog servos run at ~60 Hz updates
  pwm.setPWMFreq(60);
  pwm1.setPWMFreq(60);
  pwm2.setPWMFreq(60);
  pwm3.setPWMFreq(60);
  pwm4.setPWMFreq(60);
  pwm5.setPWMFreq(60);
  pwm6.setPWMFreq(60);
  pwm7.setPWMFreq(60);
  pwm8.setPWMFreq(60);
  pwm9.setPWMFreq(60);

//Zero
  for(int i = 0; i < 16; i += 1){
    pwm.setPWM(i, 0, SERVOMIN);
    pwm1.setPWM(i, 0, SERVOMIN);
    pwm2.setPWM(i, 0, SERVOMIN);
    pwm3.setPWM(i, 0, SERVOMIN);
    pwm4.setPWM(i, 0, SERVOMIN);
    pwm5.setPWM(i, 0, SERVOMIN);
    pwm6.setPWM(i, 0, SERVOMIN);
    pwm7.setPWM(i, 0, SERVOMIN);
    pwm8.setPWM(i, 0, SERVOMIN);
    pwm9.setPWM(i, 0, SERVOMIN);
  }
  Serial.println("Ready");
}

void loop() {
  while(Serial.available()){ //To control the starting time.
    for (int i = 0; i < 10; ++i)
    {
      Serial.print((int)(offset+ampl*sin(freq*(i+t))));
      Serial.print(" ");
      pwm.setPWM(i, 0, (int)(offset+ampl*sin(freq*(i+t))));
      pwm1.setPWM(i, 0, (int)(offset+ampl*sin(freq*(i+t+1))));
      pwm2.setPWM(i, 0, (int)(offset+ampl*sin(freq*(i+t+2))));
      pwm3.setPWM(i, 0, (int)(offset+ampl*sin(freq*(i+t+3))));
      pwm4.setPWM(i, 0, (int)(offset+ampl*sin(freq*(i+t+4))));
      pwm5.setPWM(i, 0, (int)(offset+ampl*sin(freq*(i+t+5))));
      pwm6.setPWM(i, 0, (int)(offset+ampl*sin(freq*(i+t+6))));
      pwm7.setPWM(i, 0, (int)(offset+ampl*sin(freq*(i+t+7))));
      pwm8.setPWM(i, 0, (int)(offset+ampl*sin(freq*(i+t+8))));
      pwm9.setPWM(i, 0, (int)(offset+ampl*sin(freq*(i+t+9))));
    }
    Serial.println("");
    delay(100);
    t++;
  }
}