Team CMSC730 Group 7:

•

Celia Chen

Published December 13, 2023 © CC BY-NC-ND

Strain

An emotive interactive sculpture that uses a Myo armband to control pneumatic actuators via the Programmable Air

ExpertShowcase (no instructions)8 hours376

Things used in this project

Hardware components

Myo Gesture Control Armband

Raspberry Pi 3 Model B+

Tinkrmind Programmable-Air

Skyn Original Condoms

Bamboo Skewers 6 inch

Silicone Tubing, Food Grade 1/8ID x 3/16OD

Bamboo Stir Sticks - Generic Square End

Heavy thread or common twine

Medium Density Fiberboard

6-32 Threaded Insert, Brass, Barbed

Stainless Steel Socket Head Screws 6-32 1/4"

Straight male luer lock Cole-Parmer EW-12028-51

Female Luer Lock to 1/8 Hose Barb

PLA Filament

Software apps and online services

Arduino IDE

Raspberry Pi Raspbian

PuTTy

Used to run the Raspberry Pi headless

Autodesk Fusion

Adobe Illustrator

Hand tools and fabrication machines

Prusa MK3S

VLS 6.60 Laser Cutter

Stepped Drill Bit

Story

Demonstration of mechanics

Introduction/Abstract

This project was designed as an aesthetic output for a class final at the University of Maryland, for CMSC730, “Interactive Technologies For Human-Computer Interaction” by Dr. Huaishu Peng.

Sculpture Introduction

A key element of many HCI systems is that they claim to make people “feel” things - to alleviate upset, or to read faces and correct for emotional states. This seems like a lot to put on an AI system - but artworks do this quite regularly. Therefore, we aimed to put together a sculpture that would make an audience “feel” something immediately.

In addition, we had some aesthetic concerns:

Because this project is happening within a computer science department, it needed to look like it was a “rapid prototype,” and make use of that design language. This meant exposing the raw materials we used to assemble the project, and restricting obvious processes to digital fabrication outputs
The sculpture should be made of things which are usually considered to be garbage, to reduce material costs and reinforce a DIY aesthetic
Because we were inspired by Mire Lee’s work, the project should be very organic
We also work from the basis of making a "flower" - in this case, the notorious amorphophallus titanum, the Titan Arum "corpseflower," which blooms extremely inconsistently and smells like rot.
Other inspiration included the Stinkhorn mushroom, with which our final work had much in common, shape-wise

A lot of the difficulty of ensuring audience response was solved by making use of condoms as the core actuation structure, which tends to directly focus audience attention while reducing the likelihood that an actuator will explode during demonstrations.

The sculpture was designed to be assembled from modular parts, based on "trash" or disposable work like stir-sticks and bamboo skewers and condoms, with most effort going into ensuring the 3D printed parts would be printable in a short window of time for rapid iteration.

This system was inspired in part by the book Make: Soft Robotics, by Kari Love and Matthew Borgatti (2019, ISBN: 9781680450934), in part by Mire Lee’s work “Milk Of Dreams” at the Venice Biennale 2022. It was also inspired by Penn & Teller’s work “Shadows, ” a standard part of their act for many years.

We made extensive use of the Programmable Air’s demonstration libraries, especially the “suck” library, while learning to use the hardware system effectively.

Hardware Systems Introduction

For this project we wanted to work with pneumatic systems, which give a great organic affect relatively easily. This pointed to using the Programmable Air pneumatic control system by Tinkrmind. We also wanted the system to appear effortless, or at least to have no obvious or overt controls - so we chose to work with the semi-abandoned Myo armband by Thalmic Labs.

The Myo armband has been controlled by many companies - most recently Meta - but remains the industry standard for documenting myoelectric connections in neuroscience labs, and we were interested in seeing if it could be reasonably updated and paired to do some types of practical effects/performance work in contained systems. It can! Over the course of the project we did discover that unfortunately, its initial reviews in 2014 as a “deeply impractical” way to control any human-computer system are very accurate.

It was fun to work with!

We would not elect to work with it this again.

The Programmable Air, on the other hand, is a delightful and useful system for working with pneumatics of all types! It has really good prototyping provisions. By combining the two systems we were able to make a novel project that offers some scope for fun performances with “invisible” controls.

System Function

The system works by detecting “poses” from the Myo armband, once paired to the Raspberry Pi 3B+, and then sending those “poses” over serial 9600 to the Programmable Air project in order to control inflation or deflation.

It can additionally be controlled using the onboard buttons for the Programmable Air in order to test robot response.When not actively inflating or deflating, the system maintains peristalsis, and gives the impression of “breathing” while doing so.

If all goes well, the two systems should connect relatively seamlessly. The system is designed to be updated mainly in the Arduino sketch, which should be able to listen for any of the six core Myo poses.

Arduino Nano Instructions

Installing the basic arduino sketch included in this project should result in the peristalsis code running immediately
The Nano must be booted before the Myo serial connection tries to attach or the Myo script on Raspi will fail

Raspberry Pi Instructions

Install relevant scripts to the Raspberry Pi
Make sure it is accessible over WiFi
Open an SSH connection to it through Putty
Plug in the Myo armband and pair it to the system using the advice from Thalmic Labs
Run the myo_serial.py script. When the script is live, it should show detected poses in your Putty terminal window
The Nano will respond, by default, to "fist" and "spread" instructions

Milestone 1 & 2 summaries

This project took place over three months.

For our first milestone, we assembled our related works and inspirations, and decided which systems to use.

For our second milestone, we completed the systems engineering and integration, which gave us a functioning Programmable Air controlled by a Myo armband.

For the third milestone, we completed the physical fabrication and Raspberry Pi integration for the project.

Schematics

Strain Schematic

This schematic shows the input, control, and output components of Strain, our interactive sculpture system. The input is a Myo armband, worn on the arm, which detects hand/arm gestures through EMG sensors. These gestures are interpreted by a Raspberry Pi 3B+ running at 5V. Based on the gesture detected, the Raspberry Pi sends control signals to an Arduino Nano v3, which operates at 12V to provide sufficient power for the outputs. The Arduino controls a Programmable Air unit, which uses pneumatic actuators to create mechanical motion and interactivity with the sculpture. By detecting gestures with the Myo armband, this system allows the user to manipulate the sculpture by inflating and deflating the sculpture.

Code

Programmable Air Code

C/C++

This code requires an Arduino Nano V.3 to work correctly.

== USE ==
Upload this file to the Nano running the pneumatic system.

=== Information ===
The first state in this code block is probably the most important. It maintains peristalsis in the system, keeping it pulsing whenever the robot falls below a certain pressure difference from basic atmospheric pressure.

After that, it has a series of flags that are set and checked each loop. These flags can then be set to fire off either an inflation or deflation cycle.

Within the inflation cycle is a pressure differential limit to prevent overinflation.

Note: The programmable_air.h library was modified to communicate over 9600 baud in order to work with the Myo armband.

#include <programmable_air.h>

#define atmospheric_pressure 508;
#define targetPressure 2;

#define WAVE_IN "WAVE_IN"  // 7 char
#define REST "REST"
#define FIST "FIST"
#define WAVE_OUT "WAVE_OUT"
#define FINGERS_SPREAD "FINGERS"  // 7 char
#define THUMB_TO_PINKY "THUMB"

int upper_threshold = 10;
int lower_threshold = -10;

int pressureSetupInterval = 1500;

int pressurePollMillis = 0;
int pressurePollInterval = 50;

int serialPollMillis = 0;
int serialPollInterval = 75;

int inflatePollMillis = 0;
int inflatePollInterval = 200;

int currentMillis = 0;

bool deflateFlag = 0;
bool inflateFlag = 0;

bool buttonActiveFlag = 0;

String pose = "none";

void setup() {
  Serial.begin(9600);
  initializePins();
  unsigned long setupMillis = millis();

  // for an interval, check a pressure sensor
  // if it exceeds the necessary pressure
  // turn off

  // RED BUTTON ALSO
  switchOnPump(2, 100);
  blow();

  while (setupMillis < pressureSetupInterval) {  // Record max sensor value
    // switch on pumps to 50% power
    int pressure = readPressure();
    int pressureDiff = pressure - atmospheric_pressure;

    // targetPressure is a magic pressure number for internal boof
    if (pressureDiff > 2) {
      switchOffPumps();
      setupMillis = pressureSetupInterval;
    } else {
      setupMillis = millis();
    }
  }
}

void loop() {
  currentMillis = millis();

  // // ==========
  // /*  STATE 1: Peristalsis */
  // // every loop we have to check the basic pressure status
  // // the pressure should never really be below -1 or -2 or things get Floppy
  bool periTimeCheck = currentMillis - pressurePollMillis >= pressurePollInterval;
  bool pumpsOnFlag = inflateFlag || deflateFlag;

  if (periTimeCheck) {
    int pressure = readPressure();
    int pressureDiff = pressure - atmospheric_pressure;
    Serial.println(pressureDiff);
    if (pressureDiff < -4 && !pumpsOnFlag) {
      // RED BUTTON ALSO
      switchOnPump(2, 100);
      blow();
    }

    else if (pressureDiff > 1 && !pumpsOnFlag) {
      // BLUE BUTTON
      switchOnPump(1, 100);
      suck();
    }
    // no shutdown here

    pressurePollMillis += pressurePollInterval;
  }

  // ==========
  /*  STATE 2: Myo */
  bool serialTimeCheck = currentMillis - serialPollMillis >= serialPollInterval;
  if (Serial.available()) {
    String data = Serial.readString();  // get incoming poses
    pose = data.substring(0, 4);
  }

  // ==========
  // STATE 3: Check Buttons

  // These override myo commands by setting the pump flags
  // But not CHECKING the pump flags?

  bool inflateTimeCheck = currentMillis - inflatePollMillis >= inflatePollInterval;
  if (inflateTimeCheck) {
    int pressure = readPressure();
    int pressureDiff = pressure - atmospheric_pressure;
    
    if (readBtn(BLUE) || pose == "FING") {
      // if the BLUE button is pressed suck air out of the output, else, vent to atmosphere
      deflate_blue();
    } else if (readBtn(RED) || pose == "FIST") {
      // if the RED button is pressed blow air into the output, else, vent to atmosphere
      Serial.println(pressureDiff);
      if (pressureDiff < 3) { // prevent overinflation
        Serial.print("pressureRed ");
        Serial.println(pressureDiff);
        inflate_red();
      } else {
        shutDownPumps();
      }
    } else {
      // turn things off and up the interval
      shutDownPumps();
      inflatePollMillis += inflatePollInterval;
    }
  }
}
// pseudocode
// on setup, inflate until pressure sensor detects a certain core pressure
// pause inflation

// every tick
// check to see if pressure is steady - if not, inflate slightly

// for on fist control, pressurize more
// on release, relax flower back

// on no value, return to pre-set pressure

// FUNCTION LIST
void deflate_blue() {
  // switch on Pump 1 (suck) to 100% duty cycle
  deflateFlag = 1;
  inflateFlag = 0;
  switchOnPump(1, 100);
  suck();
}

void inflate_red() {
  // switch on Pump 2 (blow) to 100% duty cycle
  deflateFlag = 0;
  inflateFlag = 1;
  switchOnPump(2, 100);
  blow();
}

void shutDownPumps() {
  deflateFlag = 0;
  inflateFlag = 0;
  switchOffPumps();
}

Myo Serial code

Python

This code requires a Raspberry Pi 3B+, Arduino Nano V.3, and a Myo armband which must be updated to at least firmware v1.0 in order to work correctly.

== USE ==
Upload this file to the Raspberry Pi synced to the Myo armband and connected to the Arduino Nano.

=== Information ===
This code opens a serial connection between the Arduino in Myo armband gestures and sending them over serial to the Arduino that controls the Programmable Air.

Note: This code imports libraries from the RaspberryPy respository (GitHub linked below)

import sys
sys.path.append("/home/clichen/Projects/RaspberryPy")

import time
import serial
from raspberrypy.control.myo import Myo, Pose

arduino_port = "/dev/ttyUSB0"
arduino_baud_rate = 9600

#Open the serial connection to the Arduino
ser = serial.Serial(arduino_port, arduino_baud_rate, timeout=1)

def send_command(command):
	#Send the command to the Arduino
	ser.write(command.encode())
	#Wait so the command is processed
	time.sleep(0.1)

def on_pose(pose):
	print ("Detected pose:", pose)

	if pose == Pose.REST:
		send_command("REST")
	elif pose == Pose.FIST:
		send_command("FIST")
	elif pose == Pose.FINGERS_SPREAD:
		send_command("FINGERS_SPREAD")
	elif pose == Pose.WAVE_IN:
		send_command("WAVE_IN")
	elif pose == Pose.WAVE_OUT:
		send_command("WAVE_OUT")
	elif pose == Pose.THUMB_TO_PINKY:
		send_command("THUMB_TO_PINKY")

if __name__ == '__main__':
	myo = Myo()
	myo.add_pose_handler(on_pose)
	myo.connect()

	try:
		while True:
			myo.run(1)
		#	time.sleep(0.1)
	except KeyboardInterrupt:
		pass
	finally:
		myo.disconnect()
		ser.close()

Credits

Alex Leitch

1 project • 1 follower

Contact

Celia Chen

1 project • 1 follower

Contact

Thanks to Amitabh "Tinkrmind" Shrivastava and Kari Love.

Comments

Please log in or sign up to comment.

Strain