Maria Vittoria Colombo

•

belll

•

ziming shang

•

Kevin Enriquez Ambrocio

Published March 7, 2025

Human-microenvironment interaction

Exploring interspecies interaction via a heart beat sensor triggering water movement, mapped to sound and visuals

BeginnerFull instructions provided4 hours143

Things used in this project

Hardware components

Arduino UNO

Brown Dog Gadgets Solar Cockroach Vibrating Disc Motor

DFRobot Gravity: Heart Rate Monitor Sensor for Arduino

Microscope

Petri dish

Software apps and online services

TouchDesigner

Arduino IDE

Hand tools and fabrication machines

Biomaterials

Laser cutter (generic)

Story

A microchallenge is a week long workshop inside the Master in Design for Emergent Futures at IAAC Barcelona. During this week we have the time, space and support of professionals to create a functioning prototype, which can also be a speculative artifact, that contributes to our personal research and practice. This prototype should be based on the iteration and usage of: digital fabrication tools, biofabrication, AI, electronics, robotics, coding. For this particular microchallenge the topic was about "Intelligences", because of our shared interests in our personal research we grouped:

Belen Comotto

Maria Vittoria Colombo

Ziming Shang

Kevin Giovanni Enriquez Ambrocio

During this week, we wanted to decided to address "more than human" intelligences and ways into making visible even the tiniest life forms that conform our reality. We explore more-than-human interactions in a micro scale, diving into the concept of interspecies collaboration. Specifically, we’re interested in the speculative exchange between human input and micro organisms living in water systems.

The interaction unfolds in a loop, beginning with a human input via the heart rate sensor. This triggers a vibrating sensor attached to a Petri dish filled with water, causing the water to move in patterns based on the heart beat rhythm.

These movements are captured through a microscope connected to a computer, where the images are analyzed in TouchDesigner. The system identifies key features and maps them out, creating a matrix of coordinates, which is then translated into sound.

initial brainstorming

Fabrication and iteration process:

Along the way of the development and testing of the working of the prototype we tested several different zooms from the microscope, setups of the Petri dish, different vibration intensities and different mediums of water with microorganisms. In the end we concluded that the best outcome resulted from using a mixture of different origins of the water from different ponds, medium intensity vibration (150 / 255), and 10x magnification from the microscope. These parameters provided a clear enough visual from the camera sensor that would be usable in the TouchDesigner file.

Sensors:

The first sensor we needed to approach for this prototype was the one measuring our heart beats which translated into vibrations for the Petri dish. We tested various setups, adjusting values and coding repeatedly for 3 different sensors: a heart monitor sensor, a finger heart rate sensor and finally a heart beat sensor. However, the results were often imprecise and unreliable with the first two, so we decided to stick to the heart beat sensor for better accuracy (v1706 heart beat sensor).

As can output of the heart beat we needed to achieve a vibration to create the ripples in the water of the Petri dish. We experimented with vibrating motor, speakers, and piezo parts. The vibrating motors, placed on the sides of the Petri dish, provided the most consistent performance. To optimize vibration, we chose a plastic Petri dish and placed it on a silicone base.

TouchDesigner:

We used Python code with the OpenCV plugin in TouchDesigner to apply computer vision, detecting edge points in the video, as well as blob tracking. From these points, we extracted a dataset of x and y coordinates, which we translated into sound by mapping them across four layers of audio oscillators, each using different waveforms and frequencies. This process generated the final audio output.

Water:

We analyzed three different types: stagnant puddle water, seawater, and water from a public park. Each sample revealed distinct elements, which we identified and visualized using TouchDesigner’s features.

1 / 3

Wearable:

We also decided to create a wearable, gelatin-based biomaterial infused with spirulina for coloring. We crafted a rectangular sheet of this material to integrate with the sensor.

Recipe: 100g gelatin, 60g alginate, 550ml water — bring to a boil, then pour into a mold. Add spirulina for pigmentation.

The concept was to design a garment around the neck and chest that appeals to a fish like/ algae like concept allowing the biomaterial to hold the sensor against the neck while proposing a hybridization of the human with the living organisms that live in the water.

concept prototype

FUTURE DEVELOPMENT OPPORTUNITIES

For the future we would like to:

Wearable > focus on developing a more ergonomic and practical design, experimenting with various biomaterials, fabrics, and forms that allow a more easy reading and wear.

Electronics > right now the structure with the arduino in the breadboard and the cables around is not very compact and gives space for error, we would like to minimize as much as we can the circuit and create a seamless integration.

Scale > once the design is solidified, it could be intriguing to scale the concept into a larger installation — a multisensory experience where sound and visuals merge, allowing the audience to engage with the piece in a more immersive and impactful way.

Tweaking > while presenting the project on the final expo of the workshop we realized that the idea was quite complex and not properly understood, as a further iteration we would like that through the experience and design the concept would be clearly understood. Also the fact that the design is very delicate makes it not that approachable and limits the play and freedom, this is a major aspect that would make the experiment more engaging.

Finally as being a large group each of us had different interests that wanted to bring up in this process and thus it added several layers of complexity that could be further explored: the design of functioning wearables, the visual processing, the audio generator and seamless circuits. In the future I believe we had opened several pathways to continue further down these rabbit holes in each individual research.

Code

const int PULSE_SENSOR_PIN = 18; // Replace with the ESP32-S3 pin connected to the v1706
const int VIBRATION_PIN_1 = 12;
const int VIBRATION_PIN_2 = 14;

const int THRESHOLD = 2100; // Adjust this threshold after testing
const int BEAT_DURATION = 200;
const int PAUSE_DURATION = 200;
const int VIBRATION_INTENSITY = 150; // Fixed vibration intensity (0-255)

unsigned long previousBeatTime = 0;
bool beatDetected = false;

// --- Variables for filtering ---
const int SAMPLE_SIZE = 10; // Number of samples for averaging
int pulseReadings[SAMPLE_SIZE];
int readingIndex = 0;
int averagePulse = 0;

// --- Variables for BPM calculation ---
unsigned long lastBeatTime = 0;
unsigned long interBeatInterval = 0;
int bpm = 0;

void setup() {
    Serial.begin(115200);
    pinMode(VIBRATION_PIN_1, OUTPUT);
    pinMode(VIBRATION_PIN_2, OUTPUT);

    // --- Initialize pulseReadings array ---
    for (int i = 0; i < SAMPLE_SIZE; i++) {
        pulseReadings[i] = 0;
    }
}

void loop() {
    // Read sensor value
    int pulseValue = analogRead(PULSE_SENSOR_PIN);

    // --- Simple Moving Average Filtering ---
    pulseReadings[readingIndex] = pulseValue;
    readingIndex = (readingIndex + 1) % SAMPLE_SIZE; // Wrap around
    int sum = 0;
    for (int i = 0; i < SAMPLE_SIZE; i++) {
        sum += pulseReadings[i];
    }
    averagePulse = sum / SAMPLE_SIZE;

    // --- Beat Detection ---
    if (averagePulse > THRESHOLD && !beatDetected) {
        beatDetected = true;
        unsigned long currentTime = millis();

        // --- Calculate inter-beat interval and BPM ---
        interBeatInterval = currentTime - lastBeatTime;
        if (interBeatInterval > 0) {
            bpm = 60000 / interBeatInterval; // 60000 milliseconds in a minute
        }
        lastBeatTime = currentTime;

        previousBeatTime = millis();

        // --- Trigger vibration (using analogWrite for PWM)---
        analogWrite(VIBRATION_PIN_1, VIBRATION_INTENSITY);
        analogWrite(VIBRATION_PIN_2, VIBRATION_INTENSITY);
    } else if (millis() - previousBeatTime > BEAT_DURATION) {
        // --- Stop vibration after BEAT_DURATION ---
        analogWrite(VIBRATION_PIN_1, 0);
        analogWrite(VIBRATION_PIN_2, 0);

        if (millis() - previousBeatTime > BEAT_DURATION + PAUSE_DURATION) {
            // --- Reset beat detection after PAUSE_DURATION ---
            beatDetected = false;
        }
    }

    // --- Print Data to Serial Monitor ---
    Serial.print("Raw: ");
    Serial.print(pulseValue);
    Serial.print(" | Filtered: ");
    Serial.print(averagePulse);
    Serial.print(" | BPM: ");
    Serial.println(bpm);

    delay(10);
}

# me - this DAT
# scriptOp - the OP which is cooking

import numpy as np
import cv2 as cv

# press 'Setup Parameters' in the OP to call this function to re-create the parameters.
def onSetupParameters(scriptOp):

	return

# called whenever custom pulse parameter is pushed
def onPulse(par):
	return


def onCook(scriptOp):
    img = op('null2').numpyArray(delayed=True)
    
    
    gray = cv.cvtColor(img,cv.COLOR_BGR2GRAY)
     
    # find Harris corners
    gray = np.float32(gray)
    dst = cv.cornerHarris(gray,2,3,0.04)
    dst = cv.dilate(dst,None)
    ret, dst = cv.threshold(dst,0.01*dst.max(),255,0)
    dst = np.uint8(dst)
     
    # find centroids
    ret, labels, stats, centroids = cv.connectedComponentsWithStats(dst)
     
    # define the criteria to stop and refine the corners
    criteria = (cv.TERM_CRITERIA_EPS + cv.TERM_CRITERIA_MAX_ITER, 100, 0.001)
    corners = cv.cornerSubPix(gray,np.float32(centroids),(5,5),(-1,-1),criteria)

    scriptOp.store('corners', corners)

    #op('table1').clear()

    #for i in corners:
     #   op('table1').appendRow(i)

    img[dst>0.01*dst.max()]=[0,0,255,255]
    scriptOp.copyNumpyArray(img)
    #print(corners)
    return

Credits

Maria Vittoria Colombo

2 projects • 6 followers

Contact

belll

2 projects • 5 followers

Working on biofabrication and fast prototyping at IAAC Barcelona

Contact

ziming shang

1 project • 5 followers

Contact

Human-microenvironment interaction