Cyber Gnome

Gnomes have a humble and unassuming nature, especially those living in gardens. They spend early dawns clearing fallen leaves off of flowers so bees can enjoy their nectar breakfasts. Gnomes are also known to free misfortunate birds tangled in loose strings and wires left around by careless humans. During the night, they enjoy reciting melancholic tales while drinking fermented raspberries which they share with their firefly friends.

The Cyber Gnome is a high-tech arts and crafts project aimed to gently introduce anyone to the IoT Universe. The Internet of things, much like the mystical world of garden gnomes, is rarely seen during our daily bustles. Its many sprites and daemons of code running on minuscule microcontrollers ensure the techno-fauna of our homes operates as we desire.

Sculpt

I decided a flower garden is the perfect place to begin our adventure since it too is a complex environment hidden behind a demure façade. This is the primary reason why I chose an earthenware construction. Not all clay is equal and these are some discoveries I made:

• Instant concrete - the gnome is constructed from this. It is a very durable, waterproof material which will last at least a hundred years. The interior of the gnome is where I house some very large batteries, a solar panel, and a small circuit for his two illuminated eyes. When concrete cures, a significant amount of carbon dioxide is released.

• Polymer clay - the face of the flower camera is built from this. Forming any shape you can imagine is a pleasure with this material. It does, however, require an over to cure. The brand I used hardened at 175℃ in 30 minutes. Since this is a synthetic material, it will leech toxic plastic into the environment over time.

• Air dry clay - the entire mushroom is formed in this. You can find this clay near almost any river bank. It is entirely natural and will not introduce any harmful chemicals to the garden. It is not waterproof, however, requiring very high temperatures to cure. You can overcome this by applying at least three layers of paint.

• 3D-printing - don't. At least not if you plan on leaving your gnome outside. We haven't invented a printable material which won't eventually end up floating in your blood-stream, so try to avoid using plastics which degrade outdoors.

It is important to create a protective cover for the camera. After much experimentation, I found a candy container which fit perfectly. I added a cushion out of foam board so the camera wouldn't shake.

The box is attached to the back of the clay flower using hot glue and papier-mâché. Make sure to double check everything works and that the lens is unobstructed. I used this guide, but you can skip over the complicated bits for now and simply download this file, capture_images_script.py, or copy-paste the script below:

# paste me into notepad and save me as capture_images_script.py

import os
import usb1
from PIL import Image
from io import BytesIO
import argparse
import time
import cv2
import numpy as np
from threading import Thread

WEBUSB_JPEG_MAGIC = 0x2B2D2B2D
WEBUSB_TEXT_MAGIC = 0x0F100E12

VendorId = 0x2886  # seeed studio
ProductId = [0x8060, 0x8061]


class Receive_Mess():
    def __init__(self, arg, device_id):
        self.showimg = not arg.unshow
        self.saveimg = not arg.unsave
        self.interval = arg.interval
        self.img_number = 0
        self.ProductId = []
        os.makedirs("./save_img", exist_ok=True)

        self.expect_size = 0
        self.buff = bytearray()

        self.device_id = device_id
        self.context = usb1.USBContext()

        self.get_rlease_device(device_id, False)

        self.disconnect()
        self.pre_time = time.time() * 1000
        time.time_ns()

    def start(self):
        while True:
            if not self.connect():
                continue
            self.read_data()
            del self.handle
            self.disconnect()

    def read_data(self):
        # Device not present, or user is not allowed to access device.
        with self.handle.claimInterface(2):
            # Do stuff with endpoints on claimed interface.
            self.handle.setInterfaceAltSetting(2, 0)
            self.handle.controlRead(0x01 << 5, request=0x22, value=0x01, index=2, length=2048, timeout=1000)
            # Build a list of transfer objects and submit them to prime the pump.
            transfer_list = []
            for _ in range(1):
                transfer = self.handle.getTransfer()
                transfer.setBulk(usb1.ENDPOINT_IN | 2, 2048, callback=self.processReceivedData, timeout=1000)
                transfer.submit()
                transfer_list.append(transfer)
            # Loop as long as there is at least one submitted transfer.
            while any(x.isSubmitted() for x in transfer_list):
                # reading data
                self.context.handleEvents()

    def pare_data(self, data: bytearray):
        if len(data) == 8 and int.from_bytes(bytes(data[:4]), 'big') == WEBUSB_JPEG_MAGIC:
            self.expect_size = int.from_bytes(bytes(data[4:]), 'big')
            self.buff = bytearray()
        elif len(data) == 8 and int.from_bytes(bytes(data[:4]), 'big') == WEBUSB_TEXT_MAGIC:
            self.expect_size = int.from_bytes(bytes(data[4:]), 'big')
            self.buff = bytearray()
        else:
            self.buff = self.buff + data

        if self.expect_size == len(self.buff):
            try:
                Image.open(BytesIO(self.buff))
            except:
                self.buff = bytearray()
                return
            if self.saveimg and ((time.time() * 1000 - self.pre_time) > self.interval):
                with open(f'./save_img/{time.time()}.jpg', 'wb') as f:
                    f.write(bytes(self.buff))
                self.img_number += 1
                print(f'\rNumber of saved pictures on device {self.device_id}：{self.img_number}', end='')
                self.pre_time = time.time() * 1000

            if self.showimg:
                self.show_byte()
            self.buff = bytearray()

    def show_byte(self):
        try:
            img = Image.open(BytesIO(self.buff))
            img = np.array(img)
            cv2.imshow('img', cv2.cvtColor(img,cv2.COLOR_RGB2BGR))
            cv2.waitKey(1)
        except:
            return

    def processReceivedData(self, transfer):
        if transfer.getStatus() != usb1.TRANSFER_COMPLETED:
            # transfer.close()
            return

        data = transfer.getBuffer()[:transfer.getActualLength()]
        # Process data...
        self.pare_data(data)
        # Resubmit transfer once data is processed.
        transfer.submit()

    def connect(self):
        '''Get open devices'''
        self.handle = self.get_rlease_device(self.device_id, get=True)
        if self.handle is None:
            print('\rPlease plug in the device!')
            return False
        with self.handle.claimInterface(2):
            self.handle.setInterfaceAltSetting(2, 0)
            self.handle.controlRead(0x01 << 5, request=0x22, value=0x01, index=2, length=2048, timeout=1000)
            print('device is connected')
        return True

    def disconnect(self):
        try:
            print('Resetting device...')
            with usb1.USBContext() as context:
                handle = context.getByVendorIDAndProductID(VendorId, self.ProductId[self.device_id],
                                                           skip_on_error=False).open()
                handle.controlRead(0x01 << 5, request=0x22, value=0x00, index=2, length=2048, timeout=1000)
                handle.close()
                print('Device has been reset!')
            return True
        except:
            return False

    def get_rlease_device(self, did, get=True):
        '''Turn the device on or off'''
        tmp = 0
        print('*' * 50)
        print('looking for device!')
        for device in self.context.getDeviceIterator(skip_on_error=True):
            product_id = device.getProductID()
            vendor_id = device.getVendorID()
            device_addr = device.getDeviceAddress()
            bus = '->'.join(str(x) for x in ['Bus %03i' % (device.getBusNumber(),)] + device.getPortNumberList())
            if vendor_id == VendorId and product_id in ProductId and tmp == did:
                self.ProductId.append(product_id)
                print('\r' + f'\033[4;31mID {vendor_id:04x}:{product_id:04x} {bus} Device {device_addr} \033[0m',
                      end='')
                if get:
                    return device.open()
                else:
                    device.close()
                    print(
                        '\r' + f'\033[4;31mID {vendor_id:04x}:{product_id:04x} {bus} Device {device_addr} CLOSED\033[0m',
                        flush=True)
            elif vendor_id == VendorId and product_id in ProductId:
                self.ProductId.append(product_id)
                print(f'\033[0;31mID {vendor_id:04x}:{product_id:04x} {bus} Device {device_addr}\033[0m')
                tmp = tmp + 1
            else:
                print(
                    f'ID {vendor_id:04x}:{product_id:04x} {bus} Device {device_addr}')


def implement(arg, device):
    rr = Receive_Mess(arg, device)
    time.sleep(1)
    rr.start()


if __name__ == '__main__':
    opt = argparse.ArgumentParser()
    opt.add_argument('--unsave', action='store_true', help='whether save pictures')
    opt.add_argument('--unshow', action='store_true', help='whether show pictures')
    opt.add_argument('--device-num', type=int, default=1, help='Number of devices that need to be connected')
    opt.add_argument('--interval', type=int, default=300, help='ms,Minimum time interval for saving pictures')
    arg = opt.parse_args()
    if arg.device_num == 1:
        implement(arg, 0)
    elif arg.device_num <= 0:
        raise 'The number of devices must be at least one!'
    else:
        pro_ls = []
        for i in range(arg.device_num):
            pro_ls.append(Thread(target=implement, args=(arg, i,)))
        for i in pro_ls:
            i.start()

and save it to a folder named ai_camera on your desktop. Next open command prompt in Windows: 🪟 +S→ search for CMD. Then type cd desktop/ai_camera. Now plug in the camera to your computer using a USB-C cable. With your command prompt still open, type python3 capture_images_script.py

It is likely that Windows will pop up and tell you to install Python. Do that, then come back and repeat this step. Now you can finally see if the camera works and if you positioned it correctly.

You can use this script again later when you want your gnome to specifically detect certain animals, birds or insects. That process can take weeks and thousands of images where you have to draw squares around interesting subjects. Ignore any advice which directs you to use a smartphone or a database. The "AI" works best if it is trained using your specific camera and your specific location. Follow the brilliant walkthrough I previously mentioned for more information on image models.

I gave my flower a long wire stem so I can stick it straight into the ground. Make sure to paint your surfaces white at least twice. I used black paint for the interior of the mushroom since I didn't want it to reflect internal LEDs.

This project is an adventure in frugality. Ignoring the electronics, everything should cost less than $10. My paints, which I purchased the prior week, were half this budget. For the other half, I splurged on a set of new, poorly-crafted, brushes and on 500 grams of air-dry clay. You shouldn't use expensive paint brushes on clay since you will get them very muddy.

Don't forget to consider cable management. In my case, I knew I would power the Wio Terminal, sensors, and camera via a USB cable connected either to my PC or to the power bank within the gnome. So I cut a door into the mushroom and raised a channel in the back.

With all the extra space, I added some electroluminescent wire for a touch of whimsy. This wire produces a lot of interference, so don't imitate my design if you wish to implement a Wi-Fi enabled project.

The cables included with the kit are awkwardly short and you can't plug in more than one Grove sensor at once into the Wio Terminal. You can however daisy chain the sensors or cut the included wires in half and solder all the black wires together, and all the red wires, and the yellow and the white. But before undertaking this herculean task, consider ordering an extension hub or using a different prototyping platform altogether. No mater what you select, you'll be as satisfied of your project as I am of mine.

Code

Below is the code I used in the video above. It is a collection of functions from the kit's official wiki. I used this script to test whether the sensors were actually useful before I started any work. I left it running 24 hours a day for two weeks; when sparks didn't fly, only then did I start planning the cyber gnome.

/* By Constantin Bucataru as part of the project submission at https://www.hackster.io/constantinbucataru/cyber-gnome-cd4ad0 */

#include <Arduino.h>
#include "sensirion_common.h"
#include "sgp30.h"

#include <SensirionI2CSht4x.h>
#include <Wire.h>
 
SensirionI2CSht4x sht4x;

#include"TFT_eSPI.h"
TFT_eSPI tft;
#define LCD_BACKLIGHT (72Ul) // control pin of the LCD
TFT_eSprite spr = TFT_eSprite(&tft);  // always use sprites with LCD screens to avoid flickering

uint16_t error;
char errorMessage[256];

float temperature;
float humidity;

s16 err;
u32 ah = 0;
u16 scaled_ethanol_signal, scaled_h2_signal;

uint32_t serialNumber;

u16 tvoc_ppb, co2_eq_ppm;
    
void setup() {
    Serial.begin(115200);

    /*
     * TFT Screen
     * https://wiki.seeedstudio.com/Wio-Terminal-LCD-Basic/
     */
    
    tft.begin();
    tft.setRotation(3);

    spr.createSprite(TFT_HEIGHT,TFT_WIDTH);
    spr.setRotation(3);
    
    digitalWrite(LCD_BACKLIGHT, HIGH); // turn on the backlight
    
    /*
     * Light sensor
     */
    
    pinMode(WIO_LIGHT, INPUT);
    
    /*
     * K1100 VOC and eCO2 gas sensor
     * https://wiki.seeedstudio.com/K1100-VOC-and-eCO2-Gas-Sensor-Grove-LoRa-E5/
     */
 
    /*  Init module,Reset all baseline,The initialization takes up to around 15 seconds, during which
        all APIs measuring IAQ(Indoor air quality ) output will not change.Default value is 400(ppm) for co2,0(ppb) for tvoc*/
    while (sgp_probe() != STATUS_OK) {
        Serial.println("SGP failed");
        delay(1000);
    }
    /*Read H2 and Ethanol signal in the way of blocking*/
    err = sgp_measure_signals_blocking_read(&scaled_ethanol_signal,
                                            &scaled_h2_signal);
    if (err == STATUS_OK) {
        Serial.println("get ram signal!");
    } else {
        Serial.println("error reading signals");
    }
 
	/* Instead of the arbitrary 13000 g/m³, humidity should be set using the humidity sensor to 
	 * continually calibrate the outrageously innacurate TVOC sensor */
    sgp_set_absolute_humidity(13000);
    err = sgp_iaq_init();

    /*
     * K1100 Temp Humi Sensor
     * https://wiki.seeedstudio.com/K1100-Temp-Humi-Sensor-Grove-LoRa-E5/
     */
    
    Wire.begin();
 
    sht4x.begin(Wire);
 
    error = sht4x.serialNumber(serialNumber);
    if (error) {
        Serial.print("Error trying to execute serialNumber(): ");
        errorToString(error, errorMessage, 256);
        Serial.println(errorMessage);
    } else {
        Serial.print("Serial Number: ");
        Serial.println(serialNumber);
    }
}
 
void loop() {
    err = sgp_measure_iaq_blocking_read(&tvoc_ppb, &co2_eq_ppm);
    if (err == STATUS_OK) {
        Serial.print("tVOC  Concentration:");
        Serial.print(tvoc_ppb);
        Serial.println("ppb");
 
        Serial.print("CO2eq Concentration:");
        Serial.print(co2_eq_ppm);
        Serial.println("ppm");
    } else {
        Serial.println("error reading IAQ values\n");
    }

    /* */

    error = sht4x.measureHighPrecision(temperature, humidity);
    if (error) {
        Serial.print("Error trying to execute measureHighPrecision(): ");
        errorToString(error, errorMessage, 256);
        Serial.println(errorMessage);
    } else {
        Serial.print("Temperature:");
        Serial.print(temperature);
        Serial.print("\t");
        Serial.print("Humidity:");
        Serial.println(humidity);
    }

    /* */

    Serial.printf("Light value: %03d", analogRead(WIO_LIGHT));

    /* */
    spr.fillSprite(TFT_BLACK);
    
    spr.setTextColor(TFT_WHITE);

    char t_buff[40];

    spr.setFreeFont(&FreeSans24pt7b);
    
    sprintf(t_buff," %4dppm,%4d",co2_eq_ppm,tvoc_ppb);
    spr.drawString(t_buff,0,5);
    
    sprintf(t_buff," %.1f%cC  %.1f%%",float(temperature),char(176),float(humidity));
    spr.drawString(t_buff,0,110);

    sprintf(t_buff,"   %4d lux",analogRead(WIO_LIGHT));
    spr.drawString(t_buff,0,200);

    spr.pushSprite(0, 0);
    delay(1000);
}

The gas sensor and the temperature sensor I am using are attached to the back of the mushroom and protected from direct sunlight by a faux green leaf.

Since I only had one port available on the Wio Terminal, I connected the other sensor directly to the 3-volt (3V3)( 1 ), ground (GND)( 6 ), I2C1_SDA( 3 ) and I2C1_SCL( 5 ) pins using this diagram:

I also experimented with the moisture sensor by inserting it into a flower pot. I'll have to log several weeks of data to see whether it is accurate during different times of day and during varying temperatures.

Rather than adapting my Arduino code above, a far quicker method of playing with the sensors is to use the SenseCraft application distributed by Seeed Studio. Start by downloading the latest .uf2 file from this official repository. Then connect the Wio Terminal to your PC and do this fast:

Now you can drag the file you just downloaded into the new folder that pops up.

Wait a few moments and your terminal's screen should look something like this:

Sometimes the Grove sensors aren't automatically detected and the long range antenna isn't configured correctly so ask for help in the official Seeed discord forum and you'll quickly find out how to get everything connected. It is a commendable effort to design a plug-and-play system so that even after you've outgrown your creation, you can gift it to a new generation of citizen scientists. Building a reusable open science project which can be redeployed indefinitely is the greatest gift you can give our society and our planet.

Explore!

A few years ago, I built a similar project to satisfy my curiosity. I diligently logged environmental data and used statistical analysis to deduce trends. To my surprise, every Thursday while I slept, the air outside my bedroom window was significantly polluted. A while later, I was courageous enough to take my gadget and sniff out the culprit. I discovered a car tire incinerator a few kilometers away. After I notified a local politician, the company was forced to add a modern filtration system. Since then, my neighbourhood has breathed far better.

This anecdote is an example that anyone can be a citizen scientist. Even in a city with millions of people, you can notice something no one else has and take action to improve lives while saving the planet.

I hope my prototype build guide inspired your own projects. Thank you! And don't forget to always be kind to squirrels!