SHIELD - Safety Hazard ID and Environmental Lifestyle Device

It's no secret that outdoor activity is physically, psychologically and mentally beneficial. Time spent outdoors is invaluable and every attempt should be made to get good, quality time in nature. To most, being in nature doesn't present much risk, but to the impaired, there can be great risk and potential for serious injury. Sudden changes in surface elevation, such as potholes or unseen steps, can lead to injuries for the impaired even if they are making use of a walker or wheelchair. Furthermore, in the winter months there's a much more difficult risk to see, and that's black ice. Any of these situation can turn a enormously beneficial activity into a hazards and risky endeavor.

Modern advances in modular technology provide the tools needed to help detect, avoid and alleviate many common and hazardous situations. With the right sensors and technology, we can all practice healthy habits, in nature, safely.

Device

SHIELD ( Safety Hazard Identification and Environmental Lifestyle Device ) aims to leverage modern technology to provide safe conditions for outdoor enthusiast by combining sensors and software to create a device that's able to detect many different situations in real-time.

SHIELD combines sensors, Bluetooth and a powerful core processor to constantly monitor the environment for dangerous surface temperatures as well as large changes in surface elevation. The following is an early prototype that combines the lidar, temperature and buzzer components.

SHIELD can alert users to unsafe situations via an onboard audio signal or remotely over Bluetooth and the SHIELD app. The SHIELD app is an LVGL application built on top of the Arduino M5Stack software stack. The touch screen allows users to interact directly with the SHIELD device.

Users will know, in real-time, when unsafe surface temperatures (such as black ice) are approaching as well as large changes in elevation (such as potholes, steps or common debris). Users will have time to assess the situation and make the appropriate change before they find themselves in a dangerous situation.

Hardware

SHIELD is powered by the Nordic nRF52840-DK development kit. The device has two sensors and a buzzer for auditory signaling. The MLX90614 is a non-contact, infrared temperature sensor module capable of detecting both ambient and surface point temperatures. For SHIELD, we'll be utilizing the surface point detection abilities to accurately detect approaching surface temperatures. The TF-Luna is a compact LiDAR sensor that quickly and accurately measures distance within a small field of view (which is ideal to detect small surface elevation deviations). Both sensors use I2C for configuration and measurement readings. The nRF52840-DK has multiple I2C busses that can easily be configured to support multiple I2C devices. The Nordic nRF52840-DK's Bluetooth stack is used provide remote capabilities that can be utilized by smart devices. SHIELD can provide the measured values (both surface temperature as well as surface relative elevation), and commands to silence the auditory hazard signal. Lastly, a passive low level trigger buzzer is attached to then nRF52840-DK via standard GPIO an used to provide the user with an auditory signal when the SHIELD device has detected unsafe conditions.

The M5Stack Core2 is a perfect IoT development kit for creating an application for SHEILD. The Core2 is easy to program and has a rich suite of hardware to include a touchscreen, and Bluetooth stack.

Software

There are two applications provided by SHIELD, both freely available on GitHub (see code section below). The device application is written utilizing the nRF Connect SDK (awesome by the way!!!), and powers the Nordic nRF52840-DK development kit, and all SHIELD hardware. The UI application is written using Arduino SDK, and the M5Stack Core2 libraries (another fantastic suite of software).

SHIELD DeviceApplication

The device application was based off the Bluetooth server example. Additional Bluetooth characteristics were added to provide the sensor information (via get or notify) as well as control to silence the onboard SHIELD buzzer. Device driver code was also written to read and control the temperature, distance and buzzer peripherals. The following provides code highlights. Please see the link in the code section below for the complete set of SHIELD code.

Additional I2C devices, is pretty straightforward with the NRC SDK. Users need to provide device overlays, to configure/re-configure GPIOs etc. The SHIELD overlay provides configuration of two I2C busses (yes, only one is needed, but I didn't want to complicate the wiring and the board has support for two busses out of the box!), as well as GPIO configuration for the buzzer.

&i2c0 {
    tempsensor: mysensor@5a {
        compatible = "i2c-device";
        reg = <0x5a>;
        label = "temp sensor";
    };
};

&i2c1 {
    lidarsensor: lidarsensor@10 {
        compatible = "i2c-device";
        reg = <0x10>;
        label = "lidar sensor";
    };
};

/{

    buzzer:zephyr,user {
        gpios = <&gpio0 29 GPIO_ACTIVE_LOW>;
    };
	aliases {
		buzz = &buzzer;
    };
};
&i2c1 {
    status = "okay";
};

Bluetooth characteristic additions is simple as well. Once you've identified the characteristics you'd like to support, you can follow the provided example to expose these characteristics over Bluetooth as such:

BT_GATT_SERVICE_DEFINE(vnd_svc,
  BT_GATT_PRIMARY_SERVICE(&vnd_uuid),
  BT_GATT_CHARACTERISTIC(BT_UUID_GATT_DST,
  BT_GATT_CHRC_READ | BT_GATT_CHRC_NOTIFY,
  BT_GATT_PERM_READ, read_dist, NULL,&dw_bt_dist),
BT_GATT_CCC(dw_ccc_cfg_changed, 
  BT_GATT_PERM_READ | BT_GATT_PERM_WRITE), 
  BT_GATT_CHARACTERISTIC(BT_UUID_HTS_TEMP_F,
  BT_GATT_CHRC_READ | BT_GATT_CHRC_NOTIFY,
  BT_GATT_PERM_READ, read_temp, NULL,
  &dw_bt_temp),
BT_GATT_CCC(dw_ccc_cfg_changed,
  BT_GATT_PERM_READ | BT_GATT_PERM_WRITE),
  BT_GATT_CHARACTERISTIC(BT_UUID_DW_BUZZ_CNTRL,
  BT_GATT_CHRC_READ | BT_GATT_CHRC_WRITE_WITHOUT_RESP | BT_GATT_CHRC_WRITE,
  BT_GATT_PERM_READ | BT_GATT_PERM_WRITE, read_buzz_ctrl, write_buzz_ctrl,
  &dw_bt_buzz_ctrl),
BT_GATT_CCC(dw_ccc_cfg_changed,
  BT_GATT_PERM_READ | BT_GATT_PERM_WRITE),
);

Nordic provides great documentation on how Bluetooth configuration is supported via NRC.

Once the I2C device overlays are in place, you can proceed to writing the device specific drivers for configuration and measurement reading. All I2C communication can be achieved using the i2c.h NRC drivers.

The TF-Luna can be configured and measured as follows. You'll notice the i2c_reg_read_byte_dt calls that handle the I2C communication protocol on your behalf!

const struct i2c_dt_spec* g_tfluna_dev_i2c;
int init_tfluna(const struct i2c_dt_spec *dev_i2c)
{
    g_tfluna_dev_i2c = dev_i2c;
    printk("Tfluna Init \n\r");
    uint8_t dev_reg[0x23] = {0};
    int ret = i2c_read_dt(dev_i2c, &dev_reg[0], 0x23);
    if (ret != 0)
    {
        printk("TF Luna Failed to write/read I2C device address %x at Reg. %x \n\r",
               dev_i2c->addr, dev_reg[0x22]);
        return 1;
    }
    printk("Tfluna Got data %x \n\r", dev_reg[0x22]);
    return 0;
}

uint32_t get_total_dist(uint8_t dist_l, uint8_t dist_h)
{
    return (dist_h * 0xFF) + dist_l;
}

float get_dist()
{

    uint8_t dist_h, dist_l;
    int rc = i2c_reg_read_byte_dt(g_tfluna_dev_i2c, REG_DIST_H, &dist_h);

    if (rc)
    {
        printk("Unable to read reg %x\n", REG_DIST_H);
    }

    rc = i2c_reg_read_byte_dt(g_tfluna_dev_i2c, REG_DIST_L, &dist_l);

    if (rc)
    {
        printk("Unable to read reg %x\n", REG_DIST_L);
    }

    return get_total_dist(dist_l,dist_h);
}

SHIELD UI Application

The M5Stack core libraries provide a rich set of capability to the underlying hardware. With just a few lines of code, one can easily configure the Bluetooth stack, control the touch screen and process user and Bluetooth events. The SHIELD UI reports the measured readings from the SHIELD device (both distance and temperature) and provides two buttons to connect to the SHIELD device and silence the auditory warning. The main run loop is very simple. It will read measurements at 10000 ticks (~ 1 Hz.) and update the displace at every tick.

void loop()
{
  app_s++;
  if ((app_s % 10000) == 0)
  {
    updateValues();
  }
  M5.update();
}

The Bluetooth initialization is well supported in the M5Stack as well (there are many examples on how to achieve this on GitHub). The following is an example on connecting to a Bluetooth server and discovering/configuring the desired characteristics.

for (;;)
  {

    BLEDevice peripheral = BLE.available();

    if (peripheral)
    {

      if (peripheral.localName() == "Add device well known name here")
      {
        // stop scanning
        BLE.stopScan();
        if (peripheral.connect())
        {
          Serial.println("Connected");
          g_is_connected = true;
        }
        else
        {
          Serial.println("Failed to connect!");
        }
        // Get dw service
        bool r = peripheral.discoverService(DW_SVC_UUID);
        Serial.printf("svc discover %d\n", r);
        BLEService dw_svc = peripheral.service(DW_SVC_UUID);
        Serial.printf("Svc uuid %s\n", dw_svc.uuid());
        g_temp_chr = dw_svc.characteristic(DW_TEMP_CHAR_UUID);
        Serial.printf("temp uuid %s\n", g_temp_chr.uuid());
        g_dist_chr = dw_svc.characteristic(DW_DIST_CHAR_UUID);
        Serial.printf("dist uuid %s\n", g_dist_chr.uuid());
        g_buzz_ctrl_chr = dw_svc.characteristic(DW_BUZZ_CTRL_CHAR_UUID);
        Serial.printf("buzz ctrl uuid %s\n", g_buzz_ctrl_chr.uuid());
        break;
      }
    }
    else
    {
      break;
    }
  }

If you made it this far in the project documentation THANK YOU! It was truly a fun and educational experience to design and build SHIELD. While SHIELD is still very much a prototype, I do see benefit in adding sensors and remote sensing abilities to help all people to include people with mobility disabilities.