ESP32 & Cellular IoT CI Build Status Light

I've been a software engineer for over twenty years, and one of the things that has always fascinated me is testing. Unit testing, integration testing, TDD, BDD, automated testing, CI tools: all of it gives me the warm fuzzies. A solid testing approach gives an engineering team the confidence that a product works as it should, and a safety net to refactor and evolve a codebase without fear of breaking everything.

When I joined blues wireless last year, one of the tasks I took on was maintenance of the open source libraries and SDKs in our GitHub organization. We have libraries for using the Notecard (our core hardware product) with Arduino, C/C++, Go, and my personal favorite language, Python.

I figured there was no better way to jump into maintaining a new library than to write some tests. And since I'm a sucker for new hardware projects, I decided to use the blues wireless Notecard to create an IoT status light to visually display the results of test runs and CI builds.

In this article, I'll show you how I:

• Assembled the status light with the Notecard, a Notecarrier, an ESP32 Feather, Relay and 24V Light Stack.
• Used an ESP32 and the blues wireless Feather Starter Kit to talk to the Notecard.
• Sent messages (called "Notes" in the blues wireless ecosystem) to the Notecard through Notehub.io from a GitHub Actions Workflow tied to the note-python project.
• Used an interrupt to inform the ESP32 of a new Note on the Notecard and set the appropriate color on the light stack based on the contents of the Note.

The complete source for this project, including ESP32 firmware and Workflow files for GitHub Actions are in the notecard-build-monitor GitHub repo.

What is the Notecard?

The hardware I chose for this project is the blues wireless Notecard. The Notecard is a cellular and GPS-enabled device-to-cloud data-pump that comes with 500 MB of data and 10 years of cellular service for $49. No activation charges, and no monthly fees.

The Notecard is a small 30 by 34 mm SoM that’s ready to embed in a custom project via its m.2 connector. But blues also provides a series of expansion boards, called Notecarriers, that include an m.2 socket for the Notecard and a number of other features for prototyping. For this project, I used the Notecarrier-AF, which has onboard cellular and GPS antennas, a LiPo battery port, Grove and Qwiic connectors, and a Feather-compatible header socket, making it perfect for this project.

On the cloud side, the Notecard is preconfigured to securely talk to Notehub.io, the blues wireless service that enables secure device-to-cloud data flow. Notecards are assigned to a project in Notehub.io, which then sync data into those projects for routing to your cloud application. Notehub.io also allows the Notecard to function as a bi-directional device that can receive data in addition to publishing data.

Configuring the Hardware

My hardware setup for this project consists of the following:

• A blues wireless Feather Starter Kit, which includes a Notecard, a Feather-friendly Notecarrier-AF, and an Adafruit HUZZAH32 ESP32 Feather.
• An Industrial LED Signal tower. My tower is a 24V tower with five lights, but any tower will do.
• A 5V Relay Module with at least as many channels as lights you need to control. The one I have from ELEGOO has 8 channels and is pretty easy to set-up.
• A Phase Dock Workbench. I have two of these kits and they are my go-to for prototyping something new, and even for traveling with a project, back when we would go places and do things.

I started by connecting the light stack to my relay. Each colored wire corresponds to a signal light on the stack. Then, I connected five white wires to each channel on the relay.

I then connected all five white wires to the grey common wire on the light stack and secured them all with a wire nut. With the Phase Dock WorkBench, I was able to organize the mass of wires on the underside of the platform and keep my project looking clean.

My Light Stack didn't come with a barrel jack connector, so I wired one up and plugged in a 24V 5A power adapter.

Next, I wired the Feather Kit to the Relay using jumper wires, connecting power and ground to the relay, and each channel on the Relay to a GPIO pin on the ESP32.

Finally, I added a jumper wire between a GPIO on the ESP32 and the Notecard ATTN pin. This handy little pin allows me to configure the Notecard to fire an interrupt in my firmware when one or more conditions are met.

Creating a Project on Notehub.io

My next step was to create a new project at Notehub.io. It's a pretty straightforward process and the blues wireless developer portal contains step-by-step instructions for creating an account, projects, and more. The key thing a new project gets me is an identifier called a ProductUID, which is used to associate a Notecard to Notehub.io so all my data ends up in the right place.

Writing the Notecard Configuration Firmware

With my ProductUID in hand, I was ready to write the firmware. The full source is in GitHub, but I'll cover the major portions below.

For this project, I used the Arduino IDE and instructions from the blues wireless developer portal to get it configured for the ESP32.

First, I included the Notecard header from the note-arduino library and added some pin directives for readability, with each _LIGHT directive mapping to the GPIO I wired to the channel for that light on the relay.

#include <Notecard.h>
#include <Wire.h>

#define myProductID "com.blues.bsatrom:build_monitor"
Notecard notecard;

// Light Stack Pin Mappings
#define RED_LIGHT    13
#define BLUE_LIGHT   27
#define GREEN_LIGHT  A1
#define ORANGE_LIGHT A5
#define WHITE_LIGHT 32

Each light corresponds to a possible state of a build on the note-python repo:

• Red if my unit tests (and hence the build) fail.
• Blue if a new build has started.
• White if the unit tests are running.
• Orange if the deployment fails.
• Green if everything was successful.

Next, I added some directives related to interrupts from the Notecard. First, the GPIO wired to the Notecard's ATTN pin, and directives needed for configuring the Notecard.

#define ATTN_INPUT_PIN 14

#define INBOUND_QUEUE_NOTEFILE      "build_results.qi"
#define INBOUND_COMMAND_FIELD "result"

Then, I created some variables needed for managing the interrupts and build status in the main part of the program.

static bool attnInterruptOccurred = false;
String buildStatus = "success";
static bool statusChanged = false;

In the setup() function, the first thing I did was to configure my GPIOs and initialize the Notecard library with notecard.begin(). The Notecarrier-AF provides a prewired connection between the Notecard and any Feather device over I2C. This method establishes the I2C connection so I'm ready to talk to the Notecard from firmware. I also called a helper function to make sure that all my lights are off when the program first starts running. My relay is active LOW, so be sure to test yours our for yourself if you're following along.

pinMode(RED_LIGHT, OUTPUT);
pinMode(BLUE_LIGHT, OUTPUT);
pinMode(GREEN_LIGHT, OUTPUT);
pinMode(ORANGE_LIGHT, OUTPUT);
pinMode(WHITE_LIGHT, OUTPUT);

// Turn all the lights off through a helper function that brings 
// the relay pin HIGH
allLightsOff();
  
// Initialize the I2C connection to the Notecard
notecard.begin();

Next, I was ready to configure the Notecard for my project. The Notecard uses a JSON-based API. It takes JSON requests and returns JSON responses. You don’t have to learn or use any AT commands to work with this device. If you’re interested in learning more, there’s a quickstart on the developer portal that introduces the basic concepts and commands.

First, I used a hub.set request to assign my Notecard to a product in Notehub.io. The note-arduino library comes bundled with a JSON library and helpers that make it easier to work with JSON in C.

J *req = notecard.newRequest("hub.set");
JAddStringToObject(req, "product", myProductID);

notecard.sendRequest(req);

Then, I sent another hub.set request to configure how the Notecard synchronizes data with Notehub.io (I don’t have to send these requests separately, but I did so here for the sake of readability). The mode indicates that the Notecard maintains an active connection to Notehub.io. inbound means that the Notecard checks for incoming data at least every 240 minutes. outbound means that the Notecard will upload its data to Notehub.io no less often than every 60 minutes. Finally sync:true means that I want the Notecard to automatically and immediately sync each time a change is detected in Notehub.io. That's important for this project because I want to update my light stack based on changes to a CI build.

req = notecard.newRequest("hub.set");
JAddStringToObject(req, "mode", "continuous");
JAddBoolToObject(req, "sync", true);
JAddNumberToObject(req, "outbound", 60);
JAddNumberToObject(req, "inbound", 240);

notecard.sendRequest(req);

Next, with a card.attn request, I configured the Notecard to watch for changes to a file ("build_results.qi" defined above) and fire whenever a new Note shows up in that file.

req = notecard.newRequest("card.attn");
const char *filesToWatch[] = {INBOUND_QUEUE_NOTEFILE};
int numFilesToWatch = sizeof(filesToWatch) / sizeof(const char *);
J *filesArray = JCreateStringArray(filesToWatch, numFilesToWatch);

JAddItemToObject(req, "files", filesArray);
JAddStringToObject(req, "mode", "files");

notecard.sendRequest(req);

Finally, I configured the ATTN Pin GPIO as an input and configured it as an interrupt using the Arduino attachInterrupt and digitalPinToInterrupt helper functions. The ESP32 has its own board-specific interrupt APIs and there's an example for using those here, but I chose to use the Arduino APIs for this project to keep things a bit more portable between MCUs.

// Attach an interrupt pin
pinMode(ATTN_INPUT_PIN, INPUT);
attachInterrupt(digitalPinToInterrupt(ATTN_INPUT_PIN), attnISR, RISING);
 
// Arm the interrupt, so that we are notified whenever ATTN rises
attnArm();

The attnISR function sets the global attnInterruptOcurred variable to true and attnArm arms or rearms the Notecard after an interrupt occurs.

void attnISR() {
  attnInterruptOccurred = true;
}

void attnArm() {
  // Make sure that we pick up the next RISING edge of the interrupt
  attnInterruptOccurred = false;

  // Set the ATTN pin low, and wait for the earlier of file modification
  // or a timeout
  J *req = notecard.newRequest("card.attn");
  JAddStringToObject(req, "mode", "reset");
  JAddNumberToObject(req, "seconds", 120);
  notecard.sendRequest(req);
}

Writing the Light Stack Firmware

With the Notecard configured, I was ready to set up the light stack interactions. Inside my loop I added code to check the state of the interrupt variable. If an interrupt has occurred, I rearm the ATTN pin and process inbound Notes from the Notecard using a helper function.

void loop() {
  // If the interrupt hasn't occurred, exit
  if (!attnInterruptOccurred)
    return;

  // Re-arm the interrupt
  attnArm();

  // Process all pending inbound requests
  checkBuildStatus();    
}

The checkBuildStatus function retrieves Notes from the "build_results.qi" Notefile using a note.get request. If a Note is returned, I extract the "result" field from the Note body and check to see if the build status has changed. If it has, I update the status and call another helper function to turn on the appropriate light, based on the status message.

void checkBuildStatus() {
  while (true) {
    J *req = notecard.newRequest("note.get");
    JAddStringToObject(req, "file", INBOUND_QUEUE_NOTEFILE);
    JAddBoolToObject(req, "delete", true);
    J *rsp = notecard.requestAndResponse(req);
    if (rsp != NULL) {
  
      if (notecard.responseError(rsp)) {
        notecard.deleteResponse(rsp);
        break;
      }
  
      J *body = JGetObject(rsp, "body");
      if (body != NULL) {
  
        char *incomingStatus = JGetString(body,INBOUND_COMMAND_FIELD);
        
        // Determine if the status has changed and update accordingly
        if (strcmp(incomingStatus, buildStatus.c_str()) != 0) {
          buildStatus = String(incomingStatus);
          statusChanged = true;
         
          updateBuildLight();
        }
      }
  
    }
    notecard.deleteResponse(rsp);
  }
}

updateBuildLight checks the value of the buildStatus variable and performs a digitalWrite to pull a relay channel LOW, turning on a corresponding light.

void updateBuildLight() {
  allLightsOff();
 
  if (buildStatus == "building") {
    digitalWrite(BLUE_LIGHT, LOW);
    statusChanged = false;  
  } else if (buildStatus == "running_tests") {
    digitalWrite(WHITE_LIGHT, LOW);
    statusChanged = false;  
  } else if (buildStatus == "success") {
    digitalWrite(GREEN_LIGHT, LOW);
    statusChanged = false;  
  } else if (buildStatus == "upload_failed") {
    digitalWrite(ORANGE_LIGHT, LOW);
    statusChanged = false;  
  } else if (buildStatus == "tests_failed") {
    digitalWrite(RED_LIGHT, LOW);
    statusChanged = false;  
  }
}

Calling the Notehub.io API from GitHub Actions

After applying firmware to my ESP32, the next step was to modify the CI builds in the note-python repo to call the Notehub.io API and send Notes to my Notecard.

The note-python repo uses GitHub Actions for linting the code, running unit tests, and uploading new releases to PyPi. I have a workflow defined to run on each PR and merge to the main branch, and another to build installation packages and upload to PyPi when a new tagged release is created. Workflows are defined as YAML files and can be edited locally, or in the browser.

Notehub.io provides an API that can be used to communicate with the Notecard, set environment variables, and more. By sending a note.add request, I can enqueue a message in a Notefile on Notehub.io that will be synchronized to the Notecard. Once synchronized, the ATTN pin fires, the ESP32 retrieves the note and sets the relay channel that corresponds to a given light.

For the workflows in note-python, I added a cURL request for each time I wanted to send a status notification to the Notecard. All requests use the "build_results.qi" Notefile and send the status as the value of the result key in the body. The status values map to the buildStatus values used above. I'm also using secrets from my GitHub repo to keep product, device and token information safe.

- name: Send building notification
run: |
  curl --request POST \
  --url 'https://api.notefile.net/?product=${{ secrets.NOTEHUB_PRODUCT_UID }}&      device=${{ secrets.NOTECARD_DEVICE_ID }}' \
  --header 'Content-Type: application/json' \
  --header 'X-Session-Token: ${{ secrets.NOTEHUB_SESSION_TOKEN }}' \
  --data '{"req":"note.add","file":"build_results.qi","body":{"result":"building"}}'

Running a Build

With my GitHub workflow changes deployed, it was time to power up my Feather Kit, plug in the light stack and run a build. The video below shows the completed project in action.

The video above illustrates the happy path of my build process, but we all know not every build ends well. For instance, if the linter or tests fail, the red light will let me know.

And if everything passes, but the PyPi upload fails on upload, my light stack will activate the orange light.

All that's left now is to mount the project on my office pegboard, plug it in, and rest easy in the knowledge that my Notecard is keeping tabs on the build.

The Notecard is a powerful little device, and with the Feather Kit, its easy to get started with cellular IoT that's finally developer-friendly. To learn more about the Notecard, visit blues.io. I can't wait to see what you build.