Breeze - Smart Environmental Sensor Hub

Monitoring of the air quality and other environmental factors is often insufficient in many areas, and this leads to situations when the people and authorities are not aware of the air quality and environmental issues, and thus not action is taken.

In this project I will present Breeze the Smart Environmental Sensor Hub, which offers a solution to this problem.

Breeze is a battery powered PoC device with the following features:

• based on the QuickFeater development board, featuring the EOS S3 SoC
• custom made carrier board with sockets for the QuickFeater board, 3 x Sensor Modules, 1 x Communication Module and a battery
• 2 x MikroE Click (small) sockets for Sensor Modules - each socket supports I2C, SPI, UART communication, Chip Select and 3V power - one socket also has 5V power
• 1 x MikroE Click (medium) socket for a Communication Module - socket supports I2C, SPI, UART communication, Chip Select and 3V power
• 1 x socket for a Lightning Detection Sensor Module
• each Sensor and Communication Module socket can be fully powered down individually using the load switches installed on the carrier board
• the two installed Sensor Modules are a UV Light sensor and a Air Quality sensor modules - other Sensor Modules are supported
• the Communication Modules used is a WiFi ESP Click providing communication over WiFi - other Communication Modules, providing communication over BLE, GSM, LoRaWAN, etc, are also supported
• powered by a 18650 Li-ion Battery
• battery charging using a 330mA @6V Solar Panel and on-board charge circuit
• battery charging over micro-USB

Concept

Breeze is Sensor Hub capable of collecting data from the connected Sensors. The sensor data is pre-processed, and Machine Learning (ML) are used to extract meaningful information from the models. The Communication Module is used broadcast the sensor data and the ML prediction to a Cloud Backend.

The Sensor Hubs are the hardware devices responsible for collecting data from a number of Sensors. The data is pre-processed, and additionally ML models can be ran to extract meaningful data from the sensor data. A Communication Module is used to forward the sensor data and ML Predictions to a Gateway.

The Gateway is the component responsible for collecting data from one or more Sensors Hub and forwarding them to a Cloud Backend. The Gateway can be either a simple hardware router / gateway for WiFi / BLE / GSM / LoRa, or a combined hardware and software solution that acts as a proxy between the Sensor Hub and the Cloud Backend.

Finally, the Cloud Backend is the place were the data and prediction from all the Sensor Hubs are collected together and analyzed.

Now lets get into details! :)

Hardware

The project is based on the QuickFeather (v1.2) development board, an open-source hardware development kit from QuickLogic based on the EOS S3 MCU + FPGA:

The QuickFeather is It features a 16-Mbit flash memory and several built-in sensors. It can be powered either from a Micro-USB port or a Lithium battery, which is automatically charged when USB power is present.

As wanted to use some external sensors with the QuickFeather, along with several external components, I decided to design a carrier board with the following features:

• two MikroE Click Small sockets for external Sensors
• a MikroE Click Medium socket for a Communication Module
• a socket for a Lightning sensor
• 18650 type Battery Holder
• Solar Panel connector and charging circuit

The MikroE Click sockets provide 3.3V power, UART, I2C and SPI connectivity, and support Interrupts. One of the Sensor sockets also provides 5V power for modules that needs this.

To allow low power consumption, each of the external modules can be individually powered down. This is achieved using load switch IC-s controlled from GPIO pins of the QuickFeather. This means we don't need to rely on power-saving mechanism implemented by different modules.

I designed the carrier board in KiCad. The Schematics was done in Eeschema and looks like this:

The two-layer PCB was designed in PCBNew:

When the design was done, I ordered 5 black PCB-s from JLCPCB, along with the necessary components ordered from Mouser and LCSC.

This is how the PCB-s came out:

The next step was to solder all the components to the PCB:

We can then install the QuickFeather board and the Sensor modules:

The Solar Panel and the Air Quality Sensor are being mounted on a 3D Printed holder designed in FreeCAD:

This is what the assembled device looks like:

Sensors and Communication Modules

To monitor different environment factors a set of Sensors will be used. These include both built-in sensors of the QuickFeather development board, as well as external sensors.

First, lets see what built-in sensors on the QuickFeather board has, and for what we can use them:

The mCube MC3635 accelerometer offers 3-axis motion sensing. I want to use it for earthquake detection, and as well to monitor vibrations common in urban environments (ones caused by subway, heavy traffic, etc).

The Infineon IM69D130 is PDM digital microphone. No, we will not use it for wake-up word detection this time. :) But for monitoring noise pollution in urban environments.

The Infineon DPS310 pressure sensor is sensor provides atmospheric pressure and temperature measurements. Always useful for weather monitoring and forecasting.

On the external sensors side, the carrier board accepts two MikroE Click modules and a custom sensor module:

An Plantower PMS7003 sensor is an Air Quality sensor module capable of measuring amount of PM1, PM2.5 an PM10 particles in the air. The sensor will be connected to the carrier board using a DIY Click board adapter.

To monitor potentially dangerous UV radiation emitted by the Sun, a UV 3 Click module will be used. The module is based on the Vishay VEML6070 UVA Light Sensor using the I2C protocol.

Last but not least a CJMCU-3935 module will be used do detect Storms and Lightnings. The module is based on the AMS AS3935 Franklin Lightning Sensor IC which uses 500 kHz antenna and some custom algorithms to detect upcoming storms and lightnings. The modules connect to the carrier board through a custom 11-pin header.

Along the choosen Sensors the hub also support other sensor with the MikroE Click form factor:

On the outgoing side the collected sensor data is broadcast-ed using a Communication Module.

I opted for the for the MikroE WiFi ESP Click, based on the ESP-WROOM-02 modules:

The MikroE WiFi ESP Click provides WiFi connectivity over an AT command interface.

Other MikroE Click format modules are also supported. The can provide different type of connectivity like Bluetooth, GSM/3G/4G, LoRa and others:

Device Firmware

The Device Firmware is the piece of software running on the QuickFeather Dev Board's brain the EOS S3 Low Power MCU with an embeded FPGA.

It purpose it's to interface with Sensors, collect and analyze data, and transmit valuable information using Communication Module.

The firmware of this project is based on a modified version of the QF SSI AI App application from the QORK SDK.

The firmware is structured as follows:

• on the bottom we have the Cortex M4F core and eFPGA from the EOS S3 - its functionality can be accessed using the EOS S3 HAL (Hardware Abstraction Layer)
• low level Drivers are used to communicate with Sensor, Communication and Power Management Modules
• FreeRTOS is a real-time operating system responsible for task scheduling and data management
• QORK SDK offers a framework and libraries for various tasks like sensor data management
• the Application layer glues all this together - collects the Sensor Data, structures them in Data Blocks, and send the data to a remote compute

The QF SSI AI App works by collecting Sensor Data and sending over UART using the Simple Streaming Interface protocol. It also has a mode which allows running ML Models built using the SensiML Analytics Toolkit.

Over this functionality, I implemented additional features like Sensor Data down-sampling and running ML Models in data collection mode:

To implement the firmware there were a the following of steps necessary:

• implement a Power Management and Chip Select Module - this uses GPIO (native + FPGA based) to handles the Load Switches ICs (powering the external modules) and Chip Select pins
• implement / build custom FPGA bitstream for GPIO functionality
• implement low level Drivers for the Sensor Modulues - the UV and Lightning sensors uses I2C communication, the Air Quality sensor works over UART
• implement low level Driver for the Communication Module - the driver for WiFi ESP Click was implemented, this uses AT commands over UART
• implement sensor data down-sampling
• implement support for running ML Models along with data collection

To work on the project I used Embedded C/C++ version of Eclipse. To set a QORK SDK project with Eclipse, the linked README file can be used:

For programming and debugging a standard SEGGER J-Link / SWD adapter was used.

ML Models

Machine Learning models can be used extract meaningful information from Sensor data that is otherwise hard to interpret.

Running ML models on low power IoT edge devices allow the data to be processed locally. This way the amount of data needed to be sent to the cloud can be drastically reduced, thus lowering the power consumed by communication modules.

Bellow I will demonstrate how to use this method to detect Earthquakes from the Accelerometer data of the QuickFeather board.

As earthquakes are now as common events, and I did wanted to generate just by randomly shaking the QF board, I decided to collect from scientific papers. There are good number of papers (see the references section) analyzing accelerometer data during earthquakes. I tough to use the data from these papers.

The scientific papers contained earthquake accelerometer data in form of graph plots. These graphs were extracted and pre-edited them using Gimp.

The the data was extracted from the images using a Jupyiter notebook with Python:

• first the images were loaded as grayscale images using OpenCV

• the images are were transformed to black-and-white using a Thershold operation - this also eliminated noise present in the white background of the images
• the images were organized in Data Sets, with metadata (time and vertical scale, etc) associated to each
• the min/max values were extracted as time series from each image

• the min/max values were interpolated to reconstuct the acceleration data - the data was reconstructed to a sample rate of 100 Hz; the initial images have a resolution of about 1-6 Hz

• the 8 earthquake data samples were shuffeled, and concatenated to single time series; randomly generated idle periods are used between the samples
• using horizontal and vertical scaling, along with some randomization teqniques, an additional 128 earthquake samples were generated - the data then was concatenated using the above method

• the time series were transformed to match the scale of the 14 bit values generated by the MC3635
• the resulting data was exported in a big CSV file (193 MB) - we will import this into the Data Capture Lab for Labeling; we will use about 80% of the samples for Training and 20% for testing

The next step was to import the resulting CSV file in the SensiML Data Capture Lab:

and set up some Labels (Calm, Small/Medium/Big Earthquake) and Metadata (Train / Test):

The data then was labeled into two datasets:

• Train data - 200 samples (100 earthquake + 100 calm) - about 80% of total

• Test data - 64 samples (32 earthquake + 32 calm) - about 20% of total

The results are saved in our SensiML clould account, and we are ready to go to SensiML Analytics Studio to train some models.

The first step is to prepare a Query like this:

Then we can start experimenting with models in the Build Model section:

I experimented with multiple Models, with different Window Size-s and Optimization Metrics. This is what I got:

• Window Size = 1024 - 72% accuracy, 69% f1-score
• Window Size = 2048 - 75% accuracy, 76% f1-score
• Window Size = 4096 - 84% accuracy, 86% f1-score

Using a larger Window Size (4096 - about ~40 sec @ 100 Hz) gives as higher Accuracy, on the other hand a smaller Window Size results in lower Latency.

I ended up chossing the model with Window Size 2048 (about ~20 sec @ 100 Hz). It has a 75% accuracy, but the Confusion Matrix show that it only confuses Small / Medium and Big Earthquakes between them, but not Calm periods with Earthquakes.

Using the Explore Model functionality we can find out different properties of our model:

• the model is a Decision Tree Ensemble with 60 estimators
• there are 19 features used from the Statistical and Rate of Change categories - this are generated from the X, Y, Z accelerometer data

Using the Test Model tab we can cross-validate or model:

Now, it show 10% but that's not really the true picture :D. Probably did something wrong here. There are ton of Unknows, detected mostly as Calm. This is just data that is not labeled. Otherwise, the model has around 53% to 93% accuracy. The mismatches are mainly between Small / Medium / Large Eearthquakes.

Next we can download the Knowledge Pack to be run on the QuickFeather:

I opted for the Library format, as we also have other functionaly running in the QuickFeather.

The knowledge pack comes as a ZIP file and can be integrated in the qf-ssi-ai-app by running the provided bash script. Then one option is to run the app in detection mode, which send the result on the UART. I opted to not use this, but to treath detection as it would been sensor data.

The plan is add additional ML Models to extract more meaningful data from the sensors. One example would be to the PDM microphone along with ML Models to measure and classify environmental noise in certain areas.

Gateway and Cloud Apps

The Gateway component in this PoC is based on the sensiml/open-gateway application.

A couple of new feature were added in order to accommodate the changes made in the firmware:

• a new connection mode TCPIPSERVER was added - this is similar to the TCPIP mode, but the Gateway is acting as a server, while the QuickFeather connect to this
• a bug was causing the Sensor Columns to be incorrectly ordered was fixed
• the export functionality was enhanced to support export to InfluxDB and Grafana

This is what the collected data looks in the Gateway application:

The Gateway application can be configured to export data into a InfluxDB + Grafana combo. This represents the Clould Backend in this PoC.

In Grafana we can configure Dashboards showing the sensor data:

Alerts can also be set up, to generate notifications when the sensor value breaches a threshold.

Further Improvements

This project is a PoC and so it is far from being complete.

Some enhancements I plan to make are:

• build more ML Models
• implement driver for the built-in Infineon IM69D130 PDM digital microphone
• use the Audio data to measure environmental noise, build an ML Model to classify different types of environmental noise
• implement driver for the built-in Infineon DPS310 pressure sensor - this will provide atmospheric pressure and temperature data
• fix an instability issue in the UART communication between the QuickFeather an the WiFi ESP Click

Hope you enjoyed this project! :)

References

Full-scale structural monitoring using an integrated GPS and accelerometer system

Prediction of Earthquake Using 3 Axis Accelerometer Sensor (ADXL335) and ARDUINO UNO

Review of Monitoring Parameters of the Kostanjek Landslide (Zagreb, Croatia)

Analysis of Strong-Motion Accelerograph Records of the 16 April 2016 Mw 7.8 Muisne, Ecuador Earthquake

The 2015 Gorkha Nepal Earthquake: Insights from Earthquake Damage Survey

Derivation of Non-stationary Stochastic Processes Compatible with Seismic Response/Design Spectra