Distributed Datalogging Network

Submission in Helium contest.

Acronyms

Raspberry Pi 3 - RPi

Google Cloud Platform - GCP

Message Queuing Telemetry Transport - MQTT

General Purpose Input/Output - GPIO

Secure Shell - SSH

Background

I wanted to have a distributed system of sensors that I could log data from. I wanted to be able to to just toggle the sensors on and off as needed. I wanted to answer questions like, "Does an object weigh more in vacuum due a decrease in buoyancy?" or "What is the temperature gradient across a room starting form the air vent?" or any other idea that pops into my head. I wanted code and a system I could reuse for multiple projects and experiments with minimal or no setup required.

Helium made it easier to register devices and send data to cloud platforms like Azure, Amazon Web Services, and Google Cloud Platform. Our system allows the user to plug in a sensor and record data with only a few configuration variables.

A system like this could be used in classrooms to compare results from small group experiments or at home for IoT data recording. A more robust version could be used in industrial applications to monitor machines or processes.

In addition to the steps I took in setting up my project, I will also provide background information about topics I learned during this process and system tests that can be performed to verify each step of the setup works. My goal is to help out anyone else who is a newbie like me.

Demo Video

System Overview

Hardware

I wanted sensors in multiple locations and have them communicate back to a hub. I chose to wire the sensors to ESP32 boards, which have many GPIOs and connect wirelessly. I chose the RPi as a hub for the ESPs to talk to since there is a Helium adapter for it.

Setting Configuration Variables

The user sets configuration variables in Google IoT. These variables set the ESP number, the sensor name, the pin number, the start time for data logging, the stop time for data logging, the sample rate, and the frequency at which the information is sent back to be graphed.

The ESP might read a value from the sensor at a sample rate of 1000 times per second (1000 Hz), but it might report the minimum, maximum, and average value from those 1000 samples only once every 5 seconds (1/12 Hz). This is a method of edge computing and limits the data throughput required between the ESP and the cloud.

Logging

The configuration variables are sent to the RPi from Google IoT through Helium. The RPi transmits the configuration variables to the proper ESP using MQTT. The topics are broken down by ESP. When the start time is reached, the ESP begins reading data from the sensors. For a digital sensor, the number of HIGH values is tracked by a count variable. For an analog sensor, the minimum, maximum, and average raw values are tracked at the reporting frequency.

Report and Graphing

The ESP sends the current time, counts, minimum, maximum, and average values for any actively sampled pins. There are about 20 GPIO pins on each ESP that are usable.

The RPi manipulates data sent by the ESP and passes it to the GCP via Helium. The Pub/Sub service receives this data. A Cloud Function parses the data, transforms it into something that matches the column headers of a table in Big Query. The Cloud Function then fills rows of the Big Query table with the incoming data from Pub/Sub. Data Studio pulls data from the Big Query table and then plots this information.

Information to Keep Track of During the Project

There are a few codes/keys/IDs/values that you will need to keep track of during the project. Here is the list of them:

• Static IP of RPi (uses for SSH and for MQTT server address): 192.168.1.XX

• HHV (4 digit code on Atom)

• MAC Address (last 4 digits on Element)

• Registry ID

• Region

• Project ID

• Database ID

• Table ID

I. Static IP for RPi

I setup a static IP for the RPi. There is a detailed video by Adafruit and a post on StackEchange. This is method for Raspbian Jessie and above.

Use ifconfig to find your IP address.

ifconfig

Then open the config file.

sudo nano /etc/dhcpcd.conf

and add the following to the the bottom of the config file (ip_address is the static IP of the Pi), save, and exit.

interface wlan0
static ip_address=192.168.1.XX
static routers=192.168.1.1
static domain_name_servers=192.168.1.1

Then reboot.

sudo reboot

Check in terminal to see the address.

nslookup <NameofYourPI>

II. Communication between ESP and RPi (MQTT)

What is MQTT?

MQTT is a messaging protocol for devices. There is a server/broker that directs messages between individual clients (devices) using topics. A client can subscribe and/or publish messages to topics. Publishing means sending a message to a topic. Subscribing mean asking for a message from a topic. Clients do not talk to each other. They talk only to the broker via topics. Multiple clients can publish or subscribe to any topic. MQTT is made for sending small packets of data frequently. Our limit seemed to be 55 bytes per message. Messages are called payloads.

I used the Moquitto MQTT broker on the RPi and the ESPs. I also installed it on my computer to test out the two devices. The RPi was treated as both a server and device, which means it routed messages and well as being able to send and receive them.

1. Setupon aComputer (optional for testing purposes)

brew install mosquitto

This says we want to use the standard configuration file

/usr/local/sbin/mosquitto -c /usr/local/etc/mosquitto/mosquitto.conf

2. Setting up on RPi

I followed this video by AdaFruit to install Mosquitto on the RPi. You can use AdaFruit's code to test the MQTT server. Their setup had a button you toggle on the RPi side to turn on an LED connected to an ESP. There is also button on ESP side that turns on an LED connected to the RPi.

This version is not secure. You can set up users and passwords. I did create certificates for encryption for the server on the RPi and the devices using Xcode and OpenSSL, but I could not get the code to work in time for this submission.

Code used to install:

sudo apt-get update
sudo apt-get install -y python3 python3-pip
sudo pip3 install RPi.GPIO paho-mqtt
sudo nano /etc/dhcpcd.config
wget http://repo.mosquitto.org/debian/mosquitto-repo.gpg.key
sudo apt-key add mosquitto-repo.gpg.key
cd /etc/apt/sources.list.d/
sudo wget http://repo.mosquitto.org/debian/mosquitto-jessie.list
sudo apt-get update
sudo apt-get install mosquitto
systemctl status mosquitto.service

Install command line tools to publish and subscribe to topics from the command line in the RPi:

sudo apt-get install mosquitto-clients 

I think I used this to start the MQTT server on the RPi:

sudo /etc/init.d/mosquitto start

3.Setting up theArduinoIDE

The video below gives an brief overview of what MQTT is and how the RPi and ESPs will use it. It also shows you how to set up a quick test between the two to make sure they are communicating.

Basically, I installed the PubSubClient library and took pieces from the ESP8266 and non-blocking examples. My testing code is on our GitHub.

III. Setting up Communication between RPi and Helium

What is Helium?

Helium simplifies registration and communication between devices and cloud platforms.

Helium consist of an Atom, an Element, a Dashboard, and Channels. The Atom attaches to the RPi. It is the communication link between the RPi and the Element. The Element is plugged into your router and can communicate with multiple Atom modules. The Element is the link to the Helium web dashboard. You can configure your Element, Atom, and Channels from the dashboard. You can also monitor communications between device hardware and cloud platforms. Channels are the platforms that the device will link to. These can be Google Cloud Platform, Azure, or Amazon Web server and MQTT server.

Helium has guides for setting up Atoms on Arduino, RPi and other embedded devices. They also have walkthroughs and configuration code for each of the cloud platform channels.

Connecting Helium to RPi

1. Hookup Atom

Plug the Atom into the RPi. Align Pin 1 of the module up with Pin 1 of the RPi. Note the four digit code that comes after HHV on the Atom. You will need this for the dashboard later.

2. Hookup Element

Plug the ethernet from the Element to your router. Plug the Element's power cord in. Record the last 4 digits of the MAC address on the Element.

3. Install Code on RPi

This is Helium's walkthrough guide for setting up Helium on Raspberry Pi. This is a VERY quick video just showing the bits of the screen recording during my install.

sudo apt-get install python-dev
sudo raspi-config

"Then select Interface Options > Serial and disable the login shell over serial, while enabling access to the Serial ports. Save the configuration and reboot the Raspberry Pi." - I didn't see this at first. I did the other steps and had to come back to this one - I'm not sure why.

I had issues with running Helium in Python 3 (discussed later), but the MQTT client we installed needed Python 3.

sudo pip

We have a Raspberry Pi 3. We had to do this special setup part:

sudo apt-get update
sudo apt-get upgrade
sudo rpi-update
sudo nano /boot/config.txt
sudo systemctl stop hciuart.service
sudo systemctl disable hciuart.service

4. Configure Dashboard

Go to https://www.helium.com/dashboard and sign up. You can add and activate an Atom and Element by hitting the plus buttons by each. They will need a name and the HVV and MAC numbers from the Atom and Element that you recorded.

Hit the plus button next to the channel to add a new channel. For testing, I used MQTT. For the project, I used Google IoT. I recommend testing with MQTT. To setup a Google channel you'll need to set up a console account first and setup a role in IAM to generate a certificate to use in the channel.

This code is from the Channel Code section of the dashboard. It imports the Helium library, connects to a serial port, connects to the proper channel on the Helium dashboard, and then sends a message to that channel.

from helium_client import Helium
helium = Helium("/dev/serial0")
helium.connect()channel = helium.create_channel("Helium MQTT")
channel.send("hello from Python") 

You can also read the configuration value from that key/value pairs on the dashboard. In this example, test1 was the key and the printed state was the value (RubarbPi). Later we just send config variables from GCP.

config = channel.config()
state = config.get("channel.test1")
print(state)

IV. Setting up the Google Cloud Platform

The goal is to get data from Helium into a graph. I followed parts of this guide. The video below is a screen share how to setup everything. This video follows the list of steps bellow.

1. Register for GCP.

2. Start a new project.

3. Register Device

Go to IoT Core and create a new device registry. Check both protocols and create a new telemetry topics for Pub/Sub.

3. Create Private Key

Go to IAM and create a new service account with Cloud IoT Editor permissions and a private key. The key will download.

4. Connect GCP to Helium

Go to the channel section of your Helium dashboard and start a new Google IoT channel. Use the device ID and region you used to register your device. Copy and paste the key you generated in IAM in to the JSON private key box.

5. Create a Table in BigQuery

Create a database and table in BigQuery. The schema should match the keys in the JSON file you will be sending over from the RPi.

6. Connect Pub/Sub to BigQuery with Cloud Function

Create a new Cloud Function that is triggered by the Pub/Sub topic you created when registering you device. You can use my simple code. Make sure to change the function to execute from HelloWorld to subscribe.

Step 6 Side Note: I have a version of the index.js where I tried to change transform the data sent from Pub/Sub before sending to BigQuery. There is a 100 byte limit on the packages sent from Helium to GCP. To stay under this, I used Epoch time and abbreviated many of the words. I wrote an version of the cloud function where it changes Epoch into Month/Day/Year and HH:MM:SS. It also swamps out abbreviations. Sometimes the RPi would lose connection when I ran this. Not sure why. Also, Data studio doesn't recognize seconds only HH:MM. I had a separate table in BigQuery for this version.

7. Data Studio

Set up an account. Create new data course and chose BigQuery and the table you created. Then you can create scatterplots, filters for ESPs and sensors, and scoreboards for min, max, and average values.

V. Collect Data

1. Upload the.ino files to the ESPs. Change the lines of the code pointed out in the Read.Me to allow the RPi to distinguish them.

2. Wire sensors to the ESPs.

3. Run the Helium code on your RPi using Python3.

4. Send configuration variables from GPC.

5. Monitor graphs in Data Studio.

VI. Future Feature Creep

I did not have time to add all the features I wanted before the contest deadline. The first thing I would add is security to MQTT. I created certs, but could not get communication to function using them. I might have to switch MQTT libraries on the Arduino to add higher QoS, which would allow devices to know if the message was received and in what order. I would add mapping/calibration data for every sensor on the RPi. I would add actuators (controlled output from ESP32), create a dashboard with Firebase with a better UI for configuration variables, data validation, and add I2C. I would also clean up the code. I learned Python for this project. I think adding classes, methods, and functions to both codes would be helpful.

Sidenote - Creating Certs for secure MQTT

I created these, but didn't integrate them yet.

From the terminal, I installed Xcode and generated an RSA private certification for the MQTT server I was setting up for the computer for testing purposes.

I used openssl, which is already installed, to create certificates. A breakdown of the openssl req can be found here.

xcode-select --install
openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -sha256 -days 3650 -out 

Creating server key:

openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 3650 -sha256

Creating certificates for devices:

openssl genrsa -out device001.key 2048
openssl req -new -key device001.key -out device001.csopenssl x509 -req -in device001.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out device001.crt -days 3650 -sha256 -addtrust clientAuth