Environment station using Google cloud IoT and MQTT

This assignment has been done for the "Internet of Things" course at "La Sapienza University of Rome". As said in the description, the goal, for this first part, is to build a cloud-based IoT system that collects information from a set of virtual environmental sensors using the MQTT protocol. However, it has been composed during a set of assignments, increasing a lot the number of features over time. All the updates are viewable in the other articles I have published.

Since we are all in quarantine, it is hard to find the hardware, to build a real IoT application. However, learning never stops! Therefore, we simply virtualized the sensors, to practice anyway with the IoT technologies.

For the demonstration

Note: this video shows how this adventure begins, exposing what I have done in a few days, using some technologies that I already knew and many other that I have never used before. However, of course, after many updates, the code, the technology used, and the user experience, changed and increased a lot. This is only a first taste of the final project that you will find on my Github.

Structure overview

Here the components:

• 5 virtual sensors, implemented as simple nodejs programs, exchanging telemetry values with the Google cloud platform using the MQTT protocol;
• The Google cloud platform, providing the Cloud IoT core and Pub/Sub API, used to manage the connection, the devices and the messages exchanged with the web application;
• The web application, built using nodejs + express and some other related frameworks;
• The Mongodb NoSQL database for the storage;

Note: the code provided on my Github is made to work with my credentials, on my personal google account. I did not upload the keys, If you want to run it, you need to make your own setup.

Introduction to MQTT

MQTT (Message Queuing Telemetry Transport) is an open OASIS and ISO standard (ISO/IEC 20922) lightweight, publish-subscribe network protocol that transports messages between devices. The protocol usually runs over TCP/IP; however, any network protocol that provides ordered, lossless, bi-directional connections can support MQTT. It is designed for connections with remote locations where a "small code footprint" is required or the network bandwidth is limited.

The broker acts as a post office, MQTT does not use the address of the intended recipient but uses the subject line called “Topic”, and anyone who wants a copy of that message will subscribe to that topic. Multiple clients can receive the message from a single broker (one to many capability). Similarly, multiple publishers can publish topics to a single subscriber (many to one).

Note that in this case we will use multiple publishing devices, but only one subscriber, or at least only one subscriber that saves the messages, since the database is connected to the broker through the web dashboard.

Cloud platform setup

As any engineer knows, the Google cloud platform, offered by Google, is one of the most powerful suites of cloud computing services on the market. In this project, it is used as a cloud-based backend to control MQTT, and implement the Publish/Subscribe messaging pattern.

The setup is really easy and requires the following steps:

• Access the Google cloud platform and select the IoT core API;
• Create a register and 5 different devices (with the corresponding keys), one for each sensor: temperature, humidity, wind direction, wind intensity, rain height;

• Add to the register a topic and a subscription, using the Pub/Sub API;

Some useful links for the setup are:

• https://cloud.google.com/iot/docs/quickstart?authuser=1
• https://cloud.google.com/iot/docs/how-tos/devices?authuser=1

Note: the pull operation will require the authentication, make sure to create a service account key as simply explained in the following link https://cloud.google.com/docs/authentication/getting-started

The sensors

Since the code is mostly based on the one provided by the Google documentation, I will not go into depth explanation, indeed it only needs a fast setup and some changes in the publishAsync() function.

const direction = getRandomArbitrary(0, 360);
var date = parseInt(Date.now());
const payload = "virtual:"+" "+deviceId+" "+direction+" "+date;

console.log('Publishing message:', payload);
client.publish(mqttTopic, payload, {qos: 1});
publishAsync(mqttTopic, client, iatTime, 1, connectionArgs);

We have set the Pub/Sub API to accept only one topic for each kind of telemetry, so we need to specify from which device has been sent the message, as shown in the code. Therefore, each message published is composed of:

• The "virtual device" indicator, introduced for compatibility with the next assignments
• deviceId, which tells the type of telemetry sent by the device
• value of the telemetry
• date

Assuming that we do not want to lose any value sent, I set qos: 1, to allow the retransmission of the messages.

Finally, to run one of the sensors, you only need to change the following setup

// SETUP         
const projectId = `your_project_ID`;
const deviceId = `your_device_ID`;
const registryId = `your_register_ID`;
const region = `your_region`;

Then run:

$ node device_name.js

The execution will result:

Here the link to the Google cloud platform referring how to publish over the MQTT bridge:

• https://cloud.google.com/iot/docs/how-tos/mqtt-bridge?authuser=1#downloading_mqtt_server_certificates

The web application

The code is based on an MVC-like structure and uses different technologies that make it looks better, but these are not relevant for the main goal of this project, so I will just provide some references in the conclusion.

However, the picture shows the portion of the code that manages the MQTT interaction. Indeed, the listenForMessages() function, allows the application to pull the messages from Google cloud and then save the telemetries, after sending a socket.io emit to the home view.

Also in this case, to use the code, you only need to change the subscription setup and generate a config/keys.js file where to put the URI for mongoose. Then you can run:

$ node app.js

As for the other sections, here there is a link to the corresponding section of the Google documentation:

• https://cloud.google.com/pubsub/docs/pull

Note: to simply store the data inside the MongoDB, you can create an account inside mLab at this link: https://mlab.com/.

Conclusion

On the home page, the app will show the current values, refreshing them any time a new value comes. On the other pages instead, you can see the old values.

Web app technologies:

• https://nodejs.org/en/
• https://getbootstrap.com/
• https://handlebarsjs.com/

for the next part use this link:

• https://www.hackster.io/ivagnesmanuel/environment-station-using-riot-google-iot-mqtt-5069b8