Full stack RPI Chirpstack LoRaWAN Environment Dashboard

Introduction

Hi, I am Sidik from sidik.my - An accountant with interest in programming and robotics.

In my previous project, I demonstrated Point-to-Point (P2P) communication between two Grove LoRa-E5 via AT+TEST mode using CircuitPython

This AT+TEST mode is suitable for testing purpose but not for production. The main application of a LoRa Node is to be part of a larger LoRaWAN (Long Range Wide Area Network) ecosystem.

In this project, I will share how we can use Grove LoRa-E5 as a LoRaWAN node.

This will use AT+JOIN command and other AT command needed to join real LoRaWAN network.

Full AT command for Grove LoRa-E5 can be downloaded here

Full Stack LoRaWAN Raspberry Pi local network Environment Dashboard

A LoRaWAN network server can be run either :

• cloud hosted (e.g The Thing Stack, Helium, AWS IOT core for LoRaWAN); or
• self-hosted (e.g Chirpstack, TTS Enterprise or Open Source).

Both can be either free or paid based on features and support provided by the software maker.

I will also demonstrate how to built a complete LoRaWAN network on a self-hosted Raspberry Pi , using Chirpstack.

The use case of this self-hosted LoRaWAN network is:

• For learning and development where we want to rapidly test the function and features.
• For production where we want to contain the network within our own premise.

The full data flow will be:

Grove LoRa E5 Node -> RAK2245 LoRaWAN Gateway -> Chirpstack LoRaWAN Bridge -> Chirpstack LoRaWAN network server - > Chirpstack LoRaWAN application server -> MQTT -> Node-RED -> InfluxDB -> Grafana

And followings are the descriptions of each component:

Node : Grove LoRa E5 STM32WL module node is connected to microcontroller (Seeeduino Xiao RP2040) which is also connected with Sensors and OLED Display

Gateway & Servers: All other part will be installed on Raspberry PI. I installed RAK2245 gateway and Chirpstack Servers on Raspberry Pi 3b+

• RAK2245 LoRaWAN Gateway -LoRa gateways are radio modules that serve as communication devices within a LoRa network between end-devices and a LoRaWAN network server (LNS). A LoRa gateway is mostly used to transmit sensor data from an electrical device to the cloud. Especially in environments where other types of networks are not viable due to technical limitations, LoRa gateways can be used to establish a network implementation of relevant electrical devices. LoRa technology itself is only used for communication between the respective end devices and the gateways connected to them. (source)
• Chirpstack LoRaWAN Bridge -is a service which converts LoRa Packet Forwarder protocols into a ChirpStack Network Server common data-format (JSON and Protobuf) (source)
• Chirpstack LoRaWAN network server de-duplication of received LoRaWAN frames by the LoRa® gateways and for the collected frames handle the Authentication, LoRaWAN mac-layer (and mac-commands), Communication with the ChirpStack Application Server. Scheduling of downlink frames (source)
• Chirpstack LoRaWAN application server -responsible for the device "inventory" part of a LoRaWAN infrastructure, handling of join-request and the handling and encryption of application payloads (source)
• MQTT - an OASIS standard messaging protocol for the Internet of Things (IoT). It is designed as an extremely lightweight publish/subscribe messaging transport that is ideal for connecting remote devices with a small code footprint and minimal network bandwidth. MQTT today is used in a wide variety of industries, such as automotive, manufacturing, telecommunications, oil and gas, etc. (source)

Integration : Node-RED, InfluxDB & Grafana is installed on Raspberry Pi 4b. It is possible to install both servers and integration on the single Raspberry Pi but I already installed these three softwares on raspberry pi 4b for my other project, so I just left it there.

• Node-RED - To receive data from MQTT ,clean-up, convert and filter the data and connect to InfluxDB database
• InfluxDB - To store time series data in database
• Grafana - To retrieve data from InfluxDB and display time series data as nice chart, table or gauge.

This is the final Environment dashboard on Grafana

Step 1 : Installation of All Applications on Raspberry Pi

Install RAK2245 and set to Chirpstack

Attach RAK2245 Hat to Raspberry Pi 3b+

Follow installation instruction here, after installation finish, open configuration apps for RAK2245 gateway.

sudo gateway-config

Setup RAK Gateway Channel Plan

• choose 2 (Server is Chirpstack)
• choose 1 (Chirpstack channel configuration) and
• choose 1 (AS923 for Asia)

Install Chirpstack Network Server Application Server, Bridge and MQTT

• Follow installation instruction here. As we are installing Chirpstack on the same Raspberry Pi, default configuration is sufficient. For security purpose, it is advised to change all default password.
• open http://localhost:8080 with default username admin and password admin (you have to change it later)
• We can see our gateway with GPS location.

Create Service Profile with the following configuration.

Create device profile. This is my configuration for device profile

The configuration will depend on your frequency, in our case frequency is AS923 :

• Tab General: as per above picture
• Join OTAA tab – tick device support OTAA
• Class B & Class-C tab – ignore for now
• Codec Tab – insert a javascript function to decode data which is originally in Hex from from Node, for example below, we parse the character to JSON object. copy paste in uplink field. I adapted the function from Cytron example here, Sufian Kaki Aslam project here and Seeedstudio wiki here. All of the examples above was for TTS, minor change required to fulfill Chirpstack requirement.

Function Decode(fport, obj) {     
        var warnings = [];
        var errors = [];
        var msg ="" var data = {};
        for (var i = 0; i < obj.length; i++) {         
                msg += (String.fromCharCode(obj[i]));     
            }    
        data = JSON.parse(msg);        
        return data;      
        }

Create new application. On Application configuration, select device profile created earlier.

Create application, on tab keys (OTAA), generate key and copy paste it somewhere on notepad.

This key will be used later in our Node circuitpython script.

Install Node-RED

Refer official documentation, this bash code is the latest one as at today

bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered) sudo systemctl enable nodered.service

Install InfluxDB

Refer to official documentation for latest instruction, this code is the latest as at today

get -qO- https://repos.influxdata.com/influxdb.key | sudo apt-key add -source /etc/os-release echo "deb https://repos.influxdata.com/debian $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/influxdb.list

Introducing the infamous terminal command : sudo apt update 😊

sudo apt update 
sudo apt upgrade 
sudo apt install -y influxdb

Create database named "environment ", set user and password

create database environment 
use environment

create user grafana with password '<passwordhere>' with all privileges 
grant all privileges on home to grafana

Install Grafana

wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add - echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee /etc/apt/sources.list.d/grafana.list

This will finish all the installation required, for detail walkthrough, please refer here or here. This youtube video shows installation of Node-RED, InfluxDB & Grafana.

Step 2: Save CircuitPython code on node (Seeeduino Xiao RP2040 ) and Grove LoRa-E5

Now we already finish the configuration, let us start our LoRaWAN Node.

We are using Mu Editor as IDE.

Install circuitpython firmware on the microcontroller, by holding reset button while inserting USB port. (tutorial here)

Copy following circuitpython code into a file named code.py, and save it to Circuitpy drive.

Download adafruit circuitpython library bundle and copy all required library (adafruit_scd30, adafruit_ssd1306 and seed_hm3301*) into folder lib in Circuitpy drive.

* seed_hm3301 circuitpython library is not yet available in Adafruit Circuitpython bundle, download it here.

Node connection is as per follows (refer breadboard schematic for details):

• Microcontroller : Seeeduino Xiao RP2040
• Grove Shield for Xiao
• Grove I2C Hub 6 Port connected to I2C port on Grove Shield
• Grove OLED Display (SSD1315) connected to I2C Hub 6 Port
• Grove CO2, Humidity & Temperature (SCD30) connected to I2C Hub 6 port
• Grove PM2.5 (HM3301) connected to I2C Hub
• and finally Grove LoRa-E5 connected to UART port on Grove Shield

Paste the OTAA key generated in step 1 earlier with string have AT+KEY=APPKEY and replace "PUT OTAA APPKEY HERE" with the OTAA key.

# Write your code here :-)
import board
import busio
import digitalio
import time
import supervisor
import json
import binascii
import math
import adafruit_scd30
import adafruit_ssd1306
from seeed_hm3301 import HM3301_I2C

#Set UART Pin
#LCD_SDA = board.GP26
#LCD_SCL = board.GP27
#i2c = busio.I2C(scl=LCD_SCL, sda=LCD_SDA)
i2c = board.I2C() # untuk seeeduino, tidak perlu definisi I2C
oled = adafruit_ssd1306.SSD1306_I2C(128, 64, i2c)
uart=board.UART()
# uart = busio.UART(board.GP0, board.GP1, baudrate=9600)
get_input = True
message_started = False
message_print = []
allstring = ""
printshow = False

def at_send_check_response(p_ack, timeout, p_cmd):
    b = bytes(p_cmd, 'utf-8')
    print(p_ack)
    # print(b)
    uart.write(b)
    # delay(200)
    DELAY_DURATION = timeout
    LAST_TIME = 0
    now = time.monotonic()
    condition = True
    allstring=""
    while condition:
        byte_read = uart.readline()# read one line
        if byte_read != None:
            allstring += byte_read.decode()
            printshow = True
        else:
            if printshow == True:
                if allstring != "":
                    print(allstring)    
                allstring=""
                printshow ==False
        condition = time.monotonic() - now < timeout
    return 0

at_send_check_response("AT",2,"AT")
at_send_check_response("AT",3,"AT+ID")
at_send_check_response("AT",3,"AT+ADR=?")
at_send_check_response("Set AT+MODE=LWOTAA",2,"AT+MODE=LWOTAA")
at_send_check_response("Set AT+DR=AS923",2,"AT+DR=AS923")
at_send_check_response("Set AT+APPKEY",2,"AT+KEY=APPKEY \"PUT OTAA APPKEY HERE\"")
at_send_check_response("Set AT+CLASS=A",2,"AT+CLASS=A")
at_send_check_response("Set AT+PORT=8",2,"AT+PORT=8")
at_send_check_response("Set AT+JOIN",10,"AT+JOIN")

while True:
    # read sensor value
    scd = adafruit_scd30.SCD30(i2c)
    hm3301 = HM3301_I2C(i2c, address=0x40)
    
    if scd.temperature != None and hm3301.PM_2_5_conctrt_std != None: # Format if there is sensor reading only
            #Format to 1 decimal point or integer.
            PM25 = int(hm3301.PM_2_5_conctrt_std)
            temperature = float("{:.1f}".format(scd.temperature))
            relative_humidity = float("{:.1f}".format(scd.relative_humidity))
            co2_ppm_level = float("{:.1f}".format(scd.CO2))
            # Show value on OLED
            oled.fill(0)
            oled.rect(10, 10, oled.width-10, oled.height-10, True) 
            oled.text("CO2      :"+str(co2_ppm_level), 20, 20,1)
            oled.text("PM2.5    :"+str(PM25), 20, 30,1)
            oled.text("Temp     :"+str(temperature), 20, 40, 1)
            oled.text("Humidity :"+str(relative_humidity), 20, 50, 1)
            oled.show()
            brokers_out={'C':co2_ppm_level,'T':temperature,'H':relative_humidity,'A':PM25}
    print(brokers_out)
    data_out=json.dumps(brokers_out)
    send_string= "AT+CMSGHEX="+bytes(binascii.hexlify(data_out.encode('utf-8'))).decode()+""
    at_send_check_response("Sending : "+send_string, 15, send_string)

    time.sleep(15)

If all thing installed correctly, serial console should show network joined successfully, and Node start to upload the data to gateway. The data is acknowledged received by the gateway.

Step 3: Confirm data uplink on Chirpstack

Open Chirpstack dashboard at localhost:8080 (or the IP address of raspberry pi 3 if opened from different computer), select created device earlier.

The data uplink is there in Device Data tab.

We can see the payload sensor data i.e A, C, H & T is put under objectJSON. Please note that original data in "data" field is in Hex form, it was converted using javascript decoder function in step 1 above)

Step 4: MQTT - find the topics and copy

on the Raspberry Pi where Chirpstack server is installed, open terminal and type this :

mosquitto_sub -v -t '#'

copy all MQTT topic name from application until up (yellow below), we also can see the sensor data is mixed with all other information

so in my case the topic is :

application/2/devide/abebecaabebecaaa/event/up

Step 5 : Node-Red to InfluxDB

The data need to be cleaned up as original JSON contains all information and we only need the objectJSON item containing sensor reading data. Following is the summary of the Node Palette used to clean up the data.

• MQTT In – subscribe MQTT topic as per step 4 (set output a parsed JSON object)
• JSON – to convert payload objectJSON from to javascript JSON object (Action : always convert to javascript object, property: msg.payload.objectJSON)
• Change – set msg.payload to msg.payload.objectJSON, to filter objectJSON only.
• InfluxDB Out – save data in Influx DB (measurement :environment)
• debug – to display the result of the node above at each stage.

Step 6 : Grafana Dashboard

open http://localhost:3000 (or the IP address of RPI if we open from other computer)

• Set datasource to influxDB and put all the credentials set earlier in step 1
• Start creating your dashboard. I put example here to create bar graph:

This is example of other type of chart & gauge

or in white theme

Conclusion

In summary we demonstrated:

• How to install RAK2245 gateway, Chirpstack Bridge, Network Server, Application Server, MQTT, Node-RED, InfluxDB and Grafana - all in a Raspberry Pi local network
• How to upload Sensor data data using CircuitPython via Grove LoRA-E5 from Node to Gateway
• How to configure, and clean up the data before storage at influxDB
• How to Display the Environment Dashboard in Grafana.

The example above is a complete uplink function. What we do not cover is the LoRaWAN downlink function. Additionally, If you are going to setup local LoRaWAN for production purpose, please look into the possible security issues i.e setting the credential for all database and apps, consider using SSL, firewall and other security matters those are not covered in this project.

That's all, please give feedback for any comment or question, thanks

I also created a Youtube channel on my discoveries in programming (In Malay Language) here

Following is video demo of this project using different microcontroller (Maker Pi RP2040) starting 3:36 :

Side Note : For those who interested in PCB design. I discovered this campaign online, which is relevant to LoRa-E5:

To accelerate the development of IoT, Seeed is sponsoring Lora-E5 projects with the Seeed Fusion PCB Assembly Service for all hardware enthusiasts, designers, makers, and engineers around the world! Each person is limited to two PCBA boards 100% completely FREE for one design, including PCB fabrication, the cost of parts, assembly and shipping. The design must include LoRa-E5. Find out more: https://www.seeedstudio.com/blog/2021/10/21/invigorate-your-inspiration-for-iot-with-lora-e5-and-free-seeed-fusion-pcba-prototypes/