The Things Network V2 Azure IoT Hubs & IoT Central Gateway

Background

After a customer abandoned a proof of concept(PoC) project I had a RAK WirelessWisGate Developer D0+ gateway, RAK WisNode 7200 Track Lite, Dragino LHT65 LoRaWAN Temperature & Humidity Sensor, and a selection of Seeeduino LoRaWAN and Seeeduino LoRaWAN/W GPS devices in a box under my desk.

Over the last couple of years I have built a series of Windows 10 IoT Core field gateway applications for connecting LoRa devices to Azure IoT Hubs, Azure IoT Central and Adafruit.IO. These had worked well but now I needed a Microsoft Azure cloud based solution to connect my LoRaWAN devices attached to the The Things Network(TTN) to applications running in Microsoft Azure.

Another side project I had been working on libraries for.NET nanoFramework and GHI Electronics TinyCLR devices to enable LoRaWAN connectivity with a RAK811 LPWAN module (in Wisduino form factor for PoC).

In New Zealand there are two nationwide networks (Spark IoT, KotahiNet) but I had been wanting to explore the functionality of TTN and The Things Industries which looked to have sufficient diagnostic functionality for my purposes.

I have assumed that if you are reading this project story you are familiar with developing applications for Microsoft Azure and especially the IoT services. The configuration of TTN Applications and Devices has been covered in detail in several other Hackster.IO projects so I won't repeat it here.

This project is a summary of a series of posts on my blog where I cover the construction of the solution in significantly more detail.

In the beginning

I initially connected the RAK WisGate Developer Gateway and configured the RAK7200 Track Lite device. In the TTN Application Device data tab I could see uplink messages being received from the device and most of the payload getting decoded which was a good start.

Getting Connected

I configured my Arduino IDE so that I could access the Seeeduino, LoRaWAN examples, then compile and download them to my devices. I tested modified versions of the Activation By Personalisation(ABP) and Over the Air Activation(OTAA) examples with my local gateway to confirm my device configuration was good. The device code worked second time, after I remembered that I needed to turn the power on for my Seeeduino Grove I2Ctemperature and humidity sensor connector.

Deserialising the TTN messages

I configured a TTN HTTP Integration for one of my TTN Applications so it POSTed uplink messages to an Azure Function with an HTTP trigger endpoint.

I used JSON2Csharp and a sample Uplink payload I downloaded from the TTN website to generate an initial version of some C# classes to de-serialise the uplink messages.

I had some issues with the generated code due to JSON2CSharp not being able to determine whether a numeric field was an integer or unsigned long.

The TTN documentation indicated that the payload_fields property was populated when an uplink message was successfully decoded. The TTN has a built in decoder for Cayenne Low Power Payload(LPP) messages which is partially supported by the RAK7200 Wisnode Track Light (a custom decoder/encoder with enhanced functionality is also available).

I used a 3rd party library (CayenneLPP library from Electronic Cats) on my Seeeduino LoRaWAN devices to encode the payload which contained temperature and humidity information.

Outstupiding Myself

Unpacking the payload_fields property caused me some pain. I tried many different approaches but they all failed. After much experimentation I found that using a C# object was the simplest approach (even though post processing the field was more complex).

public class PayloadV4
{
   public string app_id { get; set; }
   public string dev_id { get; set; }
   public string hardware_serial { get; set; }
   public int port { get; set; }
   public int counter { get; set; }
   public bool is_retry { get; set; }
   public string payload_raw { get; set; }
   //public JsonObject payload_fields { get; set; }
   //public JObject payload_fields { get; set; }
   //public JToken payload_fields { get; set; }
   //public JContainer payload_fields { get; set; }
   //public dynamic payload_fields { get; set; }
   public Object payload_fields { get; set; }
   public MetadataV4 metadata { get; set; }
   public string downlink_url { get; set; }
}

I also had to add some code to my PoC application to unpack the RAK Wisnode 7200 Tracker accelerometer, gyroscope and location values which had nested fields.

I then used the Microsoft.Azure.Devices.Client library to connect to an Azure IoT Hub or Azure IoT Central (generating the connection string with DPS-KeyGen) and uploaded telemetry messages which I could see in Azure IoT explorer.

Processing messages and failures with Azure Storage Queues

For my HTTP Integration I need to reliably forward uplink messages to an Azure IoT Hub or Azure IoT Central so I used an Azure Storage queue to provide an elastic buffer between my Azure Function HTTPTrigger endpoint and the message processor.

My solution needed to be robust and not lose any messages even when portions of the system are unavailable because of failures or sudden spikes in inbound traffic.

The code for receiving The Things Network(TTN) HTTP integration JSON messages used an Azure Function HTTPTrigger. (secured with an APIKey) and then put them into an Azure Storage Queue for processing.

This code was intentionally kept as small and as simple as possible so there was less to go wrong. After some experimentation it took less than two dozen lines of C# to create a secure endpoint to receive uplink messages and put them in an Azure Storage queue.

By storing the raw uplink event JSON from TTN the application can recover if it they can’t be de-serialised, (message format has changed or generated class issues) When queue processor can't process a uplink event message (throws an exception) it will end up in the poison message queue after being retried a few times (just incase the failure is transient).

The uplink message queue processor uses an Azure Function Queue trigger to pull messages from the queue, and provision the device if required, retrieve the Azure IoT Hub/Azure IoT central connection string or use the cached DeviceClient.

"Automagic" Provisioning

For development and testing being able to provision an individual device is really useful, though for Azure IoT Central it is not easy (especially with the deprecation of DPS-KeyGen). With an Azure IoT Hub device connection strings are available in the portal which is convenient but not terribly scalable.

Azure IoT Hub Is integrated with, and Azure IoT Central forces the use of the Device Provisioning Service(DPS).The DPS is designed to support the management of 1000’s of devices which required some custom applications for stress and soak testing.

My HTTP Integration for The Things Network(TTN) is intended to support many devices and integrate with Azure IoT Central. The DPS supports device attestation with a Trusted Platform Module(TPM) but this approach was not possible for my application. My TTN Application Integration uses Group enrollments with Symmetric Key Attestation

Azure IoT Hub integration

Though you can provision individual devices in your Azure IoT Hub the Azure Device Provisioning Service(DPS) is the preferred approach.

The scopeID and the primary/secondary enrollment key are configured in the Azure Key Vault where they can be securely access by the Azure QueueTrigger function.

For more complex deployments The Things Network(TTN) devices can be provisioned using different GroupEnrollment keys based on the applicationID (or the applicationID + port number). An example of this would be a tracking device on a truck reporting location data with one port number and cargo temperature and humidity on another so telemetry events can be routed to the right application.

Then as the first uplink message for a device is processed it is “automagically” created in the DPS and Azure IoT Hub.

Azure IoT Central Integration

The Things Network(TTN) HTTP integration uplink messages had to be provisioned then post processed so they could be displayed by Azure IoT Central.

The first step is to copy the IDScope, and one of the primary/ secondary keys from the Administration\Device connection and store them in the Azure Key Vault.

For more complex deployments The Things Network(TTN) devices can be provisioned using different GroupEnrollment keys based on the applicationid (or the applicationID + port numer) in the first Uplink message which initiates provisioning.

Shortly after the first uplink message from a TTN device is processed, it will listed in the “Unassociated devices” blade.

The device can then be associated with an Azure IoT Central Device Template.

The device template provides for the mapping of uplink message payload_fields to device properties. In this example the payload field has been generated by the TTN Application Integration Cayenne Low Power Protocol(LPP) decoder. Many LoRaWAN devices use LPP to minimise the size of the network payload.

Once the device has been associated with a template a user friendly device name etc. can be configured.

Azure IoT Central has mapping functionality which can be used to display the location of a device.

The format of the location payload generated by the TTN LPP decoder is different to the one required by Azure IoT Central. I have added temporary code (“a cost effective modification to expedite deployment” aka. a hack) to format the TelemetryEvent payload so it can be displayed.

if (token.First is JValue)
{
   // Temporary dirty hack for Azure IoT Central compatibility
   if (token.Parent is JObject possibleGpsProperty)
   {
      if (possibleGpsProperty.Path.StartsWith("GPS", StringComparison.OrdinalIgnoreCase))
      {
         if (string.Compare(property.Name, "Latitude", true) == 0)
         {
            jobject.Add("lat", property.Value);
         }
         if (string.Compare(property.Name, "Longitude", true) == 0)
         {
            jobject.Add("lon", property.Value);
         }
         if (string.Compare(property.Name, "Altitude", true) == 0)
         {
            jobject.Add("alt", property.Value);
         }
      }
   }
   jobject.Add(property.Name, property.Value);
}

After configuring a device template, associating some devices with the template, and modifying each device’s properties I could create a dashboard to view the temperature and humidity information returned by my Seeeduino LoRaWAN devices.

Keeping Secrets Secret

The Application Integration configuration contains sensitive information like Device Provision Service(DPS) Group Enrollment Symmetric Keys and Azure IoT Hub connection strings.

The Azure Key Vault is intended for securing sensitive information like connection strings so I added one to my resource group.

I wrote a wrapper which resolves configuration settings based on the The Things Network(TTN) application identifier and port information in the uplink message payload. The resolve methods start by looking for configuration for the applicationId and port (separated by a – ), then the applicationId and then finally falling back to a default value.

This functionality is used for AzureIoTHub connection strings, DPS IDScopes, DPS Enrollment Group Symmetric Keys, and is also used to format the cache keys.

The values of Azure function configuration settings (like Azure Storage Connection strings) are replaced by a reference to the secret in the Azure Key Vault.

In the Azure Key Vault “Access Policies” I configured an “Application Access Policy” so my Azure TTNAzureIoTHubMessageV2Processor function identity can retrieve secrets.

Go Biggly or go home

In my initial implementations I used a ConcurrentDictionary to store Azure IoT Hub connections to reduce the number of calls to the Device Provisioning Service(DPS). After some testing I replaced it with a.Net ObjectCache which is in the System.Runtime.Caching namespace.

I was using a cache to store Azure IoT Hub connections to reduce the number of calls to the Device Provisioning Service(DPS) but the number of connections was still too high.

So after some research I decided to enable Advanced Message Queuing Protocol(AMQP) connection pooling.

return DeviceClient.Create(result.AssignedHub,
   authentication,
   new ITransportSettings[]
   {
      new AmqpTransportSettings(TransportType.Amqp_Tcp_Only)
      {
         PrefetchCount = 0,
         AmqpConnectionPoolSettings = new AmqpConnectionPoolSettings()
         {
            Pooling = true,
         }
      }
   }
);

After this the number of connections was significantly reduced

The "Markitecture" diagram

Oops I painted myself into a corner

After scale and soak testing of the handling of uplink messages I realised that some of the design and implementation choices I had made meant it wasn't going to be easy to process downlink messages.

If your application only needs to receive messages from LoRaWAN devices this solution would be ideal.

To support downlink messages I'm most probably going to have to transition to the MQTT Data API and remove some of the advanced configuration options.

This is going to take a while so if you're interested keep an eye on my blog where I'll be posting about my progress.