I Was There... Recycling the Waste!

Introduction

"I was there" creates secure temporary QR tokens (JSON Web Token) signed by an Azure Sphere device for later verification.

The problem

On many occasions we have to show that we have been in a certain place and at a certain time.

We do not want users to have to be previously identified, so solutions such as identification cards (magnetic, with chip, contactless cards, or smartphones with NFC tags) are not worth us. Nor are solutions worth using biometric identification.

Typicaluse cases

• Reward citizens who use recycling bins.
• A delivery woman or delivery man to show she/he went to a collection site.
• Giving loyalty points to those who came to certain stores.
• Record the check-ins and check-outs of your internal and external employees
• Record when cleaners have entered and left to clean a certain area.

The solution

IWT (I Was There) generates QR codes with a dated token independent of the person, signed and unique to avoid its modification as a guarantee that the token has not been modified later.

Prototype implementation of IWT for a program of awards to citizens who recycle:

How does it work

• The user approaches the IWT device.
• Then, the user presses a button, or a closing or opening event of a door is detected or, by means of a proximity sensor, the device is informed to generate a new QR code.
• The device generates offline a new QR code that contains a signed JWT token, JSON Web Token, including an unique identifier and the date and time of the token issuance.
• Then the device displays the QR code on a low power consumption E-Paper display.

• Asynchronously, the device sends an event to the Azure cloud in order to record and monitor new QR requests
• The user can capture the image with any camera for later validation or can use a specific application that validates the token at the moment.
• After a configurable time the token disappears from the screen and the device is ready to generate the next token.
• For loyalty programs the IWT device can issue programmed reward messages from the IOT Central platform.

Alternatively, the device may generate an inaudible sound signal that can be recorded with any sound recording device (implementation pending).

The challenges

During the design of the solution I have encountered several problems that have conditioned the final solution:

• It is not possible to use the Azure Sphere Pluton Security Subsystem to encrypt data. In addition, Azure Sphere does not currently provide an API for encryption. Also there is also no API function available to retrieve the device identifier.
• The official JSON Web Token (JWT) libraries use OpenSSL or GnuTLS, libraries that are too heavy for this embedded system.
• There was no library for the waveshare e-paper display for use with Azure Sphere devices.

Overcoming obstacles

My first decision was to try to incorporate a crypto-processor module to perform the encryption, signatures and store the cryptographic keys. Unfortunately, I still did not receive the cryptographic module on time and had to opt for a software solution.

• For cryptography I decided to use the wolfCrypt library. Fortunately wolfSSL released a Visual Studio solution that contains a cryptographic algorithm test for the wolfCrypt library.
• JWT tokens are being formed directly using string functions without using any ad hoc library.
• I have made an adaptation of the STM32 EPD driver for the waveshare 1.54 inch e-Paper Module display.

Design

Hardware

The hardware consists of an Avnet Azure Sphere MT320 Starter Kit development kit and a Waveshare 1.54inch e-Paper Module electronic ink display.

The advantages of Microsoft Azure Sphere solution

Avnet Azure Sphere MT320 Starter Kit

• The Azure Sphere MT3620 Starter Kit supports rapid prototyping using Avnet’s certified Azure Sphere module, based on the Microsoft MT3620AN device.
• The MT3620 is the first Azure Sphere certified "micro-controller", a SoC IoT device that features “end-to-end security”.
• User applications can target it's 500 MHz ARM Cortex-A7 core as well as two general purpose 200 MHz ARM Cortex-M4F I/O subsystem cores designed to support real-time requirements. The on-chip peripherals (GPIO, UART, I2C, SPI, I2S, PWM and ADC) can be mapped to any of these three user-accessible cores.

Waveshare 1.54inch e-Paper Module electronic ink display

• The electronic ink screen has been chosen for its good qualities to present QR codes and its reduced power consumption in a project of these characteristics.
• It is an E-Ink display module, 1.54 inch, 200 x 200 resolution, with embedded controller, communicating via SPI interface, and supports partial refresh.
• Characteristics: low power consumption, wide viewing angle, clear display without electricity.

Software Design

State Diagram

• Idle show the "Press A"
• Rewards show reward associated to the QR if any
• JWT_QR issue and show new QR Token
• Clock show time clock
• Settings show settings
• Synch Settings sync settings with Azure cloud
• Azure Cloud Async Tasks send/receive/synch cloud data

User Interface

Idle Screen

During the Idle state the screen shows the indication of pressing the A button to obtain a new QR

Rewards Screen

Once button A has been pressed if a new prize status was received for the next QR, the prize associated to the QR is displayed. Option for customer loyalty programs.

QR Screen

Then the new QR code containing a signed JWT token. A time set from the Azure cloud will be in this state.

Clock Screen & Settings

When the B button is pressed, a clock with the current time of the device is displayed and if pressed again the application settings are displayed.

Creating the JWT Token QR

Anatomy of the JWT Token QR

The QR code contains this string:

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJqdGkiOiJlMmRhNzc3My1hN2NkLTQ0YTAtODVkNy00M2ZlMDY1NGNiMjItN2RjNTQ4MTEtNWRjYjBjNzAiLCJpYXQiOjE1NzM1ODgwODB9.Dm8qKzv5n0EHMCcIECn08Ev_eppzPNc-hGtuICCNGK0

It is a JSON Web Token in its compact form. A JWT typically looks like the following: xxxxx.yyyyy.zzzzz

JSON Web Token (JWT) is a compact, URL-safe means of representing claims to be transferred between two parties. JSON Web Token (JWT) is an open standard (RFC 7519). The claims in a JWT are encoded as a JSON object that is used as the payload of a JSON Web Signature (JWS) structure or as the plaintext of a JSON Web Encryption (JWE) structure. This information can be verified and trusted because it is digitally signed. JWTs can be signed using a secret (with the HMAC algorithm) or a public/private key pair using RSA or ECDSA. https://tools.ietf.org/html/rfc7519

It consists of three parts separated by dots (.), which are:

• Header: contains the metadata for the token and it minimally contains the type of signature and the encryption algorithm.
• Payload/Claim: contains the information we want to transmit,
• Signature: calculated using the header and the payload, you can also verify that the content hasn't been tampered with.

Contents of the three parts are Base64Url encoded. Decoding the three parts of the above token:

The Header: declares that the encoded object is a JSON Web Token, and that it is signed using the HS256 algorithm.

{
"alg": "HS256",
"typ": "JWT"
}

The Payload /Claim: we are using two registered claim names. In the context of JWT, a claim can be defined as a statement about an entity, as well as additional metadata about the token itself. The claim and that the server can use to properly handle JSON Web Token authentication. There are multiple claims we can provide; these include registered claim names, public claim names and private claim names.

• iat: “Issued at” time, in Unix time, at which the token was issued
• jti: JWT ID claim provides a unique identifier for the JWT

{
"jti": "e2da7773-a7cd-44a0-85d7-43fe0654cb22-7dc54811-5dcb0c70",
"iat": 1573588080
}

https://tools.ietf.org/html/draft-ietf-oauth-json-web-token-32#section-10.1

Signature: digital signature or Message Authentication Codes (MACs).

https://tools.ietf.org/html/rfc7515

HMACSHA256(
  base64UrlEncode(header) + "." +
  base64UrlEncode(payload),
  [your-256-bit-secret]
)

Securing the JWT Token

Azure sphere does not provide APIs to get hardware support/acceleration for cryptographic function calls from the A7 and/or M4 subsystems. So we are using wolfSSL embedded SSL library (formerly CyaSSL)

wolfSSL

The wolfSSL embedded SSL library (formerly CyaSSL) is a lightweight SSL/TLS library written in ANSI C and targeted for embedded, RTOS, and resource-constrained environments - primarily because of its small size, speed, and feature set. It is commonly used in standard operating environments as well because of its royalty-free pricing and excellent cross platform support. wolfSSL supports industry standards up to the current TLS 1.3 and DTLS 1.2 levels, is up to 20 times smaller than OpenSSL, wolfSSL is powered by the wolfCrypt library.

https://github.com/wolfSSL/wolfssl/tree/master/IDE/VS-AZURE-SPHERE

KEYED HASH HMAC

wolfCrypt currently provides HMAC for message digest needs. The structure Hmac is found in the header "wolfssl/wolfcrypt/hmac.h".

HMAC initialization is done with wc_HmacSetKey(). For the project we are using HMAC SHA-256

Header

{"alg":"HS256","typ":"JWT"}
BASE64     eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9
BASE64URL  eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9

Payload

{"sub":"1234567890","name":"John Doe","iat":1516239022}
BASE64  eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ==
BASE64URL  eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ

Verify Signature

HMACSHA256(
base64UrlEncode(header) + "." +
base64UrlEncode(payload),
0123456789ABCDEF0123456789ABCDEF )

Results in:

HMACSHA256(eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ,
0123456789ABCDEF0123456789ABCDEF)

And finally:

HEX         A650E04A6412F8B67A06942E513556D35C9BE22FC8EA12D92970AACFB76B6EED
HEX2B64     plDgSmQS+LZ6BpQuUTVW01yb4i/I6hLZKXCqz7drbu0=
B642B64URL  plDgSmQS-LZ6BpQuUTVW01yb4i_I6hLZKXCqz7drbu0

Encoded Token

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9
.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ
.plDgSmQS-LZ6BpQuUTVW01yb4i_I6hLZKXCqz7drbu0

Hmac hmac;

byte        key[24];            /*fill key with keying material*/
byte        buffer[2048];   /*fill buffer with data to digest*/
byte        hmacDigest[SHA256_DIGEST_SIZE];
wc_HmacSetKey(&hmac, SHA256, key, sizeof(key));
wc_HmacUpdate(&hmac, buffer, sizeof(buffer));
wc_HmacFinal(&hmac, hmacDigest);

hmacDigest now contains the digest of the hashed data in buffer.

Generating the QR Code

We are using Richard Moore QR Code library, a simple library for generating QR codes in C, optimised for processing and memory constrained systems.

Features:

• Stack-based (no heap necessary; but you can use heap if you want)
• Low-memory footprint (relatively)
• Compile-time stripping of unnecessary logic
• MIT License;

https://github.com/ricmoo/QRCode

Going to Cloud

Implementation steps

• Create a custom IoT Central application
• Create a device template
• Add QR Request events to the Measurements Tab
• Add Toggle controls to the Settings Tab: RGB LED controls, WWAN and APP LED controls
• Add Application information to the Properties Tab
• Application Version
• Current SSID
• BSSID
• Current Wi-Fi Radio Frequency

Resources

https://www.element14.com/community/groups/azuresphere/blog/2019/05/09/avnets-azure-sphere-starter-kit-out-of-box-demo-part-3-of-3

Monitoring Qr requests

Whenever a new weekly QR code is issued asynchronously an event to the Azure cloud, either to Azure IoT Central or to Azure IoT Hub as the application was built.

The points seen in the next image correspond to events of QR code generation after pressing the A button or when the reed switch detects a step from open to closed.

We can see a detailed list view of the events with the generated QR ID.

Programming new messages to show

New award or information messages can be programmed from IoT Central or completely disabled.

Getting device info

Properties

Key               Value
userLedRed        true | false    enables/disables the red LED of the user RGB LED
userLedGreen      true | false    enables/disables the green LED of the user RGB LED
userLedBlue       true | false    enables/disables the blue LED of the user RGB LED
appLed            true | false    enables/disables the application LED
wifiLed           true | false    enables/disables the WI-FI LED

Integrating E-Paper Display

Waveshare 1.54inch e-Paper V2 overview

1.54inch e-Paper V2 is an Active Matrix Electrophoretic Display (AMEPD), with interface and a reference system design. The 1.54” active area contains 200×200 pixels, and has 1-bit B/W full display capabilities. An integrated circuit contains gate buffer, source buffer, interface, timing control logic, oscillator, DC-DC. SRAM.LUT, VCOM and border are supplied with each panel.

https://www.waveshare.com/wiki/1.54inch_e-Paper_Module

Pin-out diagram

This table shows how the pin-out on eINK click corresponds to the pin-out on the mikroBUS™ socket (the latter shown in the two middle columns).

Click Socket #1 Pin-out

SOURCE CODE PIN DEFINES

#define AVT_MODULE_GPIO31_SCLK1_TXD1 (GPIO_Id)31
#define AVT_MODULE_GPIO32_MOSI1_RTS1_CLK1 (GPIO_Id)32
#define AVT_MODULE_GPIO33_MISO1_RXD1_DATA1 (GPIO_Id)33
#define AVT_MODULE_GPIO34_CSA1_CTS1 (GPIO_Id)34
#define EPAPER_DC (GPIO_Id)0
#define EPAPER_BUSY (GPIO_Id)2
#define EPAPER_RESET (GPIO_Id)16
#define EPAPER_SCLK AVT_MODULE_GPIO31_SCLK1_TXD1
#define EPAPER_MOSI AVT_MODULE_GPIO32_MOSI1_RTS1_CLK1
#define EPAPER_CS AVT_MODULE_GPIO34_CSA1_CTS1

Communication protocol

SPI interface

Different from the traditional SPI protocol, the data line from the slave to the master is hidden since the device only has a display requirement.

CS is slave chip select, when CS is low, the chip is enabled. (CS#) is the chip select input connecting to the MCU. The chip is enabled for MCU communication: only when CS# is pulled LOW.

DC is data/command control pin, when DC = 0, write command, when DC = 1, write data. (D/C#) is Data/Command control pin connecting to the MCU. When the pin is pulled HIGH, the data will be interpreted as data. When the pin is pulled LOW, the data will be interpreted as command.

SCLK is the SPI communication clock.

SDIN is the data line from the master to the slave in SPI communication.

(RES#) is reset signal input. The Reset is active low. Note 1.5-4: This pin (BUSY) is Busy state output pin. When Busy is High the operation of chip should not be interrupted and any commands should not be issued to the module. The driver IC will put Busy pin High when the driver IC is working such as: - Outputting display waveform; or - Communicating with digital temperature.

(BS1) is for 3-line SPI or 4-line SPI selection. When it is “Low”, 4-line SPI is selected. When it is “High”, 3-line SPI (9 bits SPI) is selected.

SPI communication has data transfer timing, which is combined by CPHA and CPOL.

• 1. CPOL determines the level of the serial synchronous clock at idle state. When CPOL = 0, the level is Low. However, CPOL has little effect to the transmission.
• 2. CPHA determines whether data is collected at the first clock edge or at the second clock edge of serial synchronous clock; when CPHL = 0, data is collected at the first clock edge.

There are 4 SPI communication modes. SPI0 is commonly used, in which CPHL = 0, CPOL = 0.

Data transmission starts at the first falling edge of SCLK, and 8 bits of data are transferred in one clock cycle. In here, SPI0 is in used, and data is transferred by bits, MSB first.

Sensing the user

In addition to launching the new QR code when the button is pressed, we can activate it when it detects when the user using the recycling container using a proximity sensor.

#define AVT_MODULE_GPIO42_ADC1	MT3620_GPIO42
#define AVT_SK_CM1_AN AVT_MODULE_GPIO42_ADC1
#define PROXIMITY_GPIO AVT_SK_CM1_AN

We are using a Reed Switch and a magnet to detect if the user opens the container door.

The reed switch we'll be using is normally-open. We have two alternatives to detect when the door is fully closed o when the door is fully opened. For this project we will be using the first alternative:

Detect door fully closed: The magnet is close the Reed Switch when the door is closed and will be in that state until the door is opened. We connect the Reed Switch to ground and we will pull up the GPIO pin 42 via 4.7K. In closed stated will read Low. The event for a new QR will be sent when passing from High to Low, that is when the door is fully closed again, rewarding the user for closing the door :D after use.

Development tools

Validating QR

From Android device with QR Scanner:

Validating JWT

Verifying signature with JWT debugger: https://jwt.io/#debugger

Fritzing parts

https://forum.fritzing.org/t/part-request-avent-azure-sphere/7390/6