Device Mesh for Health Check

Currently, the COVID-19 pandemic requires us to take actions and daily care to protect ourselves and thus achieve a healthy life. In some countries the return to the offices in hybrid work schemes is already being seen.

DMHC HOST Probe is a device created with AWS IoT Edukit and some of its sensors called units. It allows us remotely check our heart rate, oxygen saturation level and temperature during defined time intervals.

It sends us an SMS alert to our cell phone or if we are connected with our Slack session to a specific channel, remembering when we must verify our HOSTs values.

It can be used, for example, in the office, so that each worker has a probe and we can verify the data of each one centrally.

Or in a particular patient who is connected with his doctor to verify the advance of these values ​​and thus achieve immediate corrective actions.

For this, the Amazon Web Services cloud is used in which the device sends this data to be processed, stored and consulted using Amazon QuickSight, a mobile app developed in Amazon Honeycode and Grafana.

The following sections describe the hardware, the base construction, the different AWS cloud services used, and the options for querying the sent data.

AWS IoT EduKit hardware ports

• A port : I2C interface
• B port : ADC, DAC interface
• C port: UART interface
• Ext port: I2C interface

These images show extension port Ext (pin 22 SCL and pin 21 SDA). It was created using the Proto-13.2 module white connectors and plastic box. The white solder connector is the port, and have these connections:

• Pin 1 - GND (blue wire)
• Pin 2 - 5v (red wire)
• Pin 3 - G21 SDA (magenta white)
• Pin 4 - G22 SCL (green wire)

These images show the ports location included Ext port inside plastic box and attached to AWS IoT Edukit using extension connector.

Sensors and Modules (M5Stack Units) and their connection diagram

• A Port: Heart Unit is H sensor (heart rate sensor)
• Ext Port: The Mini Hub
• Mini Hub port 1 : ENV. III Unit (pressure, humidity and temperature sensor)
• Mini Hub port 2 : NCIR Unit is T sensor (temperature sensor)
• Mini Hub port 3 : RFID Unit

This image show the connection diagram for all used units, includes a proto-13.2 board.

Base Construction

Once all the units and the extension module are connected, it is necessary to build a base to fix each unit and the AWS IoT Edukit.

Use Lego EV3 parts following the instructions located in the section Custom parts and enclosures. The pdf file shows the step by step while in the pdr file using the Ldraw program the 3D design can be followed.

DMHC HOST Probe Software

Firmware Install

The firmware used is UIFlow Core2 v1.8.4 to load it, connect the AWS IoT Edukit to one of the computer's USB ports, making sure to have the installed USB Driver depending on the platform.

Download the program to update the firmware, M5Burner and install it depending on the platform.

When executing M5Burner we must select the COM port to which our device is connected.

Create an access with the account in the Login option and once entered press the Bind With Account button in order to link a device with the UIFlow cloud.

If we select your email, it will show the menu option Device Management that opens the browser and shows us the information with which we can verify that the device is linked correctly.

Now we select option UIFLOW (CORE2) the correct version v1.8.4-core2 and press Download.

Press the Configuration button, in the UIFlow Config screen, enter the data of your Wi-Fi network SSID and Password and press button Save in order to connect the device to update the firmware.

Press the Burn button and check that the SSID and password data of your network to which will connect the device are correct and press button Start.

When this process is finished, the device reboots and shows the screen indicating that it is connected to the network and the UIFlow cloud and our API Key.

Creating Things at AWS IoT Core service

AWS IoT Core allow connect IoT devices to the AWS cloud without the need to provision or manage servers.

In AWS console go to AWS IoT Core and create a Thing at Manage, Things menu option.

• Press Create Things button
• Select Create single thing
• For thing name : DMHC_HOST_Probe
• For Device certificate: auto-generate a new certificate
• Policies: create a new Policy AllowAllAccess and copy and paste this code

{
        "Version": "2012-10-17",
        "Statement": [
        {
            "Effect": "Allow",
            "Action": "iot:*",
            "Resource": "*"
        }
    ]
}

• Press Create Thing button

A pop up window show all certificate and keys to download.

It is important to download the device certificate, xxxx.pem.crt, and the two additional keys the public xxxx-pubilc.pem.key and the private xxxx-private.pem.key, since it will be the only time we can do it, if they are lost you must create a new thing.

The device certificate and private key are required to be able to establish the communication of the device with the AWS IoT Core services.

In addition to the certificate and the two keys, a device needs the endpoint, a url in which each thing has a REST API available so that an MQTT client and AWS IoT Device SDKs can be accessed.

In the menu option Settings you can see and copy this address to use it in the software that we will build.

In summary we have a thing with:

• Name: DMHC_HOST_Probe
• AWS IoT Endpoint : a2i2bsijbyml6j-ats.iot.us-west-2.amazonaws.com
• MQTT port: 8883
• Keep Alive: 60
• Device Certificate : xxxx.pem.crt rename to certificate.pem
• Private key file: xxxx-private.pem.key rename it to private.pem

Topics to be used

• dmhc/host_data → Publish
• dmhc/reset ← Subscribe

Upload all resources and software to AWS IoT Edukit

Now go to flow.m5stack.com in browser, cancel the initial popup screen and go to account icon, at pop up login add your email created ago and your password to recover all device info linked in the previous steps of firmware update.

Plug at Usb port AWS IoT Edukit, start it and wait it show the Api Key screen. Press the reconnect icon at right of Disconnected word for reconnect device and update ApiKey and Version, verify that version was at least v1.8.4.

Now is time to upload all png images, all wav sound files and the certificate and private key to AWS IoT Edukit device. These files can be download from code section.

Each device that belong to de mesh must be an unique ID saved at EEProm key device_id.

Open setDeviceId.m5f from code section, adjust its value and run program (click arrow icon), you must see the value at screen Saved ID: xxxxx

Having the AWS configured with all the required resources, we upload the program that starts the HOST Probe. We open the dmhcV2.m5f file located in the code section and upload it to the device.

The program was created using the UIFlow online editor and can be viewed in Blockly or Python format.

The algorithm used is as follows:

/*
The current time is read.
All images are loaded.
The user interface is painted.
Read ambient temperature, humidity and pressure.
Activate Time to check your health event
    Play wav file
    Activate vibration
    RGB LEDs blink between red and green color.
Activate Heart rate and spo2 review Event
    Play wav file (H sensor)
    RGB Leds blink between red and black
    Starts to read ten values ​​from heart rate and spo2 sensor
    Updates status bar and progress bar
    Saves the values ​​in two lists, one for each concept.
Activate Temperature review event
    Play wav (T Sensor)
    RGB Leds blink between green and black
    Starts reading ten values ​​from temperature sensor
    Updates status bar and progress bar
    Saves values ​​in a list.
Activate Test completed event
    Play wav
Connect to AWS cloud using MQTT protocol.
Read values ​​of three lists created, prepares a Json and publishes it in 
topic dmhc/host_data.
*/

It is a Json example data send to topic dmhc/host_data

{
    “device_id” : “JCruz”,
    "heartrate" : 120,
    "spo2" : 97,
    "temperature" : 45
}

It is subscribed to the topic dmhc/reset so that when it receives information from the AWS Cloud that it is time to activate the reading it resets the system to start a new cycle checkup.

The list of all help functions are listed here:

# Paint user interface, get time and environment data
def DMHC_Init():

# Get time from device, format it and show at user interface
def Get_Time():

# Read from Env III unit temperature, pressure and humidity, 
# format values and show at user interface
def Read_ENV3():

# Begin check process activate vibration, play
# wav, RGB leds blink between red and green
def TimeToCheck():

# Read data from heart and temperature unit sensor 
# and put them at three value lists
def Read_Data():

# Read heart rate and spo2 from Heart Unit, put
# values in two lists and show at user interface
def Read_HeartRate_SpO2():

# Indicate that test was completed
def Test_Completed():

# Read temperature value from NCIR Unit, 
# put it in a list and show at user interface
def Read_Temperature():

# Wait from AWS Cloud that call dmhc/reset
# topic to reset device and begin new process
def fun_dmhc_reset_(topic_data):

Principal program module:

# DMHC HOST Probe program v2
# 
DMHC_Init()
TimeToCheck()
Read_Data()
Test_Completed()
lb_status.set_text('Connecting to AWS ...')
_thread.stack_size(0)

# Connect to thing endpoint at AWS IoT Core using MQTT protocol
aws = AWS(things_name='DMHC_HOST_Probe', host='a2i2bsijbyml6j-ats.iot.us-west-2.amazonaws.com', port=8883, keepalive=60, cert_file_path="/flash/res/certificate.pem", private_key_path="/flash/res/private.pem")

aws.subscribe(str('dmhc/reset'), fun_dmhc_reset_)
aws.start()
lb_status.set_text('AWS Connected')
_thread.stack_size(0)


# Create a Json with all list data read from sensors and send to AWS cloud
DataJson = {}
while not not len(temp_list):
    DataJson['device_id'] = device_id
    DataJson['heartrate'] = heart_list.pop(0)
    DataJson['spo2'] = spo2_list.pop(0)
    DataJson['temperature'] = temp_list.pop(0)

# Publish to topic dmhc/host_data
aws.publish(str('dmhc/host_data'),str((json.dumps(DataJson))))
lb_status.set_text('Sending data to AWS ...')
_thread.stack_size(0)
lb_status.set_text('Waiting for a new request')

The detailed and complete code can be downloaded in the Code section, file dmhcV2.py is a complete MIcroPython code.

Device (Thing) User Interface

Additional steps

To create the sound files in wav format, the AWS Polly service can be used, which allows converting from text to speech.

You need to create an AWS S3 Bucket dmhc-bucket and a wav/ folder to store the files that AWS Polly generates in mp3 format. Finally, the generated mp3 files are downloaded and with the online-converter.com service they were converted from mp3 to wav.

AWS Polly Text-to-Speach used options were :

• Language: English US
• Voice : Joanna, Female
• Neural

Back End Software (AWS Cloud Architecture)

AWS IoT Core Service

As seen in the architecture diagram, the main service of the backend is AWS IoT Core.

As described in the chapter Creating Things at AWS IoT Core service, it registers each of the devices (Things) to communicate with the cloud using the MQTT protocol.

For this, the dmhc/host_data topic is used, which allows publishing the data that is generated by each probe in a json format.

To process the messages received by the topic and send them to other services, rules must be defined, in our case we will create two rules in AWS IoT Core.

dmhc_rule_iot_analitycs: to send the received message to the AWS IoT Analitycs service

• Select in AWS IoT Core menu Act the option Rules
• Create rule name: dmhc_rule_iot_analytics
• Query statement: select * from 'dmhc/host_data'
• Press Add action, select Send a message to IoT Analytics
• Press configure action
• Select Quick create IoT Analytics resources name it dmhc_host
• Press quick create
• Create a new role : dmhc_host_role

At this time all AWS IoT Analytics resources are created:

• dmhc_host_channel
• dmhc_host_pipeline
• dmhc_host_datastore
• dmhc_host_dataset

dmhc_rule_timestream: to send the received message to the AWS TimeStream

• Select in AWS IoT Core menu Act the option Rules
• Create rule name: dmhc_rule_timestream
• Query statement: SELECT device_id, heartrate, spo2, temperature FROM 'dmhc/host_data'
• Press Add action, select Write a message into a Timestream table
• Press Configure action
• Create a new database: dmhc_timestreamdb, Standard
• Create a new table: dmhc_timestreamtable in database dmhc_timestreamdb
• Dimensions: Dimension Name 'DeviceID', value ${device_id}
• Timestamp: Value ${timestamp()}, Unit MILLISECONDS
• Select role dmhc_host_role

AWS IoT Analytics Service

AWS IoT Analytics automates each of the difficult steps that are required to analyze data from IoT devices. AWS IoT Analytics filters, transforms, and enriches IoT data before storing it in a time-series data store for analysis.

• dmhc_host_channel receive al messages and send them to
• dmhc_host_pipeline who validate and transform and send them to
• dmhc_host_datastore who store data and is available for
• dmhc_host_dataset to share with another services.

We need store timestamp in a new json field server_time for each incoming messages, and only allow out messages that spo2<>0 and temperature<200 avoid error data provide from sensors error read. Using a lambda function and pipeline activities we can do.

Lambda function: dmhc_TimeStamp

• Language: Node.js 14.x
• Source code at section code
• Set roles and permissions

// Handler called by IoT Analytics - pipeline dmhc_host_pipeline
exports.handler = function handler(event, context, callback) {
    //Add timestamp to incoming data and name it "server_time"
    event[0].server_time = Date.now();
    // Return the data
    callback(null, event);
};

Add to pipeline dmhc_host_pipeline two activities:

• Transform message with Lambda Function
  Call a lambda function dmhc_TimeStamp
• Filter messages conditionally
  Set spo2 is not 0 and temperature is less than 200 conditions

The dataset dmhc_host_dataset need save all data to dmhc_host_data_iotanalytics.csv file and store it in bucket dmhc-bucket folder files/ for share with AWS Honeycode app created.

We can do it creating a new Dataset with delivery rule Deliver result to S3

Change SQL Query to:

select heartrate, spo2, temperature, server_time , __dt as day 
from dmhc_host_datastore

Amazon AppFlow and Amazon Honeycode

Amazon AppFlow is a fully managed integration service that enables you to securely transfer data between AWS services like Amazon S3 and send it to Amazon Honeycode.

Amazon Honeycode

• First create an App with Amazon Honeycode,
• Create an account, login
• Create a workbook DMHC Host Probe Data with a data table dmhc_data_table
• Rename the table fields heartrate, spo2, temperature, date, server_time
• and could create with wizard assistance a detail form for table.

In detail form could activate some alert conditions for each field for example when SpO2 is less than 93 or heartrate>140 show value with red background.

Table is empty and need help from Amazon AppFlow to insert all data received from DMHC HOST Probe processed by AWS IoT Analytics and store at AWS S3 Bucket Service dmhc-bucket, file files/dmhc_host_data_iotanalytics.csv

Amazon AppFlow

Will be a bridge between S3 and Honeycode App using:

• A flow dmhc_flow
• Connect dmhc-bucket folder files with Amazon Honeycode with Connection dmhc_conn
• Honeycode workbook DMHC Host Probe Data
• Table dmhc_data_table
• Flow trigger every 5 minutes incremental transfer
• And mapped all fields from csv file with dmhc_data_table
• Apply some data truncate needed

If select the new flow dmhc_flow and press Run flow button all data will be inserted in Honeycode app dmhc_data_table and each 5 minutes will be updated.

Open the app DMHC Data Table in Honeycode app in the phone mobile and will see all data and some alerts in red background.

Amazon QuickSight

With Amazon QuickSight we can analyze and create data dashboards connected to AWS IoT Analytics datasets.

We could create datasets from a lot of data sources, databases and AWS services.

• Select datasets, New dataset,
• Search AWS IoT Analytics,
• Select dmhc_host_dataset, create source and it show a Spice dataset.

Opening it, can see all spice info and can refresh or schedule automatic refresh rate.

With dataset created we need create an Analyze to process all data in spice.

• Create an Analyze dmhc_host_analyses
• Select Spice dmhc_host_dataset
• Add two visuals type line chart
• Add the fields for X axis and Values, see the images below.
• Select Share and Publish Dashboard, create a dashboard DMHC HOST Probe.

If select the dashboard created you can see it at browser

Open the QuickSight app in the phone mobile and will see the dashboard created.

Amazon Timestream and Grafana

Grafana is another tool for create graphics and dashboards and have a plugin for Amazon Timestream. You could connect to a database and a table and analyze data time series,

We need an user credentials Access Key ID and Secret access key for Grafana access to database and tables.

AWS IAM

• Select Users, Add user
• User name: dmhc_timestreamuser
• Select AWS credential type: Access key - Programmatic access
• Add Policy : AmazonTimestreamFullAccess
• Save Access Key ID and Secret access key

Create your account at https://grafana.com/

• Login to your workspace
• Select Configuration, Plugins and add Amazon Timestream plugin
• Select Configuration, Datasources click Amazon Timestream
• Update Access Key ID and Secret Access Key from dmhc_timestreamuser created
• Default region: us-west-2
• If credentials are ok you can select the database dmhc_timestreamdb and table dmhc_timestreamtable
• Press Save&Test

• Select + Create, Dashboard
• Add empty panel, rename it DHMC HOST
• Set datasource Amazon Timestream
• Add three querys one for each value

SELECT CREATE_TIME_SERIES(time,measure_value::bigint) as temperature FROM $__database.$__table where $__timeFilter and measure_name = 'temperature'

SELECT CREATE_TIME_SERIES(time,measure_value::bigint) as spo2 FROM $__database.$__table where $__timeFilter and measure_name = 'spo2'

SELECT CREATE_TIME_SERIES(time,measure_value::bigint) as heartrate FROM $__database.$__table where $__timeFilter and measure_name = 'heartrate'

Press save and Apply and you now see all data values in a dashboard.

Amazon EventBridge, Amazon SNS, AWS Chatbot and Amazon Pinpoint

To control the frequency in which DMHC HOST Probe requests a health check, we will use the Amazon EventBridge service which will call a Lambda function dmhc_ResetDevice so that it sends a notification MQTT to the device using AWS IoT Core topic dmhc/reset that will tell you to restart and proceed to capture new data.

• Lambda Function: dmhc_ResetDevice
• Language: Node.js 14.x
• Source code at section code
• Set roles and permissions: AWSIoTFullAccess and AmazonSNSFullAccess
Change
• Change at SMS params in lambda function PhoneNume: xxxxxxxx your register phone number, see section AWS PinPoint.

var AWS = require('aws-sdk');

// Define object AWS IoT Core with endpoint
var iotdata = new AWS.IotData({endpoint: 'a2i2bsijbyml6j-ats.iot.us-west-2.amazonaws.com' });
// Define object AWS SNS for SMS and Slack notifications
var sns = new AWS.SNS();

//Call by AWS EventBridge
exports.handler = async(event, context) => {
    //Prepare json data to send notification msg to Slack
    console.log("Send msg to Slack with topic dmhc_sns_topic...");
    var params = {
        Message:  '{ \
                     "version": "0",\
                     "id":"'.concat(context.awsRequestId).concat('",').concat(' \
                     "detail-type":"Is time to check your health! Please go to DMHC Host Probe.", \
                     "source":"aws.events", \
                     "account": "537233203723", \
                     "time":"'.concat(new Date().toISOString()).concat('",').concat(' \
                     "region": "us-west-2" \
                  }')), 
        Subject: "DMHC HOST Probe",
        TopicArn: "arn:aws:sns:us-west-2:537233203723:dmhc_sns_topic"
    };
    // Publish to SNS dmhc_sns_topic
    sns.publish(params, context.done);
    

    // Create publish parameters for SMS Number
    console.log("Send msg to SMS topic...");
    var params = {
      Message: 'Is time to check your health! Please go to DMHC Host Probe.', 
      //Set here your mobile phone number
      PhoneNumber: 'YOUR PHONE NUMBER'
    };
    // Publish to SNS SMS number attach
    sns.publish(params, context.done);

    // Publish to IoT Core topic dmhc/reset
    console.log("Publish in topic dmhc/reset...");
    var params = {
        topic: "dmhc/reset", //send any msg to this topic to reset edukit
        payload: JSON.stringify(event),           
        qos: 0
    };

    //Send reset request to DMHC HOST Probe topic dmhc/reset
    const request = iotdata.publish(params);
    request
        .on('success', () => console.log("Success"))
        .on('error', () => console.log("Error"))
    return new Promise(() => request.send());
};

Amazon EventBridge

• Create a rule: dmhc_trigger_reset
• Define pattern: Fixed rate every 30 minutes
• Target: Lambda function
• Function: dmhc_ResetDevice

Prior to this, the Lambda function sends two messages to the Amazon SNS service, topic dmhc_sns_topic.

Amazon SNS

Create a topic: dmhc_sns_topic

Type Standard

One in the proper json format to be processed by an AWS Chatbot subscriber who will send a notification to a registered Slack channel dmhc-events.

AWS Chatbot

• Configure a new client
• Select Slack, configure
• Signing to your workspace
• Allow AWS Chatbot is requesting permission to access your Slack workspace
• Select slack channel to publish notifications (create dmhc-events in your slack workspace)
• Configure new channel: dmhc_chatbot_slack
• Subscribe to SNS Topic dmhc_sns_topic in us-west-2

The other with the json format suitable for a registered phone number type subscriber to send an SMS notification to a mobile.

Amazon PinPoint

We need registration a number before send SMS to it and provision a toll free number using Amazon Pinpoint.

When the number are register and have an origination number, only have a small number of tests in a sandbox environment and an amount of $1, to increment this amount need create a support case and a request to promote to production.

• Create a Project : dmhc_pinpoint_project
• Setting SMS and VoiceSetup SMS settings
• Verify a destination number for receive SMS in sandbox (Your PhoneNumber will be used by lambda function dmhc_ResetDevice)
• Request an origination toll free number

Amazon SNS

Must create a subscribe SMS number to SNS topic dmhc_sns_topic and set a subscription filter policy if exist an attribute PhoneNumber in json message because slack and SMS messages are sent to same topic and this policy avoid your receive two SMS messages.

This PhoneNumber were registered in AWS PinPoint and will be used by lambda function dmhc_ResetDevice.

Summary

Amazon Web Services used in this project (Oregon us-west-2)

• IoT Core
• IoT Analytics
• Lambda
• S3
• AppFlow
• Secrets Manager
• TimeStream
• IAM
• Polly
• SDK
• EventBridge
• SNS
• ChatBot
• PinPoint

Front End Options

• Amazon QuickSight
• App develop with Amazon Honeycode
• Grafana Dashboard
• Slack
• SMS Mobile

Hardware IoT Thing

DMHC HOST Probe = AWS IoT Edukit + Heart unit + NCIR unit + ENVIII unit