Sleep Monitoring System based on AWS IoT EduKit

Mission

This project began with the idea of merging a sleep monitoring system with machine learning capabilities. The idea was to not only generate and visualize sleep data, but to eventually create a forecast of one's sleep. The project itself can be broken down into the following three goals:

• Have a low profile, battery powered device that can report to AWS motion while sleeping and ambient temperature.
• Be able to store, process, and visualize the incoming data in a web application.
• Present a viable model based on long term data that can forecast an individual's sleep patterns

Story

I'll begin by saying that this is definitely still a work in progress, but I hope that the work which has been done on the device, thus far, will be useful to others dealing with the AWS Iot EduKit. Of the goals mentioned in the mission statement, only one has been achieved, and it's still a little buggy:

• Have a low profile, battery powered device that can report to AWS motion while sleeping and ambient temperature.

So, I'll go through the code that is being used, along with some of the logic behind sampling methods, and leave the second two bullet points as cliff hangers for further work.

As mentioned, the device being used is the AWS IoT EduKit, chosen as part of the Amazon Web Services reinventing healthy spaces competition. The device can be mounted to a bed frame in order to monitor room conditions and vibrations from any movement in bed. At first, the idea was to measure as many parameters as possible: noise, temperature, rotation and acceleration, moisture levels, pressure, ambient light... But, to keep things simple, I started with temperature, acceleration, and rotation. These were measured using the MPU6886 chip capabilities onboard the EduKit.

The accelerometer and gyro measurements from the MPU6886 are meant to serve as an indication of movement in bed, which is my proxy for sleep quality. There are probably a number of ways to go when determining this important variable, but many commercial products on the market that measure sleep do so by measuring motion. Thus, this seemed like an effective, minimalistic solution, especially since the capabilities are already present on the EduKit.

Code

The framework for the device firmware is the Blinky-Hello-World.c example found in the EduKit GitHub Repository.

All of the modifications were done in main.c of the Blinky project, along with one minor modification in the CMakeLists.txt file under the main directory:

set(COMPONENT_REQUIRES "nvs_flash" "esp-aws-iot" "esp-cryptoauthlib" "core2forAWS" "json")

"json" is included to pull in the JSON library used for packaging payloads to be sent to AWS.

In main.c, cJSON.h must be included:

#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/semphr.h"
#include "freertos/event_groups.h"
#include "esp_log.h"
#include "cJSON.h" // Included for creating the JSON file payloads

Next, a few parameters need to be defined for use later in the code:

#define STARTING_ROOMTEMPERATURE 0.0f

/* The time between each MQTT message publish in milliseconds */
#define PUBLISH_INTERVAL_MS 5000

/* The time between each sensor data gathering period (in seconds)*/
#define SENSOR_READING_AVERAGE_DELAY 4.6

/* The time between reading the sensor data once, before the next read (in seconds).
This should be a factor of SENSOR_READING_AVERAGE_DELAY, otherwise you risk doing sensor readings beyond 5 seconds*/
#define SENSOR_READING_DELAY 0.2

/* Specify variable for keeping track of time in vTaskDelayUntil */
TickType_t xLastWakeTime;

and a few variables need to be initialized:

/* Initialize all the variables to be used for gathering sensor data and averages */
float temperature = 0;
float gx = 0;
float gy = 0;
float gz = 0;
float ax = 0;
float ay = 0;
float az = 0;

float avg_temperature = 0;
float avg_gx = 0;
float avg_gy = 0;
float avg_gz = 0;
float avg_ax = 0;
float avg_ay = 0;
float avg_az = 0;

float Sum_Temperature = 0;
float Sum_Gyro_x = 0;
float Sum_Gyro_y = 0;
float Sum_Gyro_z = 0;
float Sum_Accel_x = 0;
float Sum_Accel_y = 0;
float Sum_Accel_z = 0;

Next, I created a function called sensorData() which is used to actually pull data from the MPU6886, and process it before it is sent on to be published. This is really the meat of the code. In order to reduce publish frequency, and subsequently AWS MQTT charges, this function will report periodically, but infrequently, averaged values over a certain time period to AWS. The function takes pointer type variables as inputs so that when it is called, averages at those addresses can be used in future operations. Data is gathered in time to fit within the publishing frequency window. We'll need sums of data during that time period, so those values get initialized first.

void sensorData(float *Avg_Gyro_x, float *Avg_Gyro_y, float *Avg_Gyro_z, float *Avg_Accel_x, float *Avg_Accel_y, float *Avg_Accel_z, float *Avg_Temperature){

/* Reset all summing variables and the counter to zero */
Sum_Temperature = 0;
Sum_Gyro_x = 0;
Sum_Gyro_y = 0;
Sum_Gyro_z = 0;
Sum_Accel_x = 0;
Sum_Accel_y = 0;
Sum_Accel_z = 0;
int count_i = 0;

The data needs to be averaged in a way that makes sense for the task at hand. For temperature, simply adding up the measure values over a period of time, and dividing by the number of measurements, should suffice.

Motion data is a little trickier, however. Since vibrations will likely have a closely matching negative rotation/acceleration for each positive one, the values can't just be added - they must be unsigned. In addition, because we want the spread between no motion and large motion to be wide, the measured values get squared before being summed. This allows for more obvious detection of weak motion, and a stronger indication of large motion.

/* Create a basic timer to control the data collection loop, reporting averages according to SENSOR_READING_AVERAGE_DELAY */
clock_t begin_Time = clock();

while ( ((unsigned long) (clock() - begin_Time)/CLOCKS_PER_SEC) < SENSOR_READING_AVERAGE_DELAY){

MPU6886_GetTempData(&temperature);
Sum_Temperature += (temperature * 1.8)  + 32 - 80;


/*Taking the square of each value so that averaging makes more sense with both positives and negatives coming in,
and so motion is weighted more (trying to pick up smaller motion)*/

MPU6886_GetAccelData(&ax, &ay, &az);
Sum_Accel_x += ax*ax;
Sum_Accel_y += ay*ay;
Sum_Accel_z += az*az;

MPU6886_GetGyroData(&gx, &gy, &gz);
Sum_Gyro_x += gx*gx;
Sum_Gyro_y += gy*gy;
Sum_Gyro_z += gz*gz;

count_i++;

/* To keep from accumulating huge numbers in the Sum_xxx variables, the while loop below delays for SENSOR_READING_DELAY
This is probably not the best implementation, but should work for the sleep monitor's purposes*/
clock_t start_Time = clock();
while ( ((unsigned long) (clock() - start_Time)/CLOCKS_PER_SEC) < SENSOR_READING_DELAY){}
}

Note the implementation of timers in this portion of the code. One while loop is keeping track of the averaging period ~4-5 seconds, and the other while loop is providing a delay between each measurement, so as not to flood the device memory. The values chosen for these timers are 4.6s (total gathering time) and 0.2s (delay). 4.6s allows time after measurement to average, package, and publish the data every 5 seconds.

**Warning for other makers: There is suspicion that the timer implementation (while loops instead of alarms or interrupts) is causing malloc errors after a period of time with the device running.

After collecting the data, it needs to packaged and published. This is done in the publisher() function. A JSON payload is created with all of the temperature, acceleration, and gyro measurements included. The message is then sent with Quality of Service 1, so that the device needs an ack from AWS to confirm reception.

static void publisher(AWS_IoT_Client *client, char *base_topic, uint16_t base_topic_len){

IoT_Publish_Message_Params paramsQOS1;
paramsQOS1.qos = QOS1;
paramsQOS1.isRetained = 0;

cJSON *payload = cJSON_CreateObject();

cJSON *temperature_value = cJSON_CreateNumber(avg_temperature);
cJSON_AddItemToObject(payload, "temperature_value", temperature_value);

cJSON *accel_x_value = cJSON_CreateNumber(avg_ax);
cJSON *accel_y_value = cJSON_CreateNumber(avg_ay);
cJSON *accel_z_value = cJSON_CreateNumber(avg_az);
cJSON_AddItemToObject(payload, "accel_x_value", accel_x_value);
cJSON_AddItemToObject(payload, "accel_y_value", accel_y_value);
cJSON_AddItemToObject(payload, "accel_z_value", accel_z_value);

cJSON *gyro_x_value = cJSON_CreateNumber(avg_gx);
cJSON *gyro_y_value = cJSON_CreateNumber(avg_gy);
cJSON *gyro_z_value = cJSON_CreateNumber(avg_gz);
cJSON_AddItemToObject(payload, "gyro_x_value", gyro_x_value);
cJSON_AddItemToObject(payload, "gyro_y_value", gyro_y_value);
cJSON_AddItemToObject(payload, "gyro_z_value", gyro_z_value);

const char *JSONPayload = cJSON_Print(payload);
paramsQOS1.payload = (void*) JSONPayload;
paramsQOS1.payloadLen = strlen(JSONPayload);

// Publish and check if "ack" was sent from AWS IoT Core
IoT_Error_t rc = aws_iot_mqtt_publish(client, base_topic, base_topic_len, &paramsQOS1);

if (rc == MQTT_REQUEST_TIMEOUT_ERROR) {
ESP_LOGW(TAG, "QOS1 publish ack not received.");
rc = SUCCESS;
}
}

The final piece of code to consider is calling these two functions in the aws_iot_task, and setting up the 5 second timer. To do so, in app_main, the wake time is initialized in order to be used with vTaskDelayUntil(), which will wait 5 seconds since the last execution before beginning the aws_iot_task while loop again (the loop that is run indefinitely if connection to AWS is good).

app_main addition:

/* Initialize the tick time for use in vTaskDelayUntil*/
xLastWakeTime = xTaskGetTickCount();

And the publisher(), sensorData(), and vTaskDelayUntil() in the aws_iot_task loop:

sensorData(&avg_gx, &avg_gy, &avg_gz, &avg_ax, &avg_ay, &avg_az, &avg_temperature);
publisher(&client, base_publish_topic, BASE_PUBLISH_TOPIC_LEN);
vTaskDelayUntil(&xLastWakeTime, pdMS_TO_TICKS(PUBLISH_INTERVAL_MS));

That does it for the code. It is worth noting that the timing implementation does not seem to be as precise as in theory. The data is finally packaged and sent to AWS.

Example Data

The following are examples with the device placed on top of a head board, moving around (1) minimally and (2) tossing and turning:

no motion example -

{ "temperature_value": 25.075153350830078, "accel_x_value": 0.0004551649035420269, "accel_y_value": 0.00015581845946144313, "accel_z_value": 1.1685764789581299, "gyro_x_value": 17.144529342651367, "gyro_y_value": 0.4716217517852783, "gyro_z_value": 23.804603576660156}

tossing and turning example -

{ "temperature_value": 24.038557052612305, "accel_x_value": 0.0009434580570086837, "accel_y_value": 0.0001554012269480154, "accel_z_value": 1.16836678981781, "gyro_x_value": 137.39096069335938, "gyro_y_value": 19.99363136291504, "gyro_z_value": 565.6026000976562}

Moving Forward

Obviously, there is a decent amount of work left to do on this concept. I did manage to complete the bulk of device firmware, and create/prove a method for measuring sleep motion and environment conditions. However, the novel concept of forecasting sleep will still require data engineering and app work. Future work will include:

• Storing data in AWS IoT analytics
• Creating a web application, or utilizing AWS services to visualize and correlate sleep data
• Executing a machine learning model to provide sleep forecasts, and improve data representation.