Baby NAP (Night Activity Program)

Introduction

Being a parent for the first time is a big challenge. Having a baby cry and not knowing how to calm her down faster can be extremely taxing, especially on working parents that take turns taking care of the baby. As a new parent, this is where I hope the Baby Night Activity Program I plan on building will help.

An RFID reader attached to the baby crib, coupled with RFID bracelets worn by the parents and a pressure mat sensor in the crib are used to determine when the baby is picked up, for how long, and by which parent. A sound sensor is used to record the amplitude of sound and determine if the baby is crying, for how long, and how hard. A PIR sensor and a light sensor are used to record the ambient motion and light. All these sensors are connected to an Odroid C1 development board (similar to Raspberry Pi, but with 2 analog input pins such that there is no need for a dedicated ADC convertor, as some of the above sensors are analog) that uploads the sensor data to AWS. As extras, a Phillips Hue bulb can be used to control the light color in the baby room, and a microphone can filter out the baby noise and determine if the parents are singing.

With a lambda function and a dashboard with all the sensor readings, the parents are able to draw intelligent observations about the baby's sleep preferences: does he prefer a certain ambient light color? does he like to be moved around when put to sleep? is he getting asleep faster when singing? is he waking up more when there is light in the room? Parents can also check how much night time they spend with the baby, to settle more quickly the discussion on who's turn it is to put the baby to sleep. Although this project does not intend to replace a baby monitor, it could send notifications to the parents when the baby starts crying using Amazon SNS.

Part 1: Hardware Connections

The Odroid C1 has a similar 40-pin connector as the Raspberry Pi (except the 2 analog inputs), so I will be using a Pi Cobbler+ in the diagrams below. There are 2 power rails on the breadboard, one for 5V (red wires) and one for 3.3V (orange wires). There are also 2 rails for ground (black wires), on each side of the breadboard. I will be using some test programs for each sensor, written in Python and using the WiringPi2 library.

WiringPi2 library is available for the Odroid at: https://github.com/hardkernel/WiringPi2-Python.git.

The WiringPi GPIO mapping for the C1 is available at: http://www.hardkernel.com/main/products/prdt_info.php?g_code=G141578608433&tab_idx=2

• PIR motion sensor

The PIR motion sensor has 3 pins: VCC, going to the 5V rail, GND, and DATA (yellow wire). We will be using header #13 for the DATA pin, exported by WiringPi as GPIO 2. The input from the PIR sensor is 1, when there is motion, and 0 otherwise. The sensor has 2 potentiometers, one for sensitivity, and one for time (how long after the motion was detected, the DATA output will be 1). Using trial and error, I have set the potentiometer to output 1 for approx. 10 seconds.

The test program is a very simple Python script, that polls the GPIO input every second:

import wiringpi2 as wpi
import time
wpi.wiringPiSetup()
# GPIO pin setup
wpi.pinMode(2, wpi.GPIO.INPUT)
while True:
   i=wpi.digitalRead(2)
   if i==0:
       print "no motion ", i
   elif i==1:
       print "motion detected ", i
   time.sleep(1)

Running the program and moving around should result in an output similar to:

# python pir.py
no motion  0
no motion  0
no motion  0
motion detected  1
motion detected  1
motion detected  1
motion detected  1
motion detected  1
motion detected  1
motion detected  1
motion detected  1
motion detected  1
no motion  0
no motion  0

• Pressure Mat

The pressure mat acts as a switch: when the mat is pressed, the switch is turned on. There are 2 wires to be connected - one to 3.3V (with orange wire), and one to the GPIO input pin #15 (WiringPi GPIO 3, with purple wire). We will be using a pulldown 10k resistor to connect the data wire to the ground (the black wire), which means that when the switch is open there is a path to ground and the GPIO will read 0. When the mat is pressed, because of the pin connected to 3.3V there will be a lower resistance path to high and the GPIO will read 1. We will be using also a 1k current limiting resistor on the data wire, to make sure the board will handle the current drawn when the switch is on.

The test program is also very simple:

import wiringpi2 as wpi
import time
wpi.wiringPiSetup()
# GPIO pin setup
wpi.pinMode(3, 0)
while True:
   i=wpi.digitalRead(3)
   if i==0:
       print "not pressed ", i
   elif i==1:
       print "pressed ", i
   time.sleep(1)

• Sound sensor

The Sparkfun sound detector that we will be using has 5 pins: VCC (5V), GND, GATE, ENVELOPE and AUDIO. We will be using the GATE digital output, which is 1 when sound is detected, and 0 otherwise, and the ENVELOPE analog output, which represents the amplitude of the sound. We will be connecting the GATE pin to header #29 (WiringPi GPIO 21, using yellow wire), and the ENVELOPE pin to ADC.AIN0 on header #40 (blue wire).

The following script will test sound detection:

import wiringpi2 as wpi
import time
wpi.wiringPiSetup()
# GPIO pin setup
wpi.pinMode(21, 0)
while True:
   i=wpi.digitalRead(21)
   if i==0:
       print "no sound ", i
   elif i==1:
       print "sound detected ", i
   time.sleep(1)

The sound volume we can determine by trial and error. In my case, I am reducing the output to the 0-127 value range, and use the 10-30 range as moderate (conversational) sound volume. An output below 10 means the room is quiet, and above 30 there is a high volume sound (the baby crying).

import wiringpi2 as wpi
import time
wpi.wiringPiSetup()
while True:
   i=wpi.analogRead(0)
   ampl = i*255/2047 # not sure this is correct
   if ampl <= 10:
       print "quiet: ", ampl, ", ", i
   elif ampl <= 30:
       print "moderate: ", ampl, ", ", i
   else:
       print "loud: ", ampl, ", ", i
   time.sleep(0.5)

An example output:

# python sound-env.py
quiet:  10 ,  88
quiet:  3 ,  31
moderate:  27 ,  220
moderate:  24 ,  199
moderate:  28 ,  229
moderate:  16 ,  130
loud:  52 ,  419
moderate:  16 ,  129
moderate:  30 ,  244
quiet:  4 ,  33
quiet:  6 ,  49
quiet:  3 ,  31
quiet:  5 ,  44

• Light sensor

Adafruit TSL2561 connects to the Odroid board via I2C. The connections are straightforward: 3V3 to 3.3V rail (orange wire), GND to ground (black wire), SDA to SDA (green wire), SCL to SCL (white wire).

To enable I2C on the Odroid C1, you need to load the aml_i2c kernel module, which is easiest done by appending "aml_i2c" to /etc/modules. We will also be using the tentacle_pi python module, available at https://github.com/lexruee/tentacle_pi, which allows communication with TSL2561 from Python, and outputs the light intensity in lux:

from tentacle_pi.TSL2561 import TSL2561
import time
tsl = TSL2561(0x39,"/dev/i2c-1")
tsl.enable_autogain()
tsl.set_time(0x00)
while True:
   print "lux %s" % tsl.lux()
   time.sleep(1)

• RFID card reader

The RFID card reader can connect using SPI, I2C or UART. We will be using the nfc linux library, and the easiest to configure is the UART serial connection. For this, both SEL0 and SEL1 on the PN532 rfid breakout board need to be set to OFF. As any serial connection, we will be using 5 wires: VCC (5V, red wire), ground (black wire), RX and TX (green and white) - connected to TX and RX of UART1 (/dev/ttyS2) on the board on headers #8 and #10.

"libnfc" can be installed easily on Ubuntu with the command:

# apt-get install libnfc-bin libnfc-examples libnfc-pn53x-examples

and a python wrapper can be downloaded and installed from http://nfcpy.org

In order to use libnfc, you need to configure the connection by creating a file in /etc/nfc/devices.d/, containing:

# cat /etc/nfc/devices.d/pn532_uart.conf 
## Typical configuration file for PN532 device connected using UART
name = "PN532 board via UART"
connstring = pn532_uart:/dev/ttyS2
allow_intrusive_scan = true

You can then test the connection with the pn53x-diagnose program:

# pn53x-diagnose 
pn53x-diagnose uses libnfc 1.7.0
NFC device [pn532_uart:/dev/ttyS2] opened.
Communication line test: OK
ROM test: OK
RAM test: OK

nfcpy is well documented and easy to use. To read a tag, you only need a few python lines (note the tty:S2:pn532 connection string that needs to be used):

>>> def connected(tag): print(tag); return False
...
>>> clf = nfc.ContactlessFrontend('tty:S2:pn532')
>>> clf.connect(rdwr={'on-connect': connected}) # now touch a tag
Type3Tag IDm=01010501b00ac30b PMm=03014b024f4993ff SYS=12fc
<nfc.tag.tt3.Type3Tag object at 0x7f9e8302bfd0>

Also note that since the connect function is blocking, our final code will read the NFC tag in a separate thread.

Part 2: Software

• Sending sensor data to AWS

AWS IoT provides device SDK for embedded C and Javascript (Node.js), while the libraries we use to access sensor data are all written in Python. Instead of the AWS SDK, we will be using Paho MQTT, an open source MQTT messaging library that has a Python client (available at https://www.eclipse.org/paho/clients/python/)   that is able to send data to AWS IoT.

#!/usr/bin/python
import paho.mqtt.client  as mqtt
import paho.mqtt.publish as publish
import time,json,ssl
def on_connect(mqttc, obj, flags, rc):
   if rc == 0:
       print 'Connected to the AWS IoT service!'
   else :
       print('Error connecting to AWS IoT service! (Error code ' + str(rc) + ': ' + RESULT_CODES[rc] + ')')
       client.disconnect()
client = mqtt.Client(client_id='odroid-c1', protocol=mqtt.MQTTv311)
client.on_connect = on_connect
client.tls_set('certs/root-CA.crt', certfile='certs/certificate.pem.crt', keyfile='certs/private.pem.key', tls_version=ssl.PROTOCOL_SSLv23, ciphers=None)
client.tls_insecure_set(True)
client.connect('A32L40P6IYKK8W.iot.us-east-1.amazonaws.com', 8883, 60)
client.loop_start()
msg = {data: 'test'}
client.publish('topic', json.dumps(msg))

To connect to AWS IoT, you need the following:

• root-CA.crt, available here

• certificate.pem.crt and private.pem.key,  which you can download from the AWS IoT page (see below)

• Amazon endpoint and port, which you can get from the AWS IoT page (see below)

To publish our sensor data to AWS IoT, we combine the example snippets from before to read all sensors data, and create one message that is being sent and stored in a DynamoDB database in the Amazon cloud. We want to read sensor data often (we read every 3 seconds), but this will generate a large amount of data. With this in mind, we only send one reading every minute: the maximum value read that minute in 3 seconds intervals. Please not the message is sent to a topic, named sensorTopic.

while True:
   time.sleep(3)
   # read sensor data
   ts = int(time.time())
   lux = tsl.lux()
   pir = wpi.digitalRead(2)
   mat = wpi.digitalRead(3)
   sound = wpi.digitalRead(21)
   volume = wpi.analogRead(0)*255/2047 # 0-10=quiet, 10-30=moderate, 30-127=loud
   mom = 0
   dad = 0
   if mat == 0: # if baby is not on mat, we check if mom or dad picked her up
       if nfcid == 'F10B330F': # nfc tag for mom's bracelet
           mom = 1
       elif nfcid == '833BC4A2': # nfc tag for dad's bracelet
           dad = 1
   if lux > mlux:
       mlux = lux
   if pir > mpir:
       mpir = pir
   if mat < mmat:
       mmat = mat
   if sound > msound:
       msound = sound
   if volume > mvolume:
       mvolume = volume
   # send data to AWS
   if count == 0:
       msg = {'ts': ts, 'lux': mlux, 'pir': mpir, 'mat': mmat, 'sound': msound, 'volume': mvolume, 'mom': mom, 'dad': dad}
       print json.dumps(msg)
       client.publish('sensorTopic', json.dumps(msg))
       mlux = mpir = mmat = msound = mvolume = 0
       if mmat == 1: # reset nfcid after baby is placed on mat
           nfcid = 0
   count = (count + 1) % 20

• Setting up AWS IoT

The easiest way to set up a device ("thing") for AWS IoT is using the IoT webpage: 

https://console.aws.amazon.com/iot/home?region=us-east-1#/dashboard

After you create a thing, click on it to see its properties, and then click on the Connect a device button from the Detail tab. Also please note the REST API endpoint, required to connect your MQTT client to AWS IoT.

On the next page, choose an SDK (even if you don't end up using that SDK), and then press the button Generate certificate and policy. This will create a policy for your thing, and allow you to download the certificates required to connect your MQTT client to AWS IoT.

Sensor data is being sent to a topic. We use only one topic, as we intend to use all sensor data at the same time. We will be storing this data in a DynamoDB database. For this, we have to create a IoT rule, and we choose to select all the keys (*) from the JSON message sent by our MQTT client under the topic sensorTopic.

As action, we select DynamoDB and create a new resource: a table called sensors, with different hash (primary) and range (sorted) keys. We use the hash key to store the timestamp taken by the odroid device, sent in the JSON message with the key ts. In the sort key, we store the timestamp on the AWS server, although some other JSON key can be used to facilitate sorting.

Running our program on the odroid will now insert values in the sensors DynamoDB database.

• Building a website

To make sense of our data, we will be building a web dashboard where different charts will show the sensor readings. We can host our website on Amazon S3, by creating a bucket from the S3 console, and uploading the index.html file. 

In the bucket preferences, click on Enable website hosting, and define index.html as the index document. You can see the URL endpoint, e.g. http://babynap.s3-website-us-east-1.amazonaws.com, however this endpoint supports only the http protocol. As our website will contain Javascript code, https will be required. Fortunately, the same page can be access using the URL: https://s3.amazonaws.com/babynap/index.html (replace babynap with your bucket name).

To access DynamoDB, we need to authenticate to gain permissions. We will be using Login with Amazon:

• First, you need to register your application with Amazon. In the App Console register a new application by clicking the Register new Application button and complete the form.

• From the Application screen, click Web Settings. You will automatically be assigned values for Client ID and Client Secret. The client ID identifies your website, and will be used by the Web SDK for authentication

• You need to add https://s3.amazonaws.com to Allowed JavaScript Origins or Allowed Return URLs for your application.

• Add the Web SDK to the index.html :

<script src="https://sdk.amazonaws.com/js/aws-sdk-2.2.33.min.js"></script>

• Create a new role in Amazon IAM and add your application ID to the Trust relationship; in the Permissions tab also attach a policy to allow reading of DynamoDB tables.

• Use the Web SDK to authenticate and access DynamoDB:

 <script type="text/javascript">
   // AWS credentials
   var clientId = 'amzn1.application-oa2-client.7b4ade2c6f32478d8dcesddfsdfgerg';
   var roleArn = 'arn:aws:iam::906637445412:role/aws_web_dynamoDB';
 window.onAmazonLoginReady = function() {
     amazon.Login.setClientId(clientId);
     document.getElementById('login').onclick = function() {
       amazon.Login.authorize({scope: 'profile'}, function(response) {
         if (!response.error) { // logged in
           AWS.config.credentials = new AWS.WebIdentityCredentials({
             RoleArn: roleArn,
             ProviderId: 'www.amazon.com',
             WebIdentityToken: response.access_token
           });
           // you are now logged in
           // start using amazon services
           AWS.config.region = 'us-east-1';
           db = new AWS.DynamoDB();
           // ...

• Creating dynamic charts

To build the charts, we will be using the popular charting library highcharts. First, in our webpage, we define a container for the chart:

<div id="container" style="height: 300px; min-width: 600px; max-width: 960px;"></div>

Next, in the javascript code, after we have been authenticated, we create the chart with an empty dataset, and a function that requests the data from DynamoDB:

var chartel = $('#container').highcharts('StockChart', {
   chart: {
       defaultSeriesType: 'line',
       events: {
           load: requestData
       }
   },
   title: {
       text: 'Pressure Switch (Mat)'
   },
   yAxis: {
       opposite: false,
       title: { text: 'Baby in crib' }
   },
   rangeSelector: {
       enabled: false
   },
   navigator: {
       enabled: false
   },
   scrollbar: {
       enabled: false
   },
   series : [{
       type: 'area',
       name : 'Pressure',
       data : [],
       step: true
   }]
 });
 chart = chartel.highcharts();
 chart.showLoading();

Lastly, we write the function that gets the data from DynamoDB and updates the chart. This function will call itself every minute to update the chart with newer data, and will check the maximum timestamp to perform a query on DynamoDB only for data newer that the one already shown. This way, we will have a dashboard with live charts that will show the current readings from the baby room.

  // function that requests live data from AWS DynamoDB
   function requestData() {
     if (!db) return;
     // get current max timestamp from chart
     var ts_current = chart && chart.xAxis && chart.xAxis[0].getExtremes().max ? chart.xAxis[0].getExtremes() : {max: 0};
     // request chart data from AWS, with timestamp > ts_current
     db.scan({TableName: 'sensors', FilterExpression: '#ts > :ts_current', ExpressionAttributeNames: {'#ts': 'device_ts'}, ExpressionAttributeValues: {':ts_current': {'S': String(ts_current.max)}}}, function(err, data) {
       if (err) {
         console.log(err, err.stack);
         return;
       } else {
         var chartdata = [];
         console.log("update charts with " + data.Items.length + " items");
         _.each(data.Items, function(item) {
           payload = item.payload.M;
           point = [];
           point.push(Number(payload.ts.N)*1000);
           point.push(Number(payload.lux.N));
           chartdata.push(point);
         });
         // highcharts required data to be sorted
         chartdata.sort(bySeriesTimestamp);
         chart.hideLoading();
         var i = 0;
         for (i; i < chartdata.length; i++) {
           // add data points in series, no redrawing of the chart
           chart.series[0].addPoint(chartdata[i], false);
         }
         // redraw chart
         chart.redraw();
         // run function again in 1 minute to request for newer data
         setTimeout(requestData, 60000); 
       }
     });
   }

Highcharts requires data to be sorted, and DynamoDB will serve data unsorted, so we will be using this function for sorting:

 // function to sort highcharts data array of arrays by timestamp (x)
   var bySeriesTimestamp = function(a, b){
     var ats = a[0];
     var bts = b[0]; 
     return ((ats < bts) ? -1 : ((ats > bts) ? 1 : 0));
   }

Using this code, it's possible to create a chart for each sensor, or mix data from multiple sensors on the same chart.

Part 3: Putting it all together

• Hardware

• Web Dashboard

Highcharts provides many options to highlight data. In the code submitted, I am using a navigator under the Sound Sensor chart to show when sound was detected. You can use the navigator to zoom in certain events, and all the other charts will be synchronised to the same zoom level. The sound chart shows with a red area when sound was detected, and with a blue line the volume of the sound. The region between 10 and 40 has a yellow background (moderate sound levels), and above 40 has a red background (high sound levels). Similarly, the light sensor chart has a grey background for light intensity below 40lux (night time), and yellow background for light intensity above 20000lux (too bright). Note the scale on the light chart is logarithmic.

The possibilities are endless. Currently, the next step is to build a lambda function when data is inserted into the DynamoDB database that will check when sound is detected and will send a notification to the parents. As owner of a Philips Hue lightbulb, I can read the settings of the light and compare which settings work best to settle the baby, by looking at the time between a parent picked up the baby after crying was detected and the time the baby was put back in the crib.