Alexa CoCo Sensor monitor

Have you ever forgot to pick up cloths from washing machine ? Someone rings the bell or our family members come back and you did not hear it because you are at the back of home or upstair. Coco sensor monitor can help you detect these change and send SMS to warn you. Now setting sensors to monitor are easy with Alexa voice command. Only activate sensor monitoring by saying "Alexa ask coco monitor alarm" When any alarm detected, SMS will be sent to your mobile. Let's start with this short video to present how CoCo + Alexa work together.

How It Works

In this project we use EPS8266 as the main controller, because it have built in Wifi and can writing code in Arduino. We use a lot of public libraries such as MQTT, WebSockets. ESP8266 will disable all checking sensor when startup until get command from Alexa. Alexa skill set will receive command from echo dot and call Lambda function ( CocoMonitor). CocoMonitor will check what Alexa want them to do and publish message to AWS IOT (named Pigeon). Pigeon will publish message to ESP8266 via MQTT protocol using WebSockets support by Arduino library on ESP8266. After ESP8266 get command enable sensors from Pigeon, It will begin checking all sensors ( Movement, Sound, Temperature). If any sensors detect change, it will publish result back to Pigeon. Pigeon will check with the predefined rule. If SensorFlag set to true, it will publish message to Amazon SNS which will send SMS to you.

Hardware

For Hardware building. I used my old ESP8266 adapter which include DS18B20 on board. I add some socket on board and connect wire directly to the sensors. You may need to build you own adapter.

The whole project should be small to carry around and can place anywhere you want to monitor. I got this cosmetic box from my wife and it's fit perfectly. Sorry for my poor mechanic skill. The small holes on the left side used for reset and flash button. You can use little sticks (like toothpick) to push for the reset and flash. It's very convenience :-) Small hole next to two holes is LED showing when sound detection occur. For sound sensor, you need to adjust VR on sensor board for the threshold of sound level. Use small screwdriver adjust until detection LED turn off. And test with the sound level you want to trigger.

For mobility, you can connect to power bank to make the unit mobile. Because the Coco consume very small power ( No sensor checking until it is enable) It should last long many days before recharge agian.

Software

There are 3 main parts need to implement

• Alexa skill set for Coco

• AWS service, this include Lambda function, AWS IOT and AWS SNS

• Coco firmware for ESP8266 using Arduino. we call it Pigeon

This dataflow summary all the information pass between each components of the project.

Alexa skill set for Coco

You need to apply for Alexa account at developer.amazon.com and choose Amazon Alexa, select to add new Alexa skill. Please note to copy Application ID for later use in Lambda function.

Create intent schema to tell Alexa what service and arguments you need to have. Add the following lines in intent schema

{
 "intents": [
   {
     "slots": [
       {
         "name": "Degree",
         "type": "AMAZON.NUMBER"
       }
     ],
     "intent": "MonitorMinTemp"
   },
   {
     "slots": [
       {
         "name": "Degree",
         "type": "AMAZON.NUMBER"
       }
     ],
     "intent": "MonitorMaxTemp"
   },
   {
     "intent": "MonitorMovement"
   },
   {
     "intent": "MonitorSound"
   },
   {
     "intent": "CancelTemp"
   },
   {
     "intent": "CancelMovement"
   },
   {
     "intent": "CancelSound"
   },
   {
     "intent": "AMAZON.HelpIntent"
   },
   {
     "intent": "AMAZON.StopIntent"
   },
   {
     "intent": "AMAZON.CancelIntent"
   }
 ]
}

For sample utterances put the following lines

MonitorMinTemp  monitor temperature less than {Degree}
MonitorMinTemp  check temperature less than {Degree}
MonitorMaxTemp  monitor temperature more than {Degree}
MonitorMaxTemp  monitor temperature
MonitorMaxTemp  check temperature more than {Degree}
MonitorMovement  monitor movement
MonitorMovement  monitor move
MonitorMovement  monitor intruder
MonitorMovement  monitor stranger
MonitorMovement  check movement
MonitorMovement  check move
MonitorMovement  check intruder
MonitorMovement  check stranger
MonitorSound   monitor sound
MonitorSound   monitor alarm
MonitorSound   check sound
MonitorSound   check alarm
CancelTemp  cancel temperature
CancelTemp  cancel monitor temperature
CancelMovement  cancel movement
CancelMovement  cancel monitor movement
CancelSound   cancel alarm
CancelSound   cancel sound
CancelSound   cancel monitor alarm
CancelSound   cancel monitor sound 

For configuration, set Endpoint to AWS Lambda function. We will insert this field after we create Lambda function. We can come back to Alexa skill set and modify anytime. The rest put it as default. We do not need to change anything.

After you config for Lambda ARN, you can go to test by Enable the skill and add Alexa skill ( In this case CoCo in your Alexa skill for Alexa's devices). You can test by enter utterance like " monitor movement" to check what service request send to Lambda and service response from Lambda.

AWS service configuration

After create Alexa skill, the next thing to do is config AWS services. For AWS services, we need to apply another account which will be charged if you use services above certain amount ( Yes, you need to provide Credit card details)

Goto developer.amazon.com select AWS and sign in to console. Search for Lambda in AWS services.

Create function, name it CocoMonitor

Take note for ARN on top right corner. Copy and paste this ARN in Alexa skill set for Lambda setting. Add trigger by selecting Alexa skill kit on the left side. Enable the skill and setting Skill ID of the Alexa skill (which we just created)

Select code for node.js.6.10 and Edit code inline copy the following code into index.js This code adapt from sample demonstrate how Lambda function handle Alexa skill kit. You need to modify for IOT_BROKER_ENDPOINT value. This value get from setting menu in AWS IoT, look for Custom Endpoint field.

'use strict';
/**
* This sample demonstrates a simple skill built with the Amazon Alexa Skills Kit.
* The Intent Schema, Custom Slots, and Sample Utterances for this skill, as well as
* testing instructions are located at http://amzn.to/1LzFrj6
*
* For additional samples, visit the Alexa Skills Kit Getting Started guide at
* http://amzn.to/1LGWsLG
*/
/* 
 Environment configuration setting for update thing's shadow of AWS IOT
*/
var config = {};
config.IOT_BROKER_ENDPOINT      = "XXXXXXXX.iot.us-east-1.amazonaws.com".toLowerCase();
config.IOT_BROKER_REGION        = "us-east-1";
config.IOT_THING_NAME           = "Pigeon";
//Loading AWS SDK libraries
var AWS = require('aws-sdk');
AWS.config.region = config.IOT_BROKER_REGION;
//Initializing client for IoT
var iotData = new AWS.IotData({endpoint: config.IOT_BROKER_ENDPOINT});
// --------------- Helpers that build all of the responses -----------------------
function buildSpeechletResponse(title, output, repromptText, shouldEndSession) {
   return {
       outputSpeech: {
           type: 'PlainText',
           text: output,
       },
       card: {
           type: 'Simple',
           title: `SessionSpeechlet - ${title}`,
           content: `SessionSpeechlet - ${output}`,
       },
       reprompt: {
           outputSpeech: {
               type: 'PlainText',
               text: repromptText,
           },
       },
       shouldEndSession,
   };
}
function buildResponse(sessionAttributes, speechletResponse) {
   return {
       version: '1.0',
       sessionAttributes,
       response: speechletResponse,
   };
}
// --------------- Functions that control the skill's behavior -----------------------
function getWelcomeResponse(callback) {
   // If we wanted to initialize the session to have some attributes we could add those here.
   const sessionAttributes = {};
   const cardTitle = 'Welcome';
   const speechOutput = 'Welcome to CoCo Monitor ' +
       'You can monitor movement sound and temperature by just saying ask coco monitor or cancel follow by sensors you want to check';
   // If the user either does not reply to the welcome message or says something that is not
   // understood, they will be prompted again with this text.
   const repromptText = 'Please tell me what would you like to monitor by saying, ' +
       'monitor movement or sound or temperature follow by less than or more than degree celcius';
   const shouldEndSession = false;
   callback(sessionAttributes,
       buildSpeechletResponse(cardTitle, speechOutput, repromptText, shouldEndSession));
}
function handleSessionEndRequest(callback) {
   const cardTitle = 'Session Ended';
   const speechOutput = 'Thank you for trying the Alexa Skills Kit sample. Have a nice day!';
   // Setting this to true ends the session and exits the skill.
   const shouldEndSession = true;
   callback({}, buildSpeechletResponse(cardTitle, speechOutput, null, shouldEndSession));
}
/* Comment out for old template source code
function createFavoriteColorAttributes(favoriteColor) {
   return {
       favoriteColor,
   };
}
function monitorTemperature(intent, session, callback) {
   const cardTitle = intent.name;
   const favoriteColorSlot = intent.slots.Color;
   let repromptText = '';
   let sessionAttributes = {};
   const shouldEndSession = false;
   let speechOutput = '';
   if (favoriteColorSlot) {
       const favoriteColor = favoriteColorSlot.value;
       sessionAttributes = createFavoriteColorAttributes(favoriteColor);
       speechOutput = `I now know your favorite color is ${favoriteColor}. You can ask me ` +
           "your favorite color by saying, what's my favorite color?";
       repromptText = "You can ask me your favorite color by saying, what's my favorite color?";
   } else {
       speechOutput = "Monitor temperature activate";
       repromptText = "Repeat Monitor temperatur activate"; 
   }
   callback(sessionAttributes,
        buildSpeechletResponse(cardTitle, speechOutput, repromptText, shouldEndSession));
}
function monitorMovement(intent, session, callback) {
   let favoriteColor;
   const repromptText = null;
   const sessionAttributes = {};
   let shouldEndSession = false;
   let speechOutput = '';
   if (session.attributes) {
       favoriteColor = session.attributes.favoriteColor;
   }
   if (favoriteColor) {
       speechOutput = `Your favorite color is ${favoriteColor}. Goodbye.`;
       shouldEndSession = true;
   } else {
       speechOutput = "Monitor Movement Activate ";
   }
   // Setting repromptText to null signifies that we do not want to reprompt the user.
   // If the user does not respond or says something that is not understood, the session
   // will end.
   callback(sessionAttributes,
        buildSpeechletResponse(intent.name, speechOutput, repromptText, shouldEndSession));
}
*/

function createtempattributes(temp) {
   return {
       temp,
   };
}
function monitorSound(intent, session, callback) {
   const repromptText = null;
   const sessionAttributes = {};
   let shouldEndSession = true;   // ? true of false
   let speechOutput = '';
//Activate monitor sound
   var payloadObj={ "state":
                         { "desired":
                                  {"SoundFlag": true}
                         }
         };
//Prepare the parameters of the update call
   var paramsUpdate = {
       "thingName" : config.IOT_THING_NAME,
       "payload" : JSON.stringify(payloadObj)
   };
//Update Device Shadow
   iotData.updateThingShadow(paramsUpdate, function(err, data) {
    if (err){
       //Handle the error here
        speechOutput = "There are errors in Monitor sound ";
        console.log(err);
        callback(sessionAttributes,buildSpeechletResponse(intent.name, speechOutput, repromptText, shouldEndSession));
     }
     else {
       speechOutput = "Monitor sound activated!";
       console.log(data);
       callback(sessionAttributes,buildSpeechletResponse(intent.name, speechOutput, repromptText, shouldEndSession));
     }    
   });
}
function monitorMovement(intent, session, callback) {
   const repromptText = null;
   const sessionAttributes = {};
   let shouldEndSession = true;   // ? true of false
   let speechOutput = '';
//Activate monitor sound
   var payloadObj={ "state":
                         { "desired":
                                  {"MoveFlag": true}
                         }
         };
//Prepare the parameters of the update call
   var paramsUpdate = {
       "thingName" : config.IOT_THING_NAME,
       "payload" : JSON.stringify(payloadObj)
   };
//Update Device Shadow
   iotData.updateThingShadow(paramsUpdate, function(err, data) {
    if (err){
       //Handle the error here
        speechOutput = "There are errors in Monitor movement ";
        console.log(err);
        callback(sessionAttributes,buildSpeechletResponse(intent.name, speechOutput, repromptText, shouldEndSession));
     }
     else {
       speechOutput = "Monitor movement activated!";
       console.log(data);
       callback(sessionAttributes,buildSpeechletResponse(intent.name, speechOutput, repromptText, shouldEndSession));
     }    
   });
}
function monitorMinTemp (intent, session, callback) {
   const repromptText = null;
   var chkTemp = intent.slots.Degree;
   var sessionAttributes = {};
   let shouldEndSession = true;   // ? true of false
   let speechOutput = '';
    if (chkTemp) {
       var temp = chkTemp.value;
       sessionAttributes =  createtempattributes(temp);
       speechOutput = `Monitor temperature ${temp}`;
//        repromptText = "You can ask me to monitor temperature ?";
   } 
   else 
   {
       speechOutput = "Monitor temperature activate";
//        repromptText = "Repeat Monitor temperature activate"; 
   }
   var  payloadObj={ "state":
                         { "desired":
                                  {"TempFlag": 'min',
                                    "degree": temp}
                          }
         };
//Prepare the parameters of the update call
   var paramsUpdate = {
       "thingName" : config.IOT_THING_NAME,
       "payload" : JSON.stringify(payloadObj)
   };
//Update Device Shadow
   iotData.updateThingShadow(paramsUpdate, function(err, data) {
    if (err){
       //Handle the error here
        speechOutput = "There are errors in Monitor temperature ";
        console.log(err);
        callback(sessionAttributes,buildSpeechletResponse(intent.name, speechOutput, repromptText, shouldEndSession));
     }
     else {
       speechOutput = "Monitor temperature activated!";
       console.log(data);
       callback(sessionAttributes,buildSpeechletResponse(intent.name, speechOutput, repromptText, shouldEndSession));
     }    
   });
}
function monitorMaxTemp (intent, session, callback) {
   const repromptText = null;
   var chkTemp = intent.slots.Degree;
   var sessionAttributes = {};
   let shouldEndSession = true;   // ? true of false
   let speechOutput = '';
    if (chkTemp) {
       var temp = chkTemp.value;
       sessionAttributes =  createtempattributes(temp);
       speechOutput = `Monitor temperature ${temp}`;
//        repromptText = "You can ask me to monitor temperature ?";
   } 
   else 
   {
       speechOutput = "Monitor temperature activate";
//        repromptText = "Repeat Monitor temperature activate"; 
   }
   var  payloadObj={ "state":
                         { "desired":
                                  {"TempFlag": 'max',
                                    "degree": temp}
                         }
         };
//Prepare the parameters of the update call
   var paramsUpdate = {
       "thingName" : config.IOT_THING_NAME,
       "payload" : JSON.stringify(payloadObj)
   };
//Update Device Shadow
   iotData.updateThingShadow(paramsUpdate, function(err, data) {
    if (err){
       //Handle the error here
        speechOutput = "There are errors in Monitor temperature ";
        console.log(err);
        callback(sessionAttributes,buildSpeechletResponse(intent.name, speechOutput, repromptText, shouldEndSession));
     }
     else {
       speechOutput = "Monitor temperature activated!";
       console.log(data);
       callback(sessionAttributes,buildSpeechletResponse(intent.name, speechOutput, repromptText, shouldEndSession));
     }    
   });
}
function cancelSound(intent, session, callback) {
   const repromptText = null;
   const sessionAttributes = {};
   let shouldEndSession = true;   // ? true of false
   let speechOutput = '';
//Activate monitor sound
   var payloadObj={ "state":
                         { "desired":
                                  {"SoundFlag": false}
                         }
         };
//Prepare the parameters of the update call
   var paramsUpdate = {
       "thingName" : config.IOT_THING_NAME,
       "payload" : JSON.stringify(payloadObj)
   };
//Update Device Shadow
   iotData.updateThingShadow(paramsUpdate, function(err, data) {
    if (err){
       //Handle the error here
        speechOutput = "There are errors in cancel sound ";
        console.log(err);
        callback(sessionAttributes,buildSpeechletResponse(intent.name, speechOutput, repromptText, shouldEndSession));
     }
     else {
       speechOutput = "Cancel monitor sound";
       console.log(data);
       callback(sessionAttributes,buildSpeechletResponse(intent.name, speechOutput, repromptText, shouldEndSession));
     }    
   });
}
function cancelMovement(intent, session, callback) {
   const repromptText = null;
   const sessionAttributes = {};
   let shouldEndSession = true;   // ? true of false
   let speechOutput = '';
//Activate monitor sound
   var payloadObj={ "state":
                         { "desired":
                                  {"MoveFlag": false}
                         }
         };
//Prepare the parameters of the update call
   var paramsUpdate = {
       "thingName" : config.IOT_THING_NAME,
       "payload" : JSON.stringify(payloadObj)
   };
//Update Device Shadow
   iotData.updateThingShadow(paramsUpdate, function(err, data) {
    if (err){
       //Handle the error here
        speechOutput = "There are errors in cancel movemnt ";
        console.log(err);
        callback(sessionAttributes,buildSpeechletResponse(intent.name, speechOutput, repromptText, shouldEndSession));
     }
     else {
       speechOutput = "Cancel monitor movement";
       console.log(data);
       callback(sessionAttributes,buildSpeechletResponse(intent.name, speechOutput, repromptText, shouldEndSession));
     }    
   });
}
function cancelTemp(intent, session, callback) {
   const repromptText = null;
   const sessionAttributes = {};
   let shouldEndSession = true;   // ? true of false
   let speechOutput = '';
//Activate monitor sound
   var payloadObj={ "state":
                         { "desired":
                                  {"TempFlag": "disable"}
                         }
         };
//Prepare the parameters of the update call
   var paramsUpdate = {
       "thingName" : config.IOT_THING_NAME,
       "payload" : JSON.stringify(payloadObj)
   };
//Update Device Shadow
   iotData.updateThingShadow(paramsUpdate, function(err, data) {
    if (err){
       //Handle the error here
        speechOutput = "There are errors in cancel temperature ";
        console.log(err);
        callback(sessionAttributes,buildSpeechletResponse(intent.name, speechOutput, repromptText, shouldEndSession));
     }
     else {
       speechOutput = "Cancel monitor temperature";
       console.log(data);
       callback(sessionAttributes,buildSpeechletResponse(intent.name, speechOutput, repromptText, shouldEndSession));
     }    
   });
}
// --------------- Events -----------------------
/**
* Called when the session starts.
*/
function onSessionStarted(sessionStartedRequest, session) {
   console.log(`onSessionStarted requestId=${sessionStartedRequest.requestId}, sessionId=${session.sessionId}`);
}
/**
* Called when the user launches the skill without specifying what they want.
*/
function onLaunch(launchRequest, session, callback) {
   console.log(`onLaunch requestId=${launchRequest.requestId}, sessionId=${session.sessionId}`);
   // Dispatch to your skill's launch.
   getWelcomeResponse(callback);
}
/**
* Called when the user specifies an intent for this skill.
*/
function onIntent(intentRequest, session, callback) {
   console.log(`onIntent requestId=${intentRequest.requestId}, sessionId=${session.sessionId}`);
   const intent = intentRequest.intent;
   const intentName = intentRequest.intent.name;
   // Dispatch to your skill's intent handlers
   if (intentName === 'MonitorMinTemp') {
       monitorMinTemp(intent, session, callback);
   } else if (intentName === 'MonitorMaxTemp') {
       monitorMaxTemp(intent, session, callback);    
   } else if (intentName === 'MonitorMovement') {
       monitorMovement(intent, session, callback);
   } else if (intentName === 'MonitorSound') {
       monitorSound(intent, session, callback);
   } else if (intentName === 'CancelSound') {
       cancelSound(intent, session, callback);
   } else if (intentName === 'CancelMovement') {
       cancelMovement(intent, session, callback);
   } else if (intentName === 'CancelTemp') {
       cancelTemp(intent, session, callback);
   } else if (intentName === 'AMAZON.HelpIntent') {
       getWelcomeResponse(callback);
   } else if (intentName === 'AMAZON.StopIntent' || intentName === 'AMAZON.CancelIntent') {
       handleSessionEndRequest(callback);
   } else {
       throw new Error('Invalid intent');
   }
}
/**
* Called when the user ends the session.
* Is not called when the skill returns shouldEndSession=true.
*/
function onSessionEnded(sessionEndedRequest, session) {
   console.log(`onSessionEnded requestId=${sessionEndedRequest.requestId}, sessionId=${session.sessionId}`);
   // Add cleanup logic here
}
// --------------- Main handler -----------------------
// Route the incoming request based on type (LaunchRequest, IntentRequest,
// etc.) The JSON body of the request is provided in the event parameter.
exports.handler = (event, context, callback) => {
   try {
       console.log(`event.session.application.applicationId=${event.session.application.applicationId}`);
       /**
        * Uncomment this if statement and populate with your skill's application ID to
        * prevent someone else from configuring a skill that sends requests to this function.
        */
       if (event.session.application.applicationId !== 'amzn1.ask.skill.9a1f9785-8db2-43f1-bee6-64621aa056aa') {
            callback('Invalid Application ID');
       }
       if (event.session.new) {
           onSessionStarted({ requestId: event.request.requestId }, event.session);
       }
       if (event.request.type === 'LaunchRequest') {
           onLaunch(event.request,
               event.session,
               (sessionAttributes, speechletResponse) => {
                   callback(null, buildResponse(sessionAttributes, speechletResponse));
               });
       } else if (event.request.type === 'IntentRequest') {
           onIntent(event.request,
               event.session,
               (sessionAttributes, speechletResponse) => {
                   callback(null, buildResponse(sessionAttributes, speechletResponse));
               });
       } else if (event.request.type === 'SessionEndedRequest') {
           onSessionEnded(event.request, event.session);
           callback();
       }
   } catch (err) {
       callback(err);
   }
};

Select Service at the top and find for IAM ( Identity and Access Management). Go to IAM service and select users on left pane Select Add user.

Enter user name and select Access type as Programmatic access. Click Next Permissions. Select Attach existing policies directly and select for AWSIoTFullAccess. Click create user.

After you create user you will get Access key ID and Secret access key which will be put in Arduino program for connect with AWS IOT. You can download *.csv for later used. Not share this with others.

Go back to IAM service and select Roles on the left pane. Select Create role, select for AWS service, under AWS service select for Lambda, click Next Permissions.

Enter IOT in the filter and select AWSIoTFullAccess, click Next Review

Enter Role name (AlexaCoco) and Role description and create role. This new role will allow Lambda functions to call AWS services on your behalf.

Go back to Lambda and select Execution Role to use existing role which we just created

That end for config Lambda function which will be called from Alexa skill kit. The next thing we need to do is config AWS IOT for communicate with ESP8266 and AWS SNS. Lambda function will publish message about flag to monitor for each sensors using MQTT protocol to AWS IOT.

AWS IOT

Go to Services and search for AWS IoT service. Select Manage and create things, name it Pigeon and create Group ESP8266. put Pigeon into this group.

Note for Rest API Endpoint in Manage Pigeon and select Interact on the left pane. This value will use both in Arduino Code and Lambda function for IOT_BROKER_ENDPOINT to access the IOT.

Config for Amazon SNS

Go to services again and select SNS. Select create topic, enter topic name and Display name note for ARN of the topic. Go to subscriptions and select create subscription enter the following values

Topic ARN : Topic ARN that just created

Protocol: SMS

EndPoint: Phone number prefix with country code

Go back to AWS IOT again to config for Action when IOT got publish message from ESP8266. Select Act on the left pane and click Create Rules

Create rule from SQL command

SELECT state.reported.* FROM '$aws/things/Pigeon/shadow/update/accepted' WHERE state.reported.SensorFlag = true

Add action by select Send a message as an SNS push notification and configure action to use SNS

Create PigeonMonitor role, click update. Don't forget to enable Rules. Attach SNS publish policy to this role.

Go to setting and enable event-based messages to created, updated, deleted topic

Arduino program for ESP8266 firmware

Arduino Program will subscribe messages from AWS IOT about checking sensors flag. If sensor flags are enable. It will start to monitor the sensors. And if the sensors detect change. It will publish message back to IOT. We use MQTT protocol over websockets by using aws-mqtt-websockets library for Arduino. For configuration, we need to enter following variables into program

• WIfi SSID

• Wifi Password

• IoT Endpoint, get from AWS IOT Rest API Endpoint

• AWS Access Key ID and Secret Key ID, get from IAM users which attach to AWSIoTFullAccess policy.

Future Improvement

There are a lot of features waiting for enhancement, For example

• Use Alexa notification service which can respond back via Voice. I understand that it's in Beta test at this moments. And only limited developer can access it.

• SMS message still look not human readable. It should be format properly. I cannot find documents how to format SMS message nicely, only filter message.

• Design to add/remove sensors easily. So we can change any sensors we want. But we need to find ways to config via Alexa like " Alexa ask CoCo add XXX sensors on port YYY"

• Program SMS number via Alexa

I hope you find this project useful and use this project as the base for expand more features in the future. Thank you.