Introduction
Example User Experience Flow
Solution Build - Alexa Skill + AWS + Intel NUC + Arduino 101 and Grove Kit
FAQs

Team ConvoCare:

•

•

•

•

Published January 8, 2017 © MIT

Convo-Care: Alexa-Controlled E-Health App

Convo-Care provides a better e-health experience through a conversational UI.

IntermediateProtip2 hours1,792

Convo-Care: Alexa-Controlled E-Health App

Things used in this project

Hardware components

Amazon Alexa Echo Dot

Intel NUC Kit

Seeed Studio Grove starter kit plus for Intel Edison

Software apps and online services

Amazon Alexa Alexa Skills Kit

Amazon Web Services AWS Lambda

Amazon Web Services AWS IoT

Story

Introduction

Smart cities adopt connected health and technology-enabled care to provide citizens the ability to self-manage their health and well-being, alert healthcare professionals to changes in their condition and support medication adherence. Leveraging digital technology, safer, more efficient social care can be delivered.

One problem in the adoption of technology-enabled care solutions has been the limitation of human interface. The access to health information has always been limited to healthcare providers’ manual input into a centralized medical record database, and can be hard to synchronize across multiple stakeholders. The result is a lack of inter-operable patient records and inaccessibility of critical health information during medical incidents.

Imagine a situation where you become unconscious at home; paramedics arrive, but they need information such as allergies and medical histories, etc., to provide the best possible treatment. Traditionally, there's no easy way to obtain that data; however, if you have all that information stored in a database, and the paramedics are able to access that via your home Alexa-enabled device, the difference could be life or death.

Convo-Care is a voice controlled e-health platform for better health records management and accessibility. It uses natural language and voice interface to help people manage their own health records, and allows healthcare providers to retrieve accurate records at ease. It is extremely useful for people with mobility problems, or during critical or emergency conditions.

This application solves two major safety hazards that are found in urban environments: difficulty accessing accurate medical information and difficulty contacting emergency services when unable to reach a phone. The hazard is increased further when the user suffers from conditions such as dementia, Alzheimer's, or various mobility issues. With Convo-Care, the user can be assured that help is always available, even if the user lives alone.

Here's a demo of the application in action:

demo

Example User Experience Flow

User Experience Flow

Data Entry

Step 1: User talks to Alexa to input profile data, medical history, emergency contact etc.

Step 2: The Alexa skill is called, which interacts with the Convo-Care Cloud (Powered by AWS) to add records to database.

Step 3: Alexa responds to the user with a confirmation that the record has been updated.

Event Trigger

Step 1: User uses Alexa to call emergency medical services.

Step 2: The Alexa skill is called, which interacts with cloud based e-911 service.

Step 3: In parallel, the Alexa skill triggers AWS IoT to unlock the front door (assuming smart lock installed).

Step 4: Alexa responds to user with confirmation that help is on the way.

Action Scenario

Step 1: Help (EMS) arrives and sees that the home is Convo-Care enabled.

Step 2: Paramedic talks to Alexa to request retrieving of medical profile.

Step 3: The Alexa skill is called and relay critical medical info to EMS.

Solution Build - Alexa Skill + AWS + Intel NUC + Arduino 101 and Grove Kit

To emulate the most complex part of this service, where the Alexa skill triggers the smart lock, we used Intel NUC gateway developer kit to illustrate the logic.

e-health app with IoT integration for an interoperable platform

The Convo-Care Alexa skill consists of the following dynamic data input and output capabilities:

name

gender

height

weight

date of birth

blood type

allergies

medications

existing conditions

emergency contact

quick medical profile

With this skill the user can update their personal health profile with new data any time easily via voice control. Imagine the user just got back from doctor visit, and have a prescription update. The voice input will automatically change the record for "medications" in the database.

If a medical emergency occurs, and the user experience mobility or mental issues, the user can speak "emergency" or "911" to trigger Alexa skill. Upon paramedic arrival, following are some of the phrases which can be asked:

What is her blood type?

Give me the medications.

Does he have any allergies?

What is the medical profile?

Setting up backend database on AWS required these configuration and dependencies:

1 DynamoDB table

1 S3 bucket

The Intel Gateway with sensors and actuators were configured with Node-RED via Intel Developer Hub. We used:

Grove button - read button presses, emulate any active input from user

Grove light - measure light level, emulate any passive sensors in the environment

Grove LED - set LED indicator on/off, emulate a function working

mqtt - Send data to AWS IoT, the output node adds a secure broker for communications between the edge and the cloud

FAQs

Q: Doesn't this already exist with services like Life Alert?

A: This is a hands-free version which is not dependent on the existing hardware. Many situations can occur with the user unable to reach for a phone or press a button for help.

Q: Many target users may find this difficult to set up or may resist new technology. Why is that not the case here?

A: This technology would likely be purchased and implemented by the children of the elderly or family of disabled users for additional safety and peace of mind.

Code

'use strict';

const Alexa = require('alexa-sdk');
const records = require('./records');

const APP_ID = 'your app id';

const handlers = {
    'NewSession': function () {
        this.attributes.speechOutput = this.t('WELCOME_MESSAGE', this.t('SKILL_NAME'));
        this.attributes.repromptSpeech = this.t('WELCOME_REPROMT');
        this.emit(':ask', this.attributes.speechOutput, this.attributes.repromptSpeech);
    },
    'RecipeIntent': function () {
        const itemSlot = this.event.request.intent.slots.Item;
        let itemName;
        if (itemSlot && itemSlot.value) {
            itemName = itemSlot.value.toLowerCase();
        }

        const cardTitle = this.t('DISPLAY_CARD_TITLE', this.t('SKILL_NAME'), itemName);
        const myRecipes = this.t('RECORDS');
        const record = myRecipes[itemName];

        if (record) {
            this.attributes.speechOutput = record;
            this.attributes.repromptSpeech = this.t('RECORD_REPEAT_MESSAGE');
            this.emit(':askWithCard', record, this.attributes.repromptSpeech, cardTitle, record);
        } else {
            let speechOutput = this.t('RECORD_NOT_FOUND_MESSAGE');
            const repromptSpeech = this.t('RECORD_NOT_FOUND_REPROMPT');
            if (itemName) {
                speechOutput += this.t('RECORD_NOT_FOUND_WITH_ITEM_NAME', itemName);
            } else {
                speechOutput += this.t('RECORD_NOT_FOUND_WITHOUT_ITEM_NAME');
            }
            speechOutput += repromptSpeech;

            this.attributes.speechOutput = speechOutput;
            this.attributes.repromptSpeech = repromptSpeech;

            this.emit(':ask', speechOutput, repromptSpeech);
        }
    },
    'EmergencyIntent': function () {
      const sessionAttributes = {};
      const cardTitle = 'Emergency';
      const speechOutput = 'Don\'t worry, I\'m here to help, I have contacted emergency services and they are on their way.';
      const repromptText = 'I have already sent emergency services to your location.';
      const shouldEndSession = false;
      this.emit(':ask', speechOutput, repromptText);
    },
    'AddIntent': function () {
      const sessionAttributes = {};
      const cardTitle = 'Add Info';
      const addSpeechOutput = 'Okay, I\'ll record that in your profile.';
      const addRepromptText = 'I recorded what you told me in your profile.';
      const shouldEndSession = false;
      this.emit(':ask', addSpeechOutput, addRepromptText);
    },
    'AMAZON.HelpIntent': function () {
    this.emit('EmergencyIntent');
    },
    'AMAZON.RepeatIntent': function () {
        this.emit(':ask', this.attributes.speechOutput, this.attributes.repromptSpeech);
    },
    'AMAZON.StopIntent': function () {
        this.emit('SessionEndedRequest');
    },
    'AMAZON.CancelIntent': function () {
        this.emit('SessionEndedRequest');
    },
    'SessionEndedRequest': function () {
        this.emit(':tell', this.t('STOP_MESSAGE'));
    },
};

const languageStrings = {
    'en-US': {
        translation: {
            RECORDS: records.RECORD_EN_US,
            SKILL_NAME: 'ConvoCare',
            WELCOME_MESSAGE: "%s is ready to help.",
            WELCOME_REPROMT: 'You can ask for medical information such as the name, age, date of birth, allergies, or medications.',
            DISPLAY_CARD_TITLE: '%s  - Information for %s.',
            HELP_MESSAGE: "I can provide medical information on the occupant. You can ask questions such as, what is her blood type?" +
            "For a summary, you can say, give me the medical profile.",
            HELP_REPROMT: "You can say things like, what is her blood type?, or you can say exit...how can i help?",
            STOP_MESSAGE: 'Goodbye!',
            RECORD_REPEAT_MESSAGE: 'Try saying repeat.',
            RECORD_NOT_FOUND_MESSAGE: "I\'m sorry, I currently do not know ",
            RECORD_NOT_FOUND_WITH_ITEM_NAME: 'the information for %s. ',
            RECORD_NOT_FOUND_WITHOUT_ITEM_NAME: 'that information. ',
            RECORD_NOT_FOUND_REPROMPT: 'What else can I help with?',
        },
    }
};

exports.handler = (event, context) => {
    const alexa = Alexa.handler(event, context);
    alexa.APP_ID = APP_ID;
    alexa.resources = languageStrings;
    alexa.registerHandlers(handlers);
    alexa.execute();
};

module.exports = {
    RECORD_EN_US: {
        'name': 'The occupant\'s name is jane doe.',
        'gender': 'The occupant\'s gender is female.',
        'age': 'The occupant\'s age is 64.',
        'height' : 'The occupant\'s height is 5 foot 4 inches.',
        'weight' : 'The occupant\'s weight is 160 pounds.',
        'date of birth': 'The occupant\'s date of birth is august 22nd 1952.',
        'birth day': 'The occupant\'s birth day is august 22nd 1952.',
        'birth date': 'The occupant\'s birth date is august 22nd 1952.',
        'blood type': 'The occupant\'s blood type is O positive.',
        'allergies': 'The occupant is allergic to latex and peanuts.',
        'medications': 'The occupant is currently taking Verapamil, 100 milligrams daily.',
        'existing conditions': 'The occupant has a history of high blood pressure.',
        'emergency contact': 'The emergency contact\'s name is Peter Smith ' +
            'and his phone number is 8 6 7 5 3 O 9',
        'contact': 'The emergency contact\'s name is Peter Smith ' +
            'and his phone number is 8 6 7 5 3 O 9',
        'full history': 'jane doe is a 64 year old female, born august 22nd 1952,' +
            'She is 5 foot 4 inches and weighs 160 pounds,' +
            'She is allergic to latex and peanuts, Her blood type is O positive,' +
            'She is taking Verapamil 100 milligrams daily, She has a history of high blood pressure,' +
            'Her emergency contact\'s name is Peter Smith and his phone number is 8 6 7 5 3 O 9',
        'history': 'jane doe is a 64 year old female, born august 22nd 1952,' +
            'She is 5 foot 4 inches and weighs 160 pounds,' +
            'She is allergic to latex and peanuts, Her blood type is O positive,' +
            'She is taking Verapamil 100 milligrams daily, She has a history of high blood pressure,' +
            'Her emergency contact\'s name is Peter Smith and his phone number is 8 6 7 5 3 O 9',
        'medical profile': 'jane doe is a 64 year old female, born august 22nd 1952,' +
            'She is 5 foot 4 inches and weighs 160 pounds,' +
            'She is allergic to latex and peanuts, Her blood type is O positive,' +
            'She is taking Verapamil 100 milligrams daily, She has a history of high blood pressure,' +
            'Her emergency contact\'s name is Peter Smith and his phone number is 8 6 7 5 3 O 9',
        'profile': 'jane doe is a 64 year old female, born august 22nd 1952,' +
            'She is 5 foot 4 inches and weighs 160 pounds,' +
            'She is allergic to latex and peanuts, Her blood type is O positive,' +
            'She is taking Verapamil 100 milligrams daily, She has a history of high blood pressure,' +
            'Her emergency contact\'s name is Peter Smith and his phone number is 8 6 7 5 3 O 9',
        'details': 'jane doe is a 64 year old female, born august 22nd 1952,' +
            'She is 5 foot 4 inches and weighs 160 pounds,' +
            'She is allergic to latex and peanuts, Her blood type is O positive,' +
            'She is taking Verapamil 100 milligrams daily, She has a history of high blood pressure,' +
            'Her emergency contact\'s name is Peter Smith and his phone number is 8 6 7 5 3 O 9',
    }
};