Alexa Team - How Multi Alexas Work Together As a Team

Vision

The core idea of our solution is different from regular skills where user launch a skill and a session ends by either information feedback or an action executed.

Our focus is to show Alexa power, not only to use Alexa as a trigger for another service or solution.

We are using more than one Alexa skill to exchange information with each other through different Alexa devices, as one skill is available for a client to submit his request(s) with all features required to the service provider, on other hand another skill will be used by a service provider to scan all requests from different clients and be able to answer each request, then client will use his skill to check for feedback on his request(s).

The Idea

As proof of concept we used only 2 skills in this project. We have chosen the theme of doctors and their patients in our solution to present the core idea.

A patient will use Remedy Help skill to submit his problem with required specific details according to problem category, it will be then stored as request for an advice and waiting for a doctor feedback.

A doctor will use Remedy Line skill to look up requests relevant to his specialization (category) , Alexa will scan requests one by one to let doctor listen to details about the request and doctor will be able to recommend a medicine or an analysis required.

Patient will then check for answered requests and can listen to doctor recommendation to take action.

Voice User Interface Diagram

Demo Scenario

Skill, Lambda function & Database

Two skills have been created:

• Remedy Help Skill: patient use this skill to report a new inquiry. In case if user has an old inquiry, Alexa give an update about the inquiry status. Update can be the doctor feedback if doctor check the case and give his advise. If no feedback from the doctor yet, Alexa ask user to revisit again. This skill trigger Lambda function (MedicalConsultant.js) to handle user interactions.

• Remedy Line Skill: doctor use this skill to listen to reported inquiries and give his opinion about each case. This skill trigger Lambda function (MedicalInquiry.js) to handle user interactions.

Those two skills communicate with each other through DynamoDB table (inquiries table). help skill add record for patient inquiry in DB and line skill update this record with doctor feedback.

Amazon avail easy tutorials to create skill, lambda function & Dynamo table

• To create Skill, Lambda function and trigger lambda using the skill follow steps in trivia skill tutorial

• To create new table in DynamoDB, follow steps in this tutorial

• To give access for DynamoDB table from Lambda functions. you need to update IAM role (that has been chosen in Lambda function configuration page).

Remedy Help Skill

Skill allow patient to add only one inquiry. To add new one, doctor should be answered old inquiry and patient listen to doctor feedback.

This done by checking "inquiries" table to see if this user has an old inquiry or not and what is the status of "doctorFeedback" field in table if record exist.

if( doctorFeedback === " " ) 
   { 
       //scenario#3: old unasnwered inquiry 
       prepareUnansweredInquiry(callback); 
   } 
   if( doctorFeedback === "" ) 
   { 
       //scenario#1: old unasnwered inquiry 
       preparePatientNameRequest(false , session, callback); 
   } 
   else 
   { 
       //scenario#2: Answered inquiry
       deleteInquiryFromTable(context , session.user.userId, 
           function () { 
                   context.done(prepareStatusMessages( doctorFeedback , session, callback)); 
           }); 
   }  

So we have 3 scenarios

• Patient add new inquiry

Alexa: "Dear Patient, what is your first name ? "

Patient: "My name is Mohamed"

Alexa: "What is your age?"

Patient: "40 years old"

Alexa: "Since when you feel this pain? "

Patient: "since 20 minutes"

Alexa: "which area you feel pain in: bones or stomach?"

Patient: "In my bones."

Alexa: "How many joints you have bain in? multiple joints or one joint ."

Patient: "multiple joints."

Alexa: "When pain feel worst: at rest or at night? "

Patient: "at night."

Alexa: "Your case is recorded. Doctor feedback will be delivered in 24 hours."

Different questions will be asked if user choose the 2nd category (stomach). Patient case will be saved in DynamoDB so that advisor can check it from Remedy line skill.

• Patient listen to doctor feedback

Alexa: "Hi Mohamed, Have a rest. Have a healthy life. see you soon! bye. "

Skill read the advisor recommendation for user inquiry from the DynamoDB

• Inquiry still not answered by doctor

Alexa : "Dear Patient, You have unanswered inquiry. Please check back with Alexa later to check Doctor feedback. Have a healthy life. see you soon! bye. "

Remedy Line Skill

Advisor use this skill to listen to pending inquiries. Advisor give his recommended drug & analysis.

Advisor: "Alexa, launch Remedy support"

Alexa: "Dear advisor, to check pending inquiries, please choose category"

Advisor: "Bones"

Alexa: "Patient detailes is ..." "Which medicine do you advice?"

Advisor: "Asprine"

Alexa: "Which analysis do you advice? "

Advisor: "bone density"

Alexa: "Advice is recorded. Have a healthy life. Good bye! "

Advisor feedback will be saved in DynamoDB so that patient can check this recommendation later using Remedy help skill

Tips & Tricks

• Speech Synthesis Markup Language (SSML)

Alexa automatically handles normal punctuation, such as pausing after a period, or speaking a sentence ending in a question mark as a question. In some cases this not enough: For example we want to add long pauses if sentence contains options to user to select from, or in case of help when Alexa generate speech for many sentences. This is why we need SSML to to mark up text for the generation of synthetic speech.

Example of Help sentence

var speechOutput = "<speak>This skill make a connection between patient and Doctor. <break time=\"0.2s\" />" 
   + "Patient can add new  inquiry. <break time=\"0.2s\" />"  
   + "or check doctor feedback for the his old inquiry. <break time=\"0.2s\" />"  
   + "In case of new inquiry, <break time=\"0.2s\" /> patient should answer questions related to his age, <break time=\"0.2s\" /> when he felt this pain" 
   + "<break time=\"0.2s\" /> and based on category of pain, he sould answer some other questions." ; 
   + "<break time=\"0.3s\" /> Please say menu to start?</speak>"; 

Example of giving user options to select form.

speechOutput = "<speak> How many joints you have bain in? <break time=\"0.2s\" /> multiple joints <break time=\"0.1s\" /> or one joint. </speak>"; 

• Session attributes concept

During the whole session, lambda function gives you the ability to maintain the same session values during all requests in single session. You can add new attributes to session object to keep track of those variables. Please note that Global variables can't be used to pass values between different requests in the same session.

session.attributes.patientName = "Patient name is " +  patientName + " . "; 
sessionAttributes = { 
       "speechOutput": speechOutput, 
       "repromptText": speechOutput, 
       "patientName"   : session.attributes.patientName , 
       "patientCase"   : session.attributes.patientCase 
       }; 
callback(sessionAttributes, 
    buildSpeechletResponse(CARD_TITLE, speechOutput, speechOutput, shouldEndSession));

• DB synchronous & asynchronous concept

By default Lambda function is event based, so all Database actions is asynchronous. This can lead to many troubles if you understand this concept. For example, successive commands after Database read will be executed before the database return with the read value. if successive commands depend on the read value, then code will be crashed.

To handle this case, all commands that depend on database action should be handled in database callback function. As in below example the handleFinishSessionRequest function called inside the callback function of insertNewInquiryInTable

insertNewInquiryInTable(context , session , function(){ 
           console.log("inside Call back of insertNewInquiryInTable."); 
           context.succeed(handleFinishSessionRequest("<speak>Your case is recorded. Doctor feedback will be delivered in 24 hours.</speak>", session, callback)); 
       }); 

Testing & Certification

Both skills have been certified. You can access them from those links: Remedy Support & Remedy help.

Please take care from below points, maybe it will speed up the process of testing & certifying your new skill :)

• Skill names was "Medical Consultant" & "Medical Inquiries", while certification process we have an issue related to the name of skill and we have to state clearly that this skill for educational use only. Below sentence is added to ensure that users are understand that those skills for contests purpose

"This tool does not provide medical advice, and is for informational and educational purposes only, and is not a substitute for professional medical advice, treatment or diagnosis. Call your doctor to receive medical advice. If you think you may have a medical emergency, please dial 911.”

• If your skill gather information related to customer then you should "privacy policy" & "term of use" URLs. You can add those URLs in "Privacy & Compliance" tab in skill definition.

• Use Alexa skill testing tool, it will save your time while testing. This should be used with AWS Cloudwatch to check logs and ensure logic is run correctly.

• Download "Amazon Alexa" application will save your time while testing. This application will display the dialog between Alexa & user that is currently happen.

• As this skill gather some information about user, a valid privacy policy should be displayed in Alexa companion application.

• For testing purpose and to test certain intent, in Lamdba function click on "Actions -> configure test event" and from "sample event template" drop down menu choose, "Alexa intent, Get new fact". Now you can make few changes to test certain intent with certain slot values.

• if Alexa device isn't available in you country like us, you still can test your skill and have the experience using echosim.io . However we recommend to access this website using firefox web browser.

• Alexa do a good job in English speech recognition even if the speaker isn't native English like use.

Future Steps

We need to build notification system so that advisor can know that there is a pending inquiry waiting for him to answer, and the same is needed for patient to listen to advisor feedback.

Build on our vision, and develop larger systems depend on more than 2 Alexa.

It will be very beneficial if Amazon add below features to Alexa

• Alexa return not only the slots but also the whole spoken sentence.

• Protocol that skill can notify user that he needs to communicate with Alexa and not only wait for him to check randomly.