Hardware components | ||||||
 |
| × | 1 | |||
 |
| × | 1 | |||
Software apps and online services | ||||||
| ||||||
| ||||||
As soon as our team began working on the Alexa and LEGO challenge, we started brainstorming ideas for our voice enabled project. We thought to make something, which could help us in our everyday life, and we ended running in between two options:
- Create multiple smart switches for the makerspace at our school
- Create a system which will reward its users for completing the chores
We binned the idea of the smart switches, because we could never make it work, with the main reason for it being in the noise pollution of our lab. We could start the necessary equipment for our laser cutter, but could not stop it 😂.
So, we focused our attention on the chores project. Both at the lab and back at our homes we lack the incentives to keep both places in check, and for our parents, or teachers sometimes it is much easier to clean the place up by themselves. So we thought for a way to bring in the positive reinforcement in this tedious process of keeping the space organized.
The duo of Alexa and EV3 gives us a great backbone on which this project can be built. We needed to improve the user interaction with the following processes:
- Task delegation between multiple users
- Task oversight
- Visibility of the rewards for completion
- Fair rewarding process
Here is the diagram of our solution we came up with:
Here is the video of our working solution:
Let us delve deeper into the each process and describe what the challenges we faced in each one, and how we solved them.
Task delegationTo be able to delegate tasks comfortably user often tends to stick to the apps/methods they all have grown accustomed to. This can be either a to-do app, or the post it notes. We thought of a lot of different ways to keep track of the chores, but decided not to overcomplicate things and use the Alexa's built-in list functionality.
From the box, Alexa has two default lists, the shopping list and the to-do list. And custom skills can use them both, if you provide them with the right permissions.
We granted the permissions for the list access using the permission tab in Alexa developer console.
And started coding using node.js example we found on GitHub
From it we were able to figure out the way to sync the state of one list to our physical reward system. While building and testing the interaction model for our skill, we figured out that using the built-in lists bring so much inconvenience, while planning and delegating tasks, so we decided to start using separate lists for each user.
Immediately we started facing problems - apparently the skill cannot gain the access to the user-created lists, it just cannot find them. How we figured it out: we checked the getTodoListId function and tried to get all the list ids it could find. But all we were getting were the built-in lists ids, and none for the lists we created manually did not appear there. Apparently this is a security feature. Custom skill must create its own lists to be able to read/write to them. We created a CreateTripleListIntent which done all that work. Inside of it we leveraged
createList method of the ListManagementService class (more info can be found in the node.js ASK SDK docs )
const CreateTripleListHandler = {
canHandle(handlerInput) {
const request = handlerInput.requestEnvelope.request;
return request.type === 'IntentRequest' && request.intent.name === 'CreateTripleListIntent';
},
async handle(handlerInput) {
//getting list names from slots
var nameA = handlerInput.requestEnvelope.request.intent.slots.nameA.value;
var nameB = handlerInput.requestEnvelope.request.intent.slots.nameB.value;
var nameC = handlerInput.requestEnvelope.request.intent.slots.nameC.value;
const responseBuilder = handlerInput.responseBuilder;
const listClient = handlerInput.serviceClientFactory.getListManagementServiceClient(); //setting up ListManagementServiceClient for list creation
let speechOutput;
console.log('Starting CreateListIntent handler');
//Creating list objects with names from slots, and active statuses, meaning they are not archived
var listObjectA = {
'name': nameA + " list",
"state": "active"
}
//leveraging createlist method for each of the custom lists
listClient.createList(listObjectA);
var listObjectB = {
'name': nameB + " list",
"state": "active"
}
listClient.createList(listObjectB);
var listObjectC = {
'name': nameC + " list",
"state": "active"
}
listClient.createList(listObjectC);
speechOutput = `Your custom lists for ${nameA}, ${nameB} and ${nameC} have been created`;
const speechReprompt = 'Now you can populate it';
return responseBuilder
.speak(speechOutput)
.reprompt(speechReprompt)
.getResponse();
},
};This intent, along with its model:
{
"name": "CreateTripleListIntent",
"slots": [
{
"name": "nameA",
"type": "AMAZON.FirstName"
},
{
"name": "nameB",
"type": "AMAZON.FirstName"
},
{
"name": "nameC",
"type": "AMAZON.FirstName"
}
],
"samples": [
"Please create three lists one for {nameA} one for {nameB} and one for {nameC}",
"Create lists for {nameA} for {nameB} and for {nameC}",
"Create lists for {nameA} {nameB} and for {nameC}",
"Create list for {nameA} {nameB} and {nameC}"
]
}Successfully creates 3 lists, which later can be accessed by the skill, managed either by voice or an app.
Task oversightIn order to be able to oversight task easier, and get information about lists, we used two intents in our project:
- TopToDoIntent - fetches top to-do from the custom list with the tasks assigned to specific user
- CompleteTopToDoIntent - completes top to-do from the custom list with the tasks assigned to specific user
These intents were necessary, because of our beginners understanding of the Alexa ecosystem, and creating a subscription to list events was hard for us (ASK CLI scared us away😂).
Both of them have its handlers, which leverage their own functions here is the code structure for both of them:
As you can see, each handler uses its specific function, but both of them are dependent of the getToDoListId function, which gets the name of the list from the HandlerInput object passed from the handler.
async function getToDoListId(handlerInput) {
// check session attributes to see if it has already been fetched
const attributesManager = handlerInput.attributesManager;
const sessionAttributes = attributesManager.getSessionAttributes();
const name = handlerInput.requestEnvelope.request.intent.slots.name.value;
let listId;
console.log(`Name of the list should contain: ${name}`)
if (!sessionAttributes.todoListId) {
// lookup the id for the list
const listClient = handlerInput.serviceClientFactory.getListManagementServiceClient();
const listOfLists = await listClient.getListsMetadata();
if (!listOfLists) {
console.log('permissions are not defined');
return null;
}
for (let i = 0; i < listOfLists.lists.length; i += 1) {
console.log(`found ${listOfLists.lists[i].name} with id ${listOfLists.lists[i].listId}`);
const decodedListId = Buffer.from(listOfLists.lists[i].listId, 'base64').toString('utf8');
console.log(`decoded listId: ${decodedListId}`);
// The default lists (To-Do and Shopping List) list_id values are base-64 encoded strings with these formats:
// <Internal_identifier>-TASK for the to-do list
// <Internal_identifier>-SHOPPING_LIST for the shopping list
// Developers can base64 decode the list_id value and look for the specified string at the end. This string is constant and agnostic to localization.
if (listOfLists.lists[i].name.includes(name)) {
// here we check whenever name of the list contains the string we got from handlerInput
//and we stop at the first list matched
listId = listOfLists.lists[i].listId;
break;
}
}
}
attributesManager.setSessionAttributes(sessionAttributes);
console.log(JSON.stringify(handlerInput));
return listId;
}This function is the backbone of all the logic we implemented, because you need listId to perform actions with the list.
As for getTopToDoItem and completeTopToDoAction nothing really changed from the example, provided by Amazon. But we implemented some clever logic to check whenever all the tasks were completed or not.
For the getTopToDoItem we added the following in the async handle(handleInput) :
const itemName = await getTopToDoItem(handlerInput);
//thats where function called, we either get the name of the item, or
//listIsEmpty
if (!itemName) {
speechOutput = 'Alexa List permissions are missing. You can grant permissions within the Alexa app.';
const permissions = ['read::alexa:household:list'];
return responseBuilder
.speak(speechOutput)
.withAskForPermissionsConsentCard(permissions)
.getResponse();
} else if (itemName === listIsEmpty) {
//Here is our call to action,
//we form directive with 'move' type with the name of the user whose list is
//empty
speechOutput = 'Your todo list is empty, everything is complete, please collect your treat!';
const directive = Util.build(endpointId, NAMESPACE, NAME_CONTROL,
{
type: 'move',
name: name
});
return responseBuilder
.speak(speechOutput)
.addDirective(directive)
.getResponse();
}
speechOutput = `Your top to do for ${name} is ${itemName}. To mark it as complete, say complete ${name} to do.`;So, we send the name of the person who have completed all the chores to the Alexa Gadget, if the list is empty.
Logic in the completeTopToDoAction is the same, except the code also checks before completing the task, if it is the last, uncompleted task in the list, and sends the directive to the Alexa Gadget.
In order to motivate users to complete their chores, we've chosen M&M's, we could have used nuts, or other treats, but we have some severe cases of nut allergy at our school, so chocolate M&M's were the only safe solution.
We've created a system with three hoppers, which share a loading chute which displays proudly its contents to everyone around.
Each chute incorporates a single large ev3 motor, which can be opened or closed either to free up the candies or close the way for them.
There are also three boxes, where treats are being directed. All of them are named, so users can figure out which one is theirs.
By the way, building instructions are attached to the project.
Fair rewarding processNow let's get close to the code located on the EV3, its basically a adaptation of the code, you can find in the mission03 but with a twist.
Our solution should be able to track the completion of the to-do lists for three of the different users. Motor, connected to the port A should correspond with the first custom list, Motor B should correspond to the second custom list, and Motor C should correspond to the last list. And that should be persistent between sessions. We could have created a database, hosted at the AWS Lambda, but we took a different approach.
During the creation of the three lists with the CreateTripleListHandler we decided to send the payload with the 'setup' type, which contained the names of the users. And then we wrote these names into the names.ini file. Here is the CreateTripleListHandler part of the code, which is added to the end of the handler:
const endpointId = attributesManager.getSessionAttributes().endpointId || [];
//get the endpointId to send the payload to the gadget
const directive = Util.build(endpointId, NAMESPACE, NAME_CONTROL,
{
type: 'setup',
nameA: nameA,
nameB: nameB,
nameC: nameC
});
//build the payload
return responseBuilder
.speak(speechOutput)
.reprompt(speechReprompt)
.addDirective(directive)
.getResponse();
},
//build the responseAnd part of the code which runs on the EV3:
payload = json.loads(directive.payload.decode("utf-8"))
print("Control payload: {}".format(payload), file=sys.stderr)
control_type = payload["type"]
if control_type == "move":
self._move(payload["name"]) #move motors if the payload type is "move"
if control_type == "setup": #form a dict with the names if the type is "setup"
nameA = payload["nameA"]
nameB = payload["nameB"]
nameC = payload["nameC"]
logger.info("got names: {}, {}, {}".format(nameA, nameB, nameC))
namesDict = {'nameA':nameA, 'nameB':nameB, 'nameC':nameC}
with open('names.ini', 'w') as f:
print(namesDict, file=f)
f.close()This enables us to keep the process simple enough, and still achieve persistency between the runs, as the gadget changes the file only when the setup is performed.
The robot chooses which hopper to open when the directive "move" arrives. To do so, the data from names.ini file is loaded back into the dictionary with the ast module, after that its the simple matter of the if statements. Here is the code:
def _move(self, value):
"""
Handles move commands from the directive.
Picks right motor from directive to dump the candies into chosen hopper
"""
print("Move command: ({})".format(value), file=sys.stderr)
with open('names.ini', 'r') as f:
readNamesDict = ast.literal_eval(f.readline()) #getting names dictionary from the file #getting the dictionary out of the text file requires literal_eval function
logger.info ("type of readNamesDict is {} and the value is {}".format(type(readNamesDict), readNamesDict))
f.close()
if value in readNamesDict.values():
logger.info("{} with type {}".format(readNamesDict,type(readNamesDict)))
for k, v in readNamesDict.items():
if v == value:
motorSelect = k
logger.info ("we need to open bay for {}".format(motorSelect))
if motorSelect == "nameA":
self.motorA.on_for_degrees(SpeedPercent(75),-90, brake=True, block=True)
time.sleep(0.1)
self.motorA.on_for_degrees(SpeedPercent(75), 90, brake=True, block=True)
elif motorSelect == "nameB":
self.motorB.on_for_degrees(SpeedPercent(75),-90, brake=True, block=True)
time.sleep(0.1)
self.motorB.on_for_degrees(SpeedPercent(75), 90, brake=True, block=True)
else :
self.motorC.on_for_degrees(SpeedPercent(75),-90, brake=True, block=True)
time.sleep(0.1)
self.motorC.on_for_degrees(SpeedPercent(75), 90, brake=True, block=True)So, basically that's it.
Project was made by: Benjamin Gentis, student of Moscow Economical School.
Teacher, consultant: Nikolay Beliovsky.
Big thanks to Nathan Finch for his Aussie accent and his agreement to help with the video.
Nikolay Beliovsky
Comments