Hardware components | ||||||
 |
| × | 1 | |||
 |
| × | 1 | |||
In my free time I love to build robotic animals with LEGO. When I build an animial my goal is to make the experience as natural as possible. Until now I never build a LEGO dog, since you use voice commands to interact with a dog. Using Alexa it is finally possible to create an intuitive voice interaction with a LEGO dog.
To get the full dog owner experience I decided to turn the experience into a puppy training. When you buy a puppy she still has to learn the basic commands: sit, down and stand. Using an Alexa device as a puppy trainer you can train the puppy such that it understands these commands. This experience is demonstrated in this video:
The puppy training experienceWhen you adopt a puppy you have to teach your puppy everything. To do this you often get help from puppy trainers. This is the experience I want to create with this project. The commands you can teach your puppy are sit, down, and stand. The position of the Lego puppy are:
Cool tip: In this project the puppy is called Alexa, since this is the wake word of the Alexa device used. You can change the wake word of the Alexa device to rename your puppy.
Starting the puppy training
You can start the puppy training by saying:
- "Alexa, start puppy training"
The Alexa device will respond with
- "Welcome to the puppy training, you can teach your puppy the commands: sit, down and stand. Start by saying a command."
The puppy training
You can give the following commands to your puppy:
- "Alexa, sit"
- "Alexa, down"
- "Alexa, stand"
If you issue a command that is different, the Alexa device will respond with:
- "Your puppy can only understand the commands: sit, down and stand"
The puppy will respond to the commands by performing the action she thinks she has to do.
When the puppy is very young she does not understand the commands. Hence, the action she performs is not always correct. Sometimes she does the correct command. To teach your puppy what is correct and what is incorrect you can give your puppy feedback:
- When the puppy does something good:
You can pet the puppy on her back. Using a Mindstorms touch sensor this interaction with the environment is possible. Next time you say this command she will know what to do to get another pet on her back. - When the puppy is doing something incorrect:
Say: "Alexa, no". Next time you say this command she will know that this was not the correct thing and she will try something else.
When you keep on training, the dog will learn all the commands perfectly.
The dog trainer
During a training the Alexa device will help you like a real dog trainer. The Alexa device will exactly tell the new dog owner what he/she should do to teach his puppy new commands.
After the dog finished responding to a command the Alexa device will respond by telling you how to give feedback:
When the puppy is doing something good, the Alexa device will respond with one of the following sentences:
- 'Well done, praise your puppy by petting her back',
- 'Very good, she deserves a pet on her back',
- 'Good girl, that is worth a big cuddle'
When the puppy is doing something incorrect, the Alexa device will respond with one of the following sentences:
- 'Hm, still a lot to learn, correct your puppy by saying: no',
- 'That is not correct, tell her by saying: no',
- 'Almost there, correct her by saying: no'
When you gave feedback to your puppy by petting her back or saying 'Alexa, no', the Alexa device will give you a new task:
When you just petted your puppy after a doing something correct, the Alexa device will respond with one of the following sentences:
- 'Try a new command, you can teach the commands: sit, down, and stand',
- 'Try this command again to check if she really understands the command',
- 'Good job, try another command'
When you just said 'Alexa, no', the Alexa device will respond with one of the following sentences:
- 'Try the same command again, maybe she understands it this time',
- 'Maybe another command will help her understand',
- 'Try again, she will get there'
I choose to implement multiple responses from the dog trainer for each of the possibilities. This keeps the conversation with the Alexa device very intuitive. Moreover, a real dog trainer also does not say the same sentence each time.
At the end of a training
After a couple of minutes the training is finished, the Alexa device will inform you by saying:
- 'The puppy training ended. Goodbye.'
When your puppy understands all the commands, but you want to keep on training, you do not have to buy a new puppy. Simply say "Alexa, restart puppy training". Your puppy will forget all the commands and you can train them all from the start!
Building the Lego puppyThe puppy is completely build out of Lego. Since I understand that it is hard to get all the correct pieces to build an exact replica of this puppy, I decided to make a puppy that has the same mechanical functions as the Lego Puppy from the Lego Education set 45544. Hence, it is possible for everyone with this set to build a Lego Puppy that you can train using Alexa. The exact same program can be used for this puppy.
The mechanical functions are:
- Two large motors (one for each of the hind legs)
These motors allow the hind legs to move to get to each of the positions: sit, stand and down - One medium motor
This motor can move the head - One touch sensor on the back
There is a hinge in the back of the puppy, when pressing this back down (petting) the touch sensor will be pressed - A color sensor:
For this project this sensor is only used for manually calibrating the position of the head (since the buttons of the EV3 cannot be pressed due to the dogs nose)
The building instructions for the Puppy from the LEGO 45544 can be found here:
You can decorate this puppy in any color you like to match your favorite dog.
For everyone who wants to make an exact replica of the puppy I build, some detailed photo's:
To make the puppy perform commands some coding is necessary. The full code can be found in the puppy_training.py file in the attachment of this project. Next I will explain the most important parts:
Learning a puppy commands
In the initialization of the robot three arrays are created. Each array corresponds to a command (sit, down, or stand). In each array the poses are stored that the puppy relates to the command. In the beginning the puppy does not know any commands, hence each array is filled with all three positions (sit, down, stand):
# arrays of poses connected to commands: initially puppy connects all poses to each command
self.Command_Sit = ['sit', 'stand', 'down']
self.Command_Stand = ['sit', 'stand', 'down']
self.Command_Down = ['sit', 'stand', 'down']When the puppy receives a command from the Alexa device, the puppy will randomly choose a position from the array corresponding to the command received. This is achieved using the following code:
def Command(self,command):
self.command = command
new_pose = self.pose
if command in POSE.STAND:
new_pose = random.choice(self.Command_Stand) # Pick random pose out of list stand
elif command in POSE.SIT:
new_pose = random.choice(self.Command_Sit) # Pick random pose out of list sit
elif command in POSE.DOWN:
new_pose = random.choice(self.Command_Down) # Pick random pose out of list down
# Perform action to new pose
self.Action_to_Pose(new_pose)At the start of the training this random action can be any position, but by repeating the commands and giving feedback the array will change to only contain the correct pose. This is done using the following:
Negative feedback: remove the current pose from the array, next time the command is given the puppy will do another pose. This achieved using the code:
def Negative_Feedback(self):
# If negative feedback is received ('Alexa, no')
# Remove the current pose from the list corresponding to the command
# This way this pose will not be performed when the command is given
if self.command == POSE.SIT:
self.Command_Sit.remove(self.pose)
print(self.Command_Sit)
elif self.command == POSE.STAND:
self.Command_Stand.remove(self.pose)
print(self.Command_Stand)
elif self.command == POSE.DOWN:
self.Command_Down.remove(self.pose)
print(self.Command_Down)
#Send message to Alexa to indicate negative feedback received
self._send_event(EventName.FEEDBACK, {'Feedback': "negative"})Positive feedback: if a pet on the back is given the puppy knows this is the correct pose. The array corresponding to this command is replaced by the current position. Next time the command is given the puppy will immediately move to the correct pose.
def Positive_Feedback(self):
# If positive feedback is given (Pet on back)
# Make the list corresponding to the current command equivalent to current pose
# This way next time the command is given the current pose is performed
if self.command == POSE.SIT:
self.Command_Sit=[self.pose]
print(self.Command_Sit)
elif self.command == POSE.STAND:
self.Command_Stand=[self.pose]
print(self.Command_Stand)
elif self.command == POSE.DOWN:
self.Command_Down=[self.pose]
print(self.Command_Down)
# Send message to alexa to indicate positive feedback
self._send_event(EventName.FEEDBACK, {'Feedback': "positive"})Notice that after giving positive or negative feedback a message is send to the Alexa device. By doing this the Alexa device can respond with a new task during the training.
Receiving Commands from Alexa Device
When a message from the Alexa device is received the following code is executed:
def on_custom_mindstorms_gadget_control(self, directive):
"""
Handles the Custom.Mindstorms.Gadget control directive.
:param directive: the custom directive with the matching namespace and name
"""
try:
payload = json.loads(directive.payload.decode("utf-8"))
print("Control payload: {}".format(payload), file=sys.stderr)
control_type = payload["type"]
if control_type == "command":
# Expected params: [command]
self._activate(payload["command"])
if control_type == "restart":
self.Command_Sit = ['sit', 'stand', 'down']
self.Command_Stand = ['sit', 'stand', 'down']
self.Command_Down = ['sit', 'stand', 'down']
print("learned behaviour is resetted")If a restart message is received all the arrays with poses are reset to the initial values and you have the train the puppy form the start.
If a command is received the following code is executed:
def _activate(self, command):
"""
Handles preset commands.
:param command: the preset command
:param speed: the speed if applicable
"""
print("Command: ({})".format(command), file=sys.stderr)
if command not in Command.NO.value:
# perform command
if command in Command.SIT.value:
self.Command(POSE.SIT)
if command in Command.DOWN.value:
self.Command(POSE.DOWN)
if command in Command.STAND.value:
self.Command(POSE.STAND)
self._send_event(EventName.POSE_FINISHED, {"NewPose": self.pose, "Command": self.command})
else: # if command is no: give negative feedback
self.Negative_Feedback()If the command is no, the negative feedback code is executed
If a command sit, stand or down is given the command code given above is executed and a new pose from the array is selected. After moving to the new pose a message is send to the Alexa device with the current pose and command. By doing this the Alexa device can respond and tell you to respond with negative or positive feedback.
Moving to a new pose
The following code is used to move from the current pose to the next:
def Action_to_Pose(self,command):
#Choose actions according to current pose and new pose
if self.pose in POSE.SIT and command in POSE.STAND:
self.Action_Sit2Stand()
elif self.pose in POSE.STAND and command in POSE.DOWN:
self.Action_Stand2Down()
elif self.pose in POSE.DOWN and command in POSE.STAND:
self.Action_Down2Stand()
elif self.pose in POSE.STAND and command in POSE.SIT:
self.Action_Stand2Sit()
elif self.pose in POSE.SIT and command in POSE.DOWN:
self.Action_Sit2Stand()
self.Action_Stand2Down()
elif self.pose in POSE.DOWN and command in POSE.SIT:
self.Action_Down2Stand()
self.Action_Stand2Sit()This code detects the current pose, and determines the correct actions to go to the new pose.
The code to go from one position is pretty simple. The code to go from stand to down is given by:
def Action_Stand2Down(self):
#goto down position
self.lleg.run_to_abs_pos(position_sp=-140,speed_sp=-70,stop_action="brake") #left leg
self.rleg.run_to_abs_pos(position_sp=-140,speed_sp=-70,stop_action="brake") # right leg
# change current pose to down
self.pose = POSE.DOWNThe other position changes can be found in the puppy_training.py file in the attachment of this project.
Creating the puppy trainingThe code taking care of the puppy training can be found in the index.js file in the attachment of this project. The code is based on the code of mission-04 of this contest.
Next i will explain the most important parts that create the puppy training.
The sentences that are used as a response during the training are stored in arrays:
const Positive_Behavior =
['Well done, praise your puppy by petting her back',
'Very good, she deserves a pet on her back',
'Good girl, that is worth a big cuddle']
const Negative_Behavior =
['Hm, still a lot to learn, correct your puppy by saying: no',
'That is not correct, tell her by saying: no',
'Almost there, correct her by saying: no']
const After_Positive_Feedback =
['Try a new command, you can teach the commands: sit, down, and stand',
'Try this command again to check if she really understands the command',
'Good job, try another command']
const After_Negative_Feedback =
['Try the same command again, maybe she understands it this time',
'Maybe another command will help her understand',
'Try again, she will get there']when a message from the EV3 is received the following code takes care of the responses of the Alexa device:
let customEvent = handlerInput.requestEnvelope.request.events[0];
let payload = customEvent.payload;
let name = customEvent.header.name;
let speechOutput = ''
if (name === 'Pose_Finished'){
let NewPose = payload.NewPose;
let Command = payload.Command;
if (NewPose === Command){
speechOutput = Positive_Behavior[Math.floor((Math.random() * 3))];
}
else{
speechOutput = Negative_Behavior[Math.floor((Math.random() * 3))];
}
} else if (name === 'Feedback'){
let feedback = payload.Feedback;
if (feedback === 'negative'){
speechOutput = After_Negative_Feedback[Math.floor((Math.random() * 3))];
} else{
speechOutput = After_Positive_Feedback[Math.floor((Math.random() * 3))];
}This code selects the right response type and picks a random sentence out of the arrays defined above. This random sentence ensures the conversation is unpredictable and therefore more natural.
SummaryTo summarize this project contains all elements to create an intuitive Lego puppy training with Alexa. If you want to create this experience yourself you can start by making the puppy from the Lego Education 45544 set and decorate it to look like your favorite dog. All the code necessary for this project is included and some code snippets are explained. So lets get started!
Comments