Give It a WHIRL

The dream of having advanced robots that effortlessly handle all our household chores has long captivated our imagination. From sci-fi movies to ambitious technological aspirations, we often envision a world where humanoid robots efficiently clean, cook, and organize our homes, allowing us to enjoy more leisure time. However, the reality of the robotic assistants we actually have in our homes is quite different from this idealized dream. At present, we only have a limited range of such devices available, and their capabilities are still relatively basic.

While there have been significant advancements in robotics, the practical implementation of sophisticated household robots is still in its early stages. One of the biggest challenges is teaching a robot to navigate and operate in the largely unstructured environment of a home, where the specifics of a task greatly differ from home to home. Stirring a pot on the oven, for example, is deceptively difficult, with different utensils, appliances, and kitchen layouts present in each case.

Machine learning-based approaches are typically taken to help robots learn to generalize across a wide range of conditions. Broadly speaking, these approaches commonly fall under the umbrella of reinforcement learning or imitation. Reinforcement learning is a method by which robots learn through trial and error. This often requires running millions of simulations to train the model, and frequently has problems translating to real-world, unstructured environments. With imitation, robots learn by watching the task be performed by humans, but this method also requires many examples.

The dream of household robots may be one step closer thanks to the work of a team of engineers at Carnegie Mellon University. They have developed a new learning method for robots, called WHIRL, that teaches robots to imitate humans performing household tasks by watching only a single example.

An off-the-shelf robot was outfitted with a camera and the WHIRL software. This robot then watched a human perform a task, like opening a drawer, or turning on a faucet, a single time. It uses the information it gathered to gain a rough approximation of what steps it needs to take to reproduce the human’s actions. These steps are attempted, and the results are assessed. Based on this additional information, the approach is refined and another attempt is made. In this way, the robot iteratively improves its approach until it can perform the task as it was demonstrated.

Using the WHIRL system, the team’s robot was shown how to perform 20 common household tasks, like opening and closing appliances, cabinet doors and drawers, and taking a garbage bag out of the bin. In each case, after a single demonstration, the robot would begin practicing the task on its own. After about an hour or two, the robot had mastered each task well enough to perform it much like the human did.

These techniques not only enable a robot to learn to perform a task in different environments, but also to learn new tasks that were not even considered by the manufacturers of the robotic assistant. It has also been shown that WHIRL is scalable and can operate in realistic home environments — not just under idealized laboratory conditions.

It should be noted that all of the tasks WHIRL learned in this study were quite simple, like opening a drawer or erasing a whiteboard. So, a robot that can cook you dinner may still be a long way off, but this work represents a significant step in the right direction.