This Little PIGI Stayed Home

Robots have demonstrated remarkable proficiency in executing repetitive tasks that involve following a predetermined set of instructions. Their ability to precisely and consistently carry out these instructions with speed and accuracy is one of their greatest strengths. Unlike humans, robots do not experience fatigue or loss of focus, making them highly reliable for tasks that require repetition and precision. This characteristic has made robots invaluable in industries such as manufacturing, assembly lines, and logistics, where consistency and efficiency are crucial.

However, when it comes to operating in unstructured environments, such as the dynamic and unpredictable setting of a typical home, robots face significant challenges. Unlike controlled industrial environments, homes are filled with a wide variety of objects, varying layouts, and changing circumstances. The absence of standardized interfaces and the unpredictable nature of household tasks make it difficult for robots to navigate and manipulate objects effectively. Simple tasks like folding laundry, tidying up a room, or even preparing a meal require complex perception, planning, and manipulation skills, which robots have yet to fully master.

One of the biggest problems facing household robots is the sheer number of possible actions that could be taken to accomplish a particular goal. It is extremely computationally expensive to evaluate all of these options to find the tiny set of options that may be useful. Researchers at MIT CSAIL have taken on this challenge and developed a solution that they call PIGINet (Plans, Images, Goal, and Initial facts). This method substantially cuts down on the time that traditional systems spend iteratively refining their motion plans from the set of all possible options, leading to time savings as great as 80%.

PIGINet is a transformer-based neural network that takes in a multimodal set of data — the task plan under consideration, images of the robot’s environment, and encoded representations of the robot’s present state and desired goal state. This information is combined to produce a measurement representing the feasibility of the task plan being considered. By quickly eliminating infeasible plans, PIGINet can rapidly settle on a solid plan for more common problems. And for novel problems, robots leveraging PIGINet can still fall back on more traditional approaches to get the job done.

One major challenge the team faced in developing PIGINet was the lack of suitable training datasets for the model. This is due to the fact that the set of plans, both feasible and infeasible, need to be produced by traditional task planning systems, which can be quite slow. But by leveraging pretrained vision language models and applying some clever data augmentation techniques, the researchers were able to clear this hurdle and continue development.

To test the system, hundreds of virtual kitchens were created in a simulated environment. Many different layouts were created, with a variety of tasks being performed in each, to replicate the variety that might be seen in real-world homes. PIGINet was compared with more traditional approaches to compare their performance. It was found that PIGINet resulted in as much as an 80% time savings for simpler tasks, while the time savings were reduced to about 20% to 50% for more complex tasks.

With everyone’s home being different — and the fact that a given home can change from day to day — the work done by this research group may prove to be pivotal in bringing about the dream of having a general-purpose robot in every home to provide assistance with all manner of mundane tasks, from cooking dinner to folding laundry. More details are available on GitHub, and the team notes that the source code will be released soon.