The Most Automated Place on Earth

Pay no attention to the man behind the curtain! If you do look, you will only spoil the fun. This wisdom holds true not only for Wonderful Wizards, but also for suspiciously interactive and emotive robots at certain theme parks. Yes, it is true. The cat is out of the bag. These lifelike robots that roam around to greet park visitors are as big of a phony as Oz the Great and Powerful himself. A human operator, hidden from view, is really pulling all the strings.

But keep in mind that Disney does have some experience in the area of bringing little wooden boys with strings to life. It has been a while since it was last used, but the team at Disney Research is dusting off this special magic once again to similarly bring their robots to life. The old formula of being brave, truthful, and unselfish toward a fairy apparently failed them this time, so they turned to the next best thing — artificial intelligence (AI). And with a clever algorithm design, they proved that fairies have got nothing on a GPU. Who would have thought fairytale creatures would be among the first to lose their jobs to AI? That certainly was not on my bingo card.

Human operators do a great job of giving robots highly expressive behaviors, so the team sought to replicate their work. The key to their approach lies in a clever training pipeline — instead of laboriously coding every interaction, they trained a transformer-based AI model to mimic the actions and social cues demonstrated by expert operators.

The pipeline starts with a data collection process that involves teleoperated robots interacting with humans. These robots, controlled by a skilled operator using a gamepad interface, engage in a variety of emotionally expressive behaviors — following a guest while acting shy, shaking their heads in mock anger, or breaking into a joyful dance. During these sessions, the positions and movements of both robot and human are precisely tracked using motion capture technology. Simultaneously, both continuous joystick inputs and discrete button commands from the operator are recorded, creating a richly annotated dataset.

The model architecture is built on a transformer backbone and leverages a diffusion model for continuous command prediction. Diffusion models, often used in image generation, here serve to predict fluid and expressive analog control signals — like joystick movements. Meanwhile, discrete commands such as behavior triggers and mode switches are handled via auxiliary classification heads, all trained within the same transformer network.

To allow for real-time responsiveness to humans, the AI is conditioned on robot-relative human pose data, enabling it to interpret proximity and orientation without needing full environmental awareness. Clever preprocessing steps — like augmenting human pose data to account for varying heights, and applying post-encoding masking to handle zero-value signals — make the model robust to noise and ambiguity.

With less than an hour of training data, the resulting system can produce realistic and emotionally diverse interactions. In user studies, park guests reportedly struggled to tell whether they were engaging with a robot under human control or an AI-powered automaton. Even more impressively, the trained model demonstrated zero-shot transferability, functioning flawlessly on a completely different robotic platform that shared the same control interface.

Soon, the most magical place on Earth may actually become the most automated place on Earth — but you will not be able to tell the difference.