Relive the Past in VR
MIT’s InteRecon brings real-world objects into VR/AR — complete with motion and function — turning memories into interactive 3D experiences.
Who says you can’t take it with you? Now you can — into virtual and augmented reality environments, that is. A team led by researchers at MIT CSAIL has developed a method that makes it possible to digitize objects from the real world and insert them into virtual environments. But that’s not all — the more interesting part of this work is that it also makes these objects interactive. For example, a virtual reconstruction of a childhood doll could be made to move its arms and legs like the real deal.
The new system, called InteRecon, lets users scan real-world items using a mobile app and then recreate not only the shape and appearance of the item, but also its interactive behaviors in a mixed-reality setting. That means users can preserve and relive memories with a level of detail and realism that traditional digital methods like photos and videos simply cannot offer.
To get started, users scan an object with an iPhone, moving around it three times to capture its full geometry. The resulting 3D model is then imported into a mixed-reality interface, accessible through devices like the HoloLens 2 or Meta Quest. From there, users can either manually mark parts of the model they want to make interactive — like the arms or head of a doll — or use InteRecon’s automatic segmentation feature to do it for them.
Next comes the animation stage. InteRecon provides a library of motion demonstrations that users can apply to selected parts of the model. For example, a bunny’s ears might be given a floppy, swinging motion, or a bobblehead’s neck might be programmed to nod and swivel. These movements can be previewed and fine-tuned, giving users a lot of control over how the object will behave in the virtual world.
InteRecon also supports recreating the functional interfaces of vintage electronics and other complex devices. Users can add virtual widgets like knobs, sliders, screens, and buttons to 3D models — and even embed digital content, such as MP3s or videos, to mimic how the original item functioned. This means a virtual iPod could play songs, or a retro TV model could display old family videos, bringing back not just the object’s appearance but its personality.
Looking ahead, the team sees applications beyond personal memory preservation — including education, medical training, cultural exhibits, and even creative endeavors like fashion and design. Future updates aim to make the process more automated, expand its physical simulation accuracy, and maybe even integrate generative AI to recreate lost items from text descriptions.
By preserving the form and function of personal artifacts, InteRecon offers a new way to bridge the physical and digital worlds. And with some enhancements, the experiences could ultimately be used for a lot more valuable purposes than just a stroll down memory lane.
