Blade Runner's Image Enhancement Tool Nears Reality with an Algorithm for Seeing Around Corners

Researchers from the University of South Florida have delivered on a classic of science fiction plotting: the ability to take an already-captured two-dimensional photograph and peer around corners or outside the picture's frame to find hidden objects.

"We're turning ordinary surfaces into mirrors to reveal regions, objects, and rooms that are outside our line of vision," John Murray-Bruce, assistant professor of computer science and corresponding author on the paper, explains of his work with student Robinson Czajkowski. "We live in a 3D world, so obtaining a more complete 3D picture of a scenario can be critical in a number of situations and applications."

What the pair have created is an image processing algorithm that, in part at least, delivers on an iconic scene from Ridley Scott's 1982 sci-fi film Blade Runner, based on a novel by Philip K. Dick: the titular Blade Runner Rick Deckard, played by Harrison Ford, scans a Polaroid photo into an image enhancement system that can pan, zoom, and — crucially — somehow peer around corners to reveal objects never visible in the original photograph.

At the time, the concept was outlandish — but, as with many technologies first dreamed of in science fiction from communications satellites to automatic doors, fiction has become reality. The secret: usually-unnoticeable shadows cast into the shot by things outside the frame. "These shadows are all around us," Czajkowski explains. "The fact we can’t see them with our naked eye doesn’t mean they’re not there."

By analyzing these shadows, the team's algorithm is able to take a two-dimensional image captured by an off-the-shelf digital camera and infer the shapes of rooms and objects casting these near-invisible shadows. In addition to helping a future Deckard hunt down replicants, the technology could provide additional safety in everything from self-driving vehicles to robotics — though its creators warn it could be a decade or two before it's robust enough for those kind of applications.

The team's work has been published under open-access terms in the journal Nature Communications, while MATLAB scripts for reproduction have been published to GitHub under an unspecified license.

Main article image courtesy of Cassidy Delamarter/University of South Florida.