MILI Picks People Out of Even Low-Resolution Videos and Photos
Designed for real-world footage with multiple subjects, MILI can provide shape and 3D pose estimation from low-res 2D snaps.
Computer scientists from Tianjin University and Cardiff University have demonstrated an end-to-end multi-task machine learning framework dubbed Multi-person Inference from a Low-resolution Image, or MILI — and say it performs better than rivals for real-world footage.
"In both small-scale and large-scale scenes, MILI outperformed the state-of-the-art methods both quantitatively and qualitatively," claims Kun Li, lead author of the study of MILI's capabilities. "Different from the existing work, MILI, as an end-to-end network, encourages the multi-person reconstruction even from low-resolution images and significantly improves the robustness to occlusions with the occlusion-aware mask prediction network by refining the detection stage with segmentation."
The idea: a model which is capable of 3D pose estimation and shape isolation on the sort of images you'd find in the real world, where the people within them take up only a small number of pixels — either owing to low-resolution cameras or being a cropped portion of a bigger image. MILI, the researchers claim, delivers exactly that.
The MILI approach includes a restoration network, trained using pair-wise images at high and low resolution, and an occlusion-aware mask prediction network — which helps when there is more than one person in a given scene, preventing the model from becoming confused when one person moves over another in the footage.
"Reconstruction of 3D poses and shapes for the individuals in a surveillance scene will allow for better recognition of actions/activities, including the interaction between people, modeling crowd behavior for simulations and security monitoring, and better tracking of individuals over time."
The team's work has been published in the journal Fundamental Research under open-access terms, with more information available on the project website; while the researchers have promised to release the MILI source code in the near future, at the time of writing the project's GitHub repository was empty.