Stereo Dual-Sensor MEMS-Mirror Foveated Vision System

This project's goal is to implement FPGA/SoC design for real-time foveated camera and foveated FPGA-accelerated image processing pipeline implemented iTracker (MIT CSAIL), AlexNet-inspired CNN for foveation control by feed of Kalman filter and foveated two-stream (FVS) CNNs for video multi-frame summarization.

Foveation is a biologically motivated image transformation which has attracted interest of researchers and of computer vision and robotics. Its principal advantages are the realization of high degree of image compression as well as property of scale and rotation invariance.

Human eyes have fast mechanical movements that control how the scene is imaged in detail by the fovea, where visual acuity is highest. This concentrate computational(neuronal) resources in place where they are most needed.

Most conventional imaging techniques spread a finite number of pixel evenly across entire FOV and thus their capabilities are constrained by well-known inherent tradeoff between FOV and resolution: the higher resolving ability, the smaller FOV.

This resolution-FOV tradeoff problem may be relieved with advent of higher-resolution sensors and development of sensor tiling or image mosaicing schemes. However, processing and analysis of such high-resolution images is computationally-demanding. Foveated imaging system design aims to avoid expensive high resolution and fast image sensor as well as facilitate to improve the information throughput of low-cost off-the-shelf detectors.

IN human visual system (HVS), only narrow region around the fovea offer exceptional resolving power, contrast, and, color sensitivities, while these properties fall off rapidly with increasing retinal eccentricity. The essential goal of designing a foveated imaging system is to improve the sampling efficiency and to reduce the requirements for data bandwidth as well as computational resources by mimicking such spatially varying properties of the HVS.