Monocular 3D scene generation for autonomous navigation

Introduction

The use of autonomous vehicles for people or packages transportation is growing really fast. Solutions for those cases are available today and include the most common technologies widely used in the Machine Vision and Artificial intelligence fields, such as CPU and GPU. Those solutions are low cost, but power hungry ones.

There are other applications for autonomous vehicles and autonomous navigation explorers, such as deep under sea, caves and extra-terrestrial exploration that has constrains that, for instance, a car or a truck don't have.

Some contains for autonomous small rovers and explorers are low power consumption, extreme temperature conditions, low cost, among others.

In this project we use all the power from Kria to deliver a low cost, low power consumption and computational fast system capable of producing 3D depth maps, used by navigation systems.

Fig. 1 shows a System Overview. A global shutter camera or a video stream feed the Depth Map Generator (DMP). Depth Maps are 2D images, where each pixel represents the depths from the camera (observer) and the region captured by that pixel.

In this project, we will cover the hardware acceleration of some DMG components using Kria SoM.

Depth Map Generator

Depth map generation requires that the camera move around the scene, this means that every frame taken is different from the previous one, as the camera moves.

In order to estimate the pose of each frame in the scene, current frame and a reference frame feed the pose estimator, as shown in Fig. 2.

Pose Estimator feed the Depth Estimator, that finally generates de Depth Map.

For this project, Pose Estimator is accelerated in hardware, while Depth Estimator runs in software.

Pose Estimator HW Acceleration

Please, follow the link to https://gitlab.com/aairabella/3d-scene-generator in order to checkout the running code in CPU and Accelerated Hardware.

Standard hardware implementation

Please, look at https://circuithub.com/projects/aairabella/kria-pci104e-carrier/revisions/41026/project for updates about the carrier board implemented for this project.

Future work

In the future, we will develop a small rover capable of autonomous 3D mapping and movement, with very low power and high efficiency operation, capable of exploring caves and other surfaces.