Reinforced AI people detection using RGB-D cameras

Introduction

In this project, we have installed an RGB-D sensor camera in the ubuntu version of the SD image for the Kria SOM.

Tried to use within the apps available in Ubuntu.

Unfortunately, no straightforward use has been obtained.

First steps to have KV260 with ubuntu and examples app

Starting with SD ubuntu image SD, install the example application nlp-smartvision

sudo snap install xlnx-config --classic

xlnx-config.sysinit

reboot

sudo xlnx-config --snap --install xlnx-nlp-smartvision

sudo xlnx-config --xmutil listapps

sudo xlnx-config --xmutil loadapp nlp-smartvision

=> accelerator loaded in slot 0

sudo xlnx-config --xmutil listapps

Now the app is ready, and any USB standard might be used with:

xlnx-nlp-smartvision.nlp-smartvision -u -v

Installation of LibRealSense Camera D435

Install utilities

sudo apt-get update

sudo apt-get upgrade

sudo apt-get install v4l-utils

sudo apt-get install lsusb

Librealsense repository - clone:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE || sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-key F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE

sudo add-apt-repository "deb https://librealsense.intel.com/Debian/apt-repo $(lsb_release -cs) main" -u

sudo apt-get install librealsense2-utils

reboot

git clone https://github.com/IntelRealSense/librealsense.git

cd librealsense/

./scripts/setup_udev_rules.sh

sudo apt-get install git libssl-dev libusb-1.0-0-dev libudev-dev pkg-config libgtk-3-dev

sudo apt-get install libglfw3-dev libgl1-mesa-dev libglu1-mesa-dev at

./scripts/setup_udev_rules.sh

./scripts/patch-realsense-ubuntu-lts.sh

mkdir build && cd build

cmake../ -DBUILD_EXAMPLES=true

sudo snap install cmake

sudo apt install cmake

cmake../ -DBUILD_EXAMPLES=true

sudo make uninstall && make clean && make && sudo make install

reboot

Execute an example of RealSense RGB/D D435 camera

rs-distance

rs-capture

Results - work in progress: Next-steps

Unfortunately,I cannot succeeded to have the D435 camera connected as USB, multiple problems arised.

Besides, the stability of the librealsense library in the platform is not good, causing frequently hangs to the ubuntu OS.

Therefore, next step (work-in progress) is to make the installation of D435 camera in a petalinux OS.