Real-Time Face Recognition on Ultra96-V2

Introduction

The Xilinx Model Zoo contains many pre-built convolutional neural network models.

This project makes use of several of these models, in order to implement a foundation for creating face applications.

• WorkDetect : face detection ( densebox_640_360 )
• WorkTrack : face tracking
• WorkCapture : face quality ( face-quality )
• WorkRecog : face features ( facerec_resnet20 )
• WorkCompare : find matching face

The exampe application also implements a multi-task pipeline, that could be leveraged for other applications.

Let's get started !

Step 1 - Create the SD card

Pre-built Vitis-AI 1.3 SD card images have been provided for the following Avnet platforms:

• u96v2_sbc_base : Ultra96-V2 Development Board
• uz7ev_evcc_base : UltraZed-EV SOM (7EV) + FMC Carrier Card
• uz3eg_iocc_base : UltraZed-EG SOM (3EG) + IO Carrier Card

The download links for the pre-built SD card images can be found here:

• Vitis-AI 1.3 Flow for Avnet Vitis Platforms : https://avnet.me/vitis-ai-1.3-project

Once downloaded, and extracted, the.img file can be programmed to a 16GB micro SD card.

0. Extract the archive to obtain the .img file

1. Program the board specific SD card image to a 16GB (or larger) micro SD card

a. On a Windows machine, use Balena Etcher or Win32DiskImager (free opensource software)

b. On a linux machine, use Balena Etcher or use the dd utility

$ sudo dd bs=4M if=Avnet-{platform}-Vitis-AI-1-3-{date}.img of=/dev/sd{X} status=progress conv=fsync

Where {X} is a smaller case letter that specifies the device of your SD card. You can use “df -h” to determine which device corresponds to your SD card.

Step 2 - Download the Face Recognition source code

The code for this project is one of the benefits of the Xilinx Developer Program.

https://developer.xilinx.com/

If you are not registered in the Xilinx Developer Program, you can do it now at the following link:

https://developer.xilinx.com/en/developer-program.html

Once registered, navigate to the "Demos & Designs" tab, where you can find the source for several application examples, including face recognition.

Download the "face_recog_demo_src_v1.3.tar.gz" archive, and copy to the SD card of the Ultra96-V2 board.

Step 3 - Extract and Build the Face Recognition example.

The "face_recog_demo_src_v1.3.tar.gz" archive contains many files (.*) which are not useful. I recommend extracting the archive with the --exclude argument to prevent extracting these unwanted files.

root@u96v2-sbc-base-2020-2:~# tar xvzf  face_recog_demo_src_v1.3.tar.gz --exclude='.*'

With the archive extracted, we can build the application. We want the X-windows version of the application, so we build with the build_x.sh script.

root@u96v2-sbc-base-2020-2:~# cd face_recog_demo_src_v1.3/
root@u96v2-sbc-base-2020-2:~/face_recog_demo_src_v1.3# source ./build_x.sh

My first attempt to build the application on the Ultra96-V2 resulted in an out of memory error. This turns out to be caused by calling make with a "-j" argument that is too high (ie. 10). To fix this, modify the cmake.sh script and change the following line:

make -j 2

This can be done with sed as follows:

sed -i 's/make -j 10/make -j 2/' cmake.sh

This time, the build should succeed after a few minutes.

root@u96v2-sbc-base-2020-2:~/face_recog_demo_src_v1.3# source ./build_x.sh

Native-platform building...
cd /home/root/face_recog_demo_src_v1.3/build_x
cmake -DBUILD_TEST=ON -DBUILD_SHARED_LIBS=ON -DUSE_DRM=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/home/root/.local/petalinux.2020.2.aarch64.Release /home/root/face_recog_demo_src_v1.3
...
[100%] Linking CXX executable face_demo
[100%] Built target face_demo
make: *** No rule to make target 'install'.  Stop.

Two executables are created:

• build_database : used to create the database of face features
• face_demo : used to recognize faces in video

Step 4 - Execute with the provided images/video

The example comes with several faces and a test video.

root@u96v2-sbc-base-2020-2:~/face_recog_demo_src_v1.3# ls images
gaoyue.jpg  jiangfan.jpeg  liuxin.jpg  shengxiao.jpg  shili.jpg  yangye.jpeg  zhanghui.jpg

root@u96v2-sbc-base-2020-2:~/face_recog_demo_src_v1.3# ls *.webm
colleage_3.webm

To run the face recognition example with these provided files, we first need to create the database of faces:

root@u96v2-sbc-base-2020-2:~/face_recog_demo_src_v1.3# ./build_database ./images

This will create a face database file called face_feature.lib.

Prior to running the example, we need to set our DISPLAY environment variable and set the display resolution to 640x480.

root@u96v2-sbc-base-2020-2:~/face_recog_demo_src_v1.3# export DISPLAY=:0.0
root@u96v2-sbc-base-2020-2:~/face_recog_demo_src_v1.3# xrandr --output DP-1 --mode 640x480

To execute the face recognition with the test video:

root@u96v2-sbc-base-2020-2:~/face_recog_demo_src_v1.3# ./face_demo ./colleage_3.webm

Here are my observations on this version of the example:

• test video contains the exact same faces that were used to create the database of features
• test video is "out of focus" in the middle, resulting in lower face qualify, and thus no face recognition
• the face that is displayed on the bottom left of the monitor is the last recognized face (with a match
• only one recognized face is displayed per frame (even if multiple faces were recognized)

Refer to the provided readme.txt for more information on the face recognition application example.

Step 5 - Execute with your custom faces, and USB camera

I wanted to push the example a little further and see if it worked with custom faces, and a video that contains the same people, but not the same face images.

root@u96v2-sbc-base-2020-2:~/face_recog_demo_src_v1.3# ls ./my_faces/
Kevin.jpg  Mario.jpg  Monica.jpg
root@u96v2-sbc-base-2020-2:~/face_recog_demo_src_v1.3# ./build_database ./my_faces/

I tested this with a video my team and I created a while back for the Dobble Challenge project.

Summary

I was really impressed by how easy it was to compile and execute this Face Recognition example on the Ultra96-V2.

If I find time, I may try to make the following modifications in order to better understand the code:

• display face quality
• display name in addition to face ID
• display recognized face image next to each detected face's bounding box

Create your own project and win 500$

I hope you can make use of this great face recognition example as a starting point for your next project. What application would you create ?

Don't forget to Post it on the Hackster.io Xilinx page for a chance to be featured on the Xilinx Developer Site!

In addition, Hackster.io will also award the project of the month winner $500 US dollars based on project application, inventiveness, and creativity as part of the Big Xcellent Adventure program.

Good luck !

Revision History

2021/05/03 - First Version