Real-Time Object Tracking Using a Moving Camera

Introduction

Object tracking is finding region of interest in each consecutive frame of video sequence and can be useful in many applications, such as automotive industry, automatic video surveillance systems and gestures recognition.

This project shows some FPGA accelerated tracking algorithms using Zybo Z7-20 SoC. The system also contains AXIS V5915 PTZ camera that is controlled by on-board processing system via TCP connection with camera's server.

Skin color tracking

Skin color recognition is done in YCbCr color space, which can by achieved by multiplying RGB pixel by 3x3 conversion matrix:

After conversion, a skin color classification mask can be created, 255 value means that pixel is considered a skin.

Ta, Tb, Tc, Td values were found experimentally.

After classification, center coordinates of skin regions can be calculated:

Where:

Im(i, j) is mask value for (i, j) pixel coordinates.

Controlling camera's position

Camera's server accepts HTTP requests on TCP port 80 and its structure looks like this:

For example, if we want to turn camera around vertical axis by 10 degrees to the right, the following request should be sent:

Other parameters are shown in the table below:

System overview

Zybo development board was used for skin color tracking experiment. With heterogenous Zynq-7000 SoC it was possible to divide project into two subsystems: skin color tracking algorithm implemented in programmable logic which was responsible for computing and writing tracked object position to AXI4 Lite interface, and camera control system responsible for fetching object coordinates from AXI4 interface and controlling camera's velocity through TCP loop. General diagram is shown on Figure 1.

Processing system

A modified version of Xilinx SDK TCP Client example project was used. Main program loop is shown on Figure 2.

Programmable logic

Top block design is shown on Figure 3.

To do

Multiscale Kanade-Lucas-Tomasi feature tracking algorithm is currently under tests.