Preventing Bird Strike at Airport with Vision Ai

Thousands of airplane bird strikes occur each year, in spite of airports having many safety and conservation measures in place to minimize these potentially damaging and dangerous situations to both the Aircraft and the Crew / Passengers onboard. To prevent this I am going to build an active Bird detection and Prevention system using Xilinx Kria KV260 module which can detect the birds in the vicinity of the airfield and can trigger the necessary measures (Shot Guns, Sonic Cannons, Flare, Laser etc.) to distract the bird and keep them away. Also this system can flag the Warning to the Runway Control OPs to immediately rush the spot and take necessary actions. Also If the birds are directly on the way to to approaching/departing aircraft on the Runway, it can even flag the pilots to make an go around and prevent the bird strike completely.

Background

Background

A bird strike is strictly defined as a collision between a bird and an aircraft which is in flight or on a take off or landing roll. The term is often expanded to cover other wildlife strikes - with bats or ground animals.

Bird Strike is common and can be a significant threat to aircraft safety. For smaller aircraft, significant damage may be caused to the aircraft structure and all aircraft, especially jet-engined ones, are vulnerable to the loss of thrust which can follow the ingestion of birds into engine air intakes. This has resulted in a number of fatal accidents.

Bird strikes may occur during any phase of flight but are most likely during the take-off, initial climb, approach and landing phases due to the greater numbers of birds in flight at lower levels. Since most birds fly mainly during the day, most bird srikes occur in daylight hours as well.

According to the Federal Aviation Administration, over 10, 000 bird strikes occur each year, averaging more than 26 hits a day !!

That's why I decided to develop Birds Detection and Prevention System using Xilinx Kria KV260 Vision AI Starter Kit and I have interface the board with High Res camera installed at the strategic location on the Airfield and use the Vitis AI to train the ML model to detect the birds. Once the birds are detected by the Model the solution can trigger the necessary measure/actions like Shot Guns, Airport Bird Wailer, Sonic Cannons, Flare, Laser etc. which are connected conventionally (Relay Module) with the Xilinx Kria KV260 Vision AI Starter Kit.

Preparing the KV260 Board

Preparing the KV260 Board

First you need to Download this petalinux image (https://www.xilinx.com/member/forms/download/design-license-xef.html?filename=xilinx-kv260-dpu-v2020.2-v1.4.0.img.gz). Also you need to have a Xilinix Developer Account to to download the petalinux image.

• First you need to Download this petalinux image (https://www.xilinx.com/member/forms/download/design-license-xef.html?filename=xilinx-kv260-dpu-v2020.2-v1.4.0.img.gz). Also you need to have a Xilinix Developer Account to to download the petalinux image.
  

Download the Balena Etcher (recommended; available for Window, Linux, and macOS)

• Download the Balena Etcher (recommended; available for Window, Linux, and macOS)
  

Follow the instructions in the tool and select the downloaded image to flash onto 16 GB Micro-SD Card that comes with the Starter Kit.

• Follow the instructions in the tool and select the downloaded image to flash onto 16 GB Micro-SD Card that comes with the Starter Kit.
  

Once the Card is Ready connect everything up to Kria KV260 Board as follows-

Insert the microSD card containing the boot image in the microSD card slot (J11) on the Starter Kit

• Insert the microSD card containing the boot image in the microSD card slot (J11) on the Starter Kit
  

Get your USB-A to micro-B cable (a.k.a. micro-USB cable), which supports data transfer.* Do not connect the USB-A end to your computer yet. Connect the micro-B end to J4 on the Starter Kit.

• Get your USB-A to micro-B cable (a.k.a. micro-USB cable), which supports data transfer.* Do not connect the USB-A end to your computer yet. Connect the micro-B end to J4 on the Starter Kit.
  

Connect the Webcam to USB port.

• Connect the Webcam to USB port.
  

Grab the Power Supply and connect it to the DC Jack (J12) on the Starter Kit.

• Grab the Power Supply and connect it to the DC Jack (J12) on the Starter Kit.
  

You can either connect a HDMI monitor, a keyboard, and a mouse with KV260 Board

or

Connect your laptop to the Kria using a micro-USB cable. Then to remotely connect to the board,

for Mac and Linux users, `ssh` into the Kria,

• for Mac and Linux users, `ssh` into the Kria,
  

ssh root@<Ip-Address>

ssh root@<Ip-Address>

for Windows users, use Putty or TeraTerm

• for Windows users, use Putty or TeraTerm
  

Config for Putty or TeraTerm are as Below

1.Baud rate = 115200

2.Data bits = 8

3.Stop bits = 1

4.Flow control = None

5.Parity = None

and Select lower COM port as two COM ports should be enumerated. The lower numbered COM port corresponds to the UART.

Download Vitis-AI Runtime Libraries

Download Vitis-AI Runtime Libraries

These libraries are needed to run models on the DPU. These models can be from the Vitis AI model zoo or your own compiled & quantized.

1. Connect your board to Ethernet (Either connect it to your router or share internet of your pc via ethernet)

2.Download the Libraries from here and SCP to the board as follows.

scp vitis-ai-runtime-1.4.0.tar.gz root@<ip-address>:~/

scp vitis-ai-runtime-1.4.0.tar.gz root@<ip-address>:~/

where replace <ip-address> with the ip address of the board you get from the ifconfig command from the terminal window of Putty or TerraTerm.

3. Decompress the downloaded.tar.gz:

tar -xzvf vitis-ai-runtime-1.4.x.tar.gz

tar -xzvf vitis-ai-runtime-1.4.x.tar.gz

4. cd /vitis-ai-runtime-1.4.0/2020.2/aarch64/centos/setup.sh

5. setup Vitis AI Runtime Libs:

bash setup.sh

bash setup.sh

Run BirdDetect Model

Run BirdDetect Model

1. Clone my repository:

git clone https://github.com/shubhamj16/BirdDetect.git

git clone https://github.com/shubhamj16/BirdDetect.git

2. cd BirdDetect

3. Build the C++ code that performs inference on the RefineDet model and returns `1` for Bird detected and returns `0` for no Bird detected:

./build.sh

./build.sh

4. Connect the camera and Digilent Relay Module to the Kria Board.

5. Run the python script that calls the C++ code which trigger the Relay Module on/off connected to PMOD connector of Kria Board :

python3 main.py

python3 main.py

Lets see the Bird Detection in Action

Lets see the Bird Detection in Action

I have tested The Bird Detection with Kria KV260 in the Field and the Performance of it really impressed me! It's able to detects more than hundred's of birds at a moment with success rate of more than 95%, while maintaining the 30 FPS mark.

Due to the Covid Restriction I was not able to test it in theAirfield But iam going to test that soon by placing the camera to the strategic location (R/W ends near Approach Lights) at the airfield to detect birds!

Action Triggering

Action Triggering

In order to trigger Action Like Shot Guns, Airport Bird Wailer, Sonic Cannons, Flare, Laser etc. we can Map those Units conventionally with the Relay module to turn ON/ OFF so like that we can use the existing measures at the airport and Retro Fit them to take actions whenever the Birds are detected.

We can Connect Kria Board Output via Realy Module to the Airport Bird Wailer which is a system that causes birds to move away from airport runways and helipads, using high fidelity natural sounds such as distress calls, alarm calls and cries of predators. This system creates a zone which is uncomfortable and threatening for most bird species, and so they leave and hence reduces the chances of Bird Strikes.

Future Plans And Scopes

Future Plans And Scopes

I'm going to make a Companion Apps for the Ground Ops so that they can also get the notification of the birds with location so they rushed to the spot where the Birds are detected and manually keep them out of the way of the aircraft.

I'll also develop a complete integration with RT (Radio Transmission between ATC/ Ground OPs ans Pilots) so that they can also get the warning over it. And if required the Pliot may Go-around to prevent the Bird Strike by this Automated Bird Detection Warning.

The scopes are endless and stay tunes for the future iteration.

Thanks For Reading 🙏