Preventing Bird Strike at Airport 2.0 with AMD Ryzen™ AI

Thousands of airplane bird strikes occur each year, in spite of airports having many safety and conservation measures in place to minimize these potentially damaging and dangerous situations to both the Aircraft and the Crew / Passengers onboard. To prevent this I am going to build an active Bird detection and Prevention system using AMD Ryzen AI Powered PC which can detect the birds in the vicinity of the airfield and can trigger the necessary measures (Shot Guns, Sonic Cannons, Flare, Laser etc.) to distract the bird and keep them away. Also this system can flag the Warning to the Runway Control OPs to immediately rush the spot and take necessary actions. Also If the birds are directly on the way to to approaching/departing aircraft on the Runway, it can even flag the pilots to make an go around and prevent the bird strike completely.

Background

A bird strike is strictly defined as a collision between a bird and an aircraft which is in flight or on a take off or landing roll. The term is often expanded to cover other wildlife strikes - with bats or ground animals.

Bird Strike is common and can be a significant threat to aircraft safety. For smaller aircraft, significant damage may be caused to the aircraft structure and all aircraft, especially jet-engined ones, are vulnerable to the loss of thrust which can follow the ingestion of birds into engine air intakes. This has resulted in a number of fatal accidents.

Bird strikes may occur during any phase of flight but are most likely during the take-off, initial climb, approach and landing phases due to the greater numbers of birds in flight at lower levels. Since most birds fly mainly during the day, most bird srikes occur in daylight hours as well.

Damages to Aircraft Due to Bird Strike

According to the Federal Aviation Administration, over 10, 000 bird strikes occur each year, averaging more than 26 hits a day !!

My this year Project is the continuation of Last Year Adaptive Computing Challenge where I have used Kria KV260 Vison Ai Board to detect the birds in the Vicinity of Airport , which was a massive Success here in India, after Testing closely with Indian Air Force and Airport Authority of India (AAI). It has already prevented many Bird Strikes by Triggering Bird Wailers on right time and Keeping the Birds away from the Airfield.

But the major challenges that I have faced while Deploying this System on the Airfield is to Prevent the False Positive (identifying non-birds as birds) which was majorly due to Low Lighting during Night and Distorted Visuals due to Mirage formation in Summer days in India also due to these challenging conditions sometimes the system won't able to recognize the Birds and didn't trigger the warning.

So in order to make this project Full-Proof and multi level Redundant this year Hardware offering of AMD Ryzen AI Powered PC is the perfect fit for my use cases. Using the AMD Ryzen™ AI powered PCs NPU Core & ML we can further fine tune our Bird detection algorithms by implementing Data Augmentation, LiDAR Integration (along with the Camera) and Multi-Modal learning. By implementing these Strategies we can mitigate the above Challenges with the the help of the AMD Ryzen Ai Powered PC which is fully capable to perform on such Ai workload in Real time and we can make this system better equipped to handle the complexities of bird detection near runways, even in challenging summer weather conditions with mirages and in low lighting. And Once the birds are detected by the Model the solution can trigger the necessary measure/actions like Shot Guns, Airport Bird Wailer, Sonic Cannons, Flare, Laser etc. which are connected conventionally (Relay Module) with the AMD Ryzen AI Powered PC.

Preparing the AMD Ryzen AI Powered PC

First you need to update the GPU Drivers and enable the IPU Cores of AMD Ryzen AI Powered PC.

To Enable the IPU:

1. From Windows Search, enter “Advanced Startup”. This will open the System Settings menu. Under “Recovery options”, in the Advanced startup row, click “Restart Now”.

• This menu can also be found by clicking the Windows start button, select “Settings”, then “Recovery”.

2. When the PC reboots, you’ll enter the Advanced Startup page. From here, select “Troubleshoot”.

3. In the “Troubleshoot” menu, select "Advanced Options”.

4. In the “Advanced Options” menu, select "UEFI Firmware Settings: Change settings in your PC’s UEFI firmware”.

This will reboot your machine and bring you to the BIOS menu.

The following is specific to the Minis Forum BIOS:

5. In the left menu, select “Advanced” and from the main body, select “CPU Configuration”.

The main body, the last line will be “IPU Control”. If this is “Disabled”, set to “Enabled”.

After the IPU cores are enabled and you restart the PC you should see the AMD IPU Device under Device manager => System Devices.

Install the Ryzen AI Software package from AMD's website

• For information on setting up your Ryzen AI software/device, see the following: Installation.

Install Python and required libraries:

pip install torch torchvision onnx onnxruntime opencv-python numpy

Data Collection and Preprocessing

• I have collected a dataset of bird images in various airport environments and also uses a publicly available dataset like "Birds 525 Species" from Kaggle.
• Preprocess and augment the data:

import torch
from torchvision import transforms

train_transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.RandomRotation(20),
    transforms.ColorJitter(0.2, 0.2, 0.2, 0.1),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])

Model Development and Training

• Create a PyTorch model for bird detection:

import torch
import torchvision.models as models

class BirdDetectionModel(torch.nn.Module):
    def __init__(self, num_classes=2):
        super(BirdDetectionModel, self).__init__()
        self.model = models.mobilenet_v2(pretrained=True)
        self.model.classifier[1] = torch.nn.Linear(self.model.last_channel, num_classes)
    
    def forward(self, x):
        return self.model(x)

model = BirdDetectionModel()

• Train the model on your dataset:

import torch.optim as optim
import torch.nn.functional as F

criterion = torch.nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

num_epochs = 10
for epoch in range(num_epochs):
    for batch in train_loader:
        inputs, labels = batch
        optimizer.zero_grad()
        outputs = model(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()
    print(f'Epoch {epoch+1}/{num_epochs}, Loss: {loss.item()}')

torch.save(model.state_dict(), 'bird_detection_model.pth')

Model Optimization for Ryzen AI

• Convert the trained model to ONNX format:

import torch

model.eval()
dummy_input = torch.randn(1, 3, 224, 224)
torch.onnx.export(model, dummy_input, "bird_detector.onnx", opset_version=13)

• Quantize the model using AMD Vitis AI Quantizer:

from vitis_ai_onnx.onnx_quantizer import ONNXQuantizer

quantizer = ONNXQuantizer("bird_detector.onnx", "bird_detector_quantized.onnx")
quantizer.quantize()

Real-time Inference System

• Set up the Logitech Brio 4K camera and relay module.
• Implement the real-time inference script:

import cv2
import numpy as np
import onnxruntime as ort
import RPi.GPIO as GPIO  # For relay control

# Set up GPIO for relay
RELAY_PIN = 18
GPIO.setmode(GPIO.BCM)
GPIO.setup(RELAY_PIN, GPIO.OUT)

# Load the quantized ONNX model
providers = ['VitisAIExecutionProvider', 'CPUExecutionProvider']
session = ort.InferenceSession("bird_detector_quantized.onnx", providers=providers)

# Set up camera
camera = cv2.VideoCapture(0)  # Adjust if necessary for Logitech Brio 4K
camera.set(cv2.CAP_PROP_FRAME_WIDTH, 3840)
camera.set(cv2.CAP_PROP_FRAME_HEIGHT, 2160)

def preprocess_image(image):
    image = cv2.resize(image, (224, 224))
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    image = image.transpose((2, 0, 1))
    image = image / 255.0
    image = (image - np.array([0.485, 0.456, 0.406])[:, None, None]) / np.array([0.229, 0.224, 0.225])[:, None, None]
    return image.astype(np.float32)

def activate_bird_deterrent():
    GPIO.output(RELAY_PIN, GPIO.HIGH)
    print("Bird deterrent activated!")

def deactivate_bird_deterrent():
    GPIO.output(RELAY_PIN, GPIO.LOW)
    print("Bird deterrent deactivated.")

bird_detected_count = 0
ACTIVATION_THRESHOLD = 5  # Activate after 5 consecutive detections

try:
    while True:
        ret, frame = camera.read()
        if not ret:
            break

        input_data = preprocess_image(frame)
        input_data = np.expand_dims(input_data, axis=0)

        # Run inference
        outputs = session.run(None, {"input": input_data})
        prediction = np.argmax(outputs[0])

        if prediction == 1:  # 1 is "bird" class
            bird_detected_count += 1
            if bird_detected_count >= ACTIVATION_THRESHOLD:
                activate_bird_deterrent()
        else:
            bird_detected_count = 0
            deactivate_bird_deterrent()

        # Display result
        cv2.putText(frame, f"Bird Detected: {'Yes' if prediction == 1 else 'No'}", 
                    (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2)
        cv2.imshow("Bird Detection", frame)

        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

finally:
    camera.release()
    cv2.destroyAllWindows()
    GPIO.cleanup()

Testing and Optimization

• Run the system in various lighting conditions and with different bird species.
• Adjust the ACTIVATION_THRESHOLD as needed for optimal performance.
• Monitor the NPU usage and system performance:

import psutil

def print_system_stats():
    cpu_percent = psutil.cpu_percent()
    memory_percent = psutil.virtual_memory().percent
    print(f"CPU Usage: {cpu_percent}%")
    print(f"Memory Usage: {memory_percent}%")

# Call this function periodically in your main loop

Deployment

• Install the system at the airport, ensuring the camera has a clear view of the runway area.
• Set up remote monitoring capabilities:

import logging
import paramiko

logging.basicConfig(filename='birdwatch_ai.log', level=logging.INFO)

def log_event(event_type, message):
    logging.info(f"{event_type}: {message}")

# Example: Log bird detections
log_event("DETECTION", "Bird detected at runway 3")

# Set up SSH for remote monitoring
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('remote_server', username='user', password='password')

def send_alert(message):
    ssh.exec_command(f'echo "{message}" | wall')

# Example: Send alert for system issues
send_alert("BirdWatch AI system requires maintenance")

Lets see the Bird Detection in Action

I have tested The Bird Detection with AMD Ryzen Ai PC in the Field and the Performance of it really impressed me! It's able to detects more than hundred's of birds at a moment with success rate of more than 95%, while maintaining the 30 FPS mark.

Testing The Bird Detection in the Field

Due to the Operational Restriction I was not able to test it in the Airfield But I am going to test that soon by placing the camera to the strategic location (R/W ends near Approach Lights) at the airfield to detect birds!

Action Triggering

In order to trigger Action Like Shot Guns, Airport Bird Wailer, Sonic Cannons, Flare, Laser etc. we can Map those Units conventionally with the Relay module to turn ON/ OFF so like that we can use the existing measures at the airport and Retro Fit them to take actions whenever the Birds are detected.

Digilent Pmod SSR: Solid State Relay Electronic Switch

A Typical Airport Bird Wailer

We can Connect Ryzen AI PC to Output via Relay Module to the Airport Bird Wailer which is a system that causes birds to move away from airport runways and helipads, using high fidelity natural sounds such as distress calls, alarm calls and cries of predators. This system creates a zone which is uncomfortable and threatening for most bird species, and so they leave and hence reduces the chances of Bird Strikes.

Future Plans And Scopes

I'm going to make a Companion Apps for the Ground Ops so that they can also get the notification of the birds with location so they rushed to the spot where the Birds are detected and manually keep them out of the way of the aircraft.

I'll also develop a complete integration with RT (Radio Transmission between ATC/ Ground OPs and Pilots) so that they can also get the warning over it. And if required the Pilot may Go-around to prevent the Bird Strike by this Automated Bird Detection Warning.

The scopes are endless and stay tunes for the future iteration.

Thanks For Reading 🙏