Ecological Damage caused by Mobile towers

The Problem:

We are aware of all the mobile phone towers that have been introduced in our ecology to provide us with the connectivity that we need, many a time at the expense of mother nature. It has been on many occasions established that birds are affected by the presence of cellphone towers in nearby regions. I have made a hardware-accelerated bird recognition model to see the effect of these towers on the ecology.

Model explanation:

Due to the similarities presented between images of various species of birds, it is difficult to find a pattern between them while compressing or making the pieces of information smaller as it may affect even the slightest of identification that could have benefited the model into distinguishing it from an animal or another bird. Thus the model had to be dense enough to not underfit the data, but at the same time data augmentation was required so overfitting won't be a problem.

The new issue that arises due to this is that birds can have a huge impact if we change the orientation of the image, for example, if you rotate the image of a bald eagle chance of it looking similar to an owl increase. Thus we needed layers that can find patterns within patterns onto unaugmented data. This was particularly tough to handle due to time constraints. I was not able to find a suitable model structure supported for the DPU with layers that can help in the current scenario. So I took the challenge the old fashioned way.

A model with several pairs of Conv2d layers and max pooling layers with a heavy flatten layer near the end before the data was fed into the dense box was made. During the initial tries the kernel size exceeded the limit presented inside the ultra96v2 but after a couple of optimization on the model layering and observing a better pattern in quantization, the model performed substantially better with the small data set. The links to the dataset can be found in the references column of this repo.

For running the inference graph, two approaches were taken. One being traditionally using threads to find the output directly from the images stored inside a directory. This was something that was able to utilize the power of ultra96v2 DPU properly with high FPS reaching up to 120 at times. However, to take things up a notch, the whole system was implemented for live feed input from a Logitech webcam.

The webcam frames were then worked upon by the runDPU function which was able to perform various image pre and post-processing with the help of OpenCV

The repository has all the commands and required screenshots for helping anyone to figure out how this model or repository can be used.

Commands used:

Clone the repo and run the following commands on a terminal.

./docker_run.sh xilinx/vitis-ai:latest 
source ./0_setenv.sh 
source ./2_keras2tf.sh 
source ./4_quant.sh 
source ./6_compile.sh

Detailed command logs for each command can be found in the logs folder of my repository that is attached in the Codes Section.

My Setup:

Video

AIOT

This project can significantly affect how we perceive the damage we cause to nature. If such setups are placed everywhere, it will help us identify and track changes in our ecology.