Automated Livestock Data Collection

The Idea

The Project is to detect the live stocks to assist farmers in identifying the sizes, numbers, and provide some prediction on harvest results. All these should be done with minimal stress to the live stocks to ensure optimum growth for harvesting.

I've decided to downscale my project scope to a general detection and a classification of the my sample crayfish.

Hardware Setup

With Xilinx Kria KV260, it provides a good starting point to begin with. I've connected the HDMI monitor, FHD USB Webcam, Wireless Keyboard & Mouse to the KV260 board. I've also connected the KV260 serial port to my laptop.

And a 3D printed Stand Xilinx Kria KV260 Starter Kit Stand to hold up the board for testing the MIPI camera.

Software Setup :

• KV260 DPU Vitis AI 1.4 Image : As for the base system, I am using the Xilinx KV260 DPU Image file 2020.2.2 : here
• After download, extract, and burn to the microsd card, I used Balena Etcher for this..

Booting KV260

Transfer the microsd card to the kv260, connect the serial port, plug-in the power and the KV260 will boot into a desktop. By default the Vitis-AI folder is in the directory :

Ensure the usb camera is available

Running out of the box 'Samples'

All along I am connected to the KV260 through the serial port of my Laptop, in order to run the samples, either I need to use some x11 forwarding software which would enable us to view the screen or to run from the KV260 Terminal screen it self.

I've decided to run from the KV260 itself through the onboard terminal.

Test 1 : Classification : resnet50

cd Vitis-Ai/demo/Vitis-Ai-Library/samples/classification

./test_video_classification resnet50 /dev/video0

Test 2 : yolov4

cd Vitis-AI/demo/Vitis-AI-Library/samples/yolov4

./test_video_yolov4 yolov4_leaky_spp_m /dev/video0

Testing with my Pet Crayfish

I was searching for the pre-trained AI models which is able to classify the crayfish, and come across the resnet50 being able to do just that. So I tried it with a real subject using the classification sample.

Resnet50

Below is a screenshot of the results, with a probability of 0.4955. (sorry for the blur image)

Yolov4

Next, I wanted to know if I am able to detect more than a single 'crayfish', I tried it with the yolov4, with the results :

video link bounding boxes around the crayfish.

another video link

Combine yolov4 & resnet50

With limited time, I've managed to combine the codes in a simple code, where yolov4 is ran initially and produces the bounding boxes, and then each bounding boxes are run through the classification. The results is as expected, however not satisfactory as the accuracy is low. I believe I could improve further by training a custom model for my desired objects specifically after this to improve the speed of detection.

To be Further Developed

I see further development ahead for my project, as there are more use case required for my sample pet crayfish

• to train a custom crayfish detector
• to convert to xilinx models
• add in a thermal camera

Summary : The advantage of KV260

Hardware wise, its a starter-kit, for those without hardware knowledge, its easy to get started without knowing any hardware of FPGA. I have not touched the hardware part to get the sample project up and running, and there are more to explore.

KV260 Vitis-AI out of the box contains a list of pretrained models which can be found in the /usr/share/vitis-ai-library/models directory, which can be readily utilised to implement our end application.

Disclaimer

• no animals were hurt or injured in this project

Credits

• Thanks to Jack Bonnell
• Thanks to AMD-Xilinx