Badger Robot Reload with ZCU104

Project Purpose and Expectations

We propose a driving robot that use image recognition and self-driving technology. optimized for golf courses. What's noteworthy in this paper is that we used Vitis AI, which Xilinx developed for all developers, to recognize images. Self-driving robots that track geese are equipped with automatic monitoring functions as they go around golf courses. The concept of our project is that several robots send and receive data from each other, recognize and track geese, and chase them away that cause damage to commercial areas. When robots recognize geese, they accumulate geese photo data and provide it to the lab, enabling efficient use in the area of animal behavior modification and Pest control Industry.

Our Badger's Story

Hello. We are BADGER, which will protect people's living environment from wild animals with self-driving robots.

On American golf courses make up 50% of the world's golf Course population. And they're suffering certain damage from wild animals.

Here is a golfer.

The golfer was embarrassed when he was competing in a golf tournament not long ago.

This is how a goose flew in and attacked the boy. He even fell, but the goose didn't stop the attack. Like this, geese that live around the golf course attack golfers. During Migration, hundreds of them are flocking to settle down, making it difficult for owners to manage them.

Today's main character is the Canadian goose.

The Canadian goose eats 5 pounds of grass every day, and poop 1-2 pounds everyday.

In fact, the PGA Golf Course in California pays $250, 000 a year to get rid of these goose droppings.

Existing Solution

These are the traditional methods used to cope with this.

The golf industry is now highly digitized, but analog methods have dominated how to deal with goose damage.

To solve this problem, companies with PEST CONTROL technology produce/sell goose chase machines with their own technology, but they are also unproductive in terms of high price and technology.

Goosinator typically sells for $5, 000 per product and requires manual manipulation by the associated personnel at the time of operation, so there is a hassle of having to call in people to manage geese whenever a problem occurs.

Bird-x offers lower cost but it can only cover seven acres, making it difficult to manage the entire area of the 100-190 acres PGA golf course at any given time. Existing services seem to have a higher purpose than preventive solutions, and post-problem measures than preventive solutions. It seems difficult to cover the size of the PGA golf course, which is not automated and has a large area.

After all, the equipment was less frequently used, and for golf owners, it was too expensive or unproductive to use on the golf course. As a result, golf market stakeholders recognize high-priced equipment and unproductive services, and needs are a precautionary solution before the goose nestles to cover the PGA area, and should also be easy to operate.

Let me introduce you to the features and strengths of our robot with Xilinx products.

Our Robot - Originality and Differentiation

We have designed a big durable amphibious goose remover badger For the PGA golf course that spends $250k a year on goose removal and maintenance. Our amphibious badger is able to photograph and detect geese and chase them away. In addition to that the badger has GPS patrol path with collision avoidance and neural network image recognition. Our badger helps drive out the geese that interfere with the business in a low maintenance and cost-effective manner instead of the old fashion way of shooting them.

Our Robot - Technical Development

1. Easy to Drive

The robot can be easily driven to the owner's own golf course. It is a GUI that the user can directly set the course for the robot to operate. When the user sets up a GPS path in the GUI and starts working, the BADGR starts working autonomously.

2. Deep Learning Image Recognition w/ Vitis AI

Previous and current solutions for geese detection are conducted by Yolov3 on the Raspberry Pi 3+ and Pi4 with two Intel Compute Stick. In this project, we apply Vitis AI library and its model loaded on the ZCU 104 board to detect our target object, Canadian Geese.

The robot recognize goose by neural network image recognition technology using Vitis AI.

3. Amphibious

Our product was designed with an amphibious model. The robot has tank drive with chains linking the motors to the wheels in order to isolate the motors from the water in the amphibious mode.

4. Autonomous

A badger running a course set by the user. BADGER can recognize obstacles and avoid them when they meet obstacles. The robot has a camera turret to detect geese during the patrol and uses time-of-flight sensors for obstacle avoidance.

5. Data

When BADGR finds a goose, after taking a picture of the goose, it goes after the goose. The pictures taken are automatically uploaded to Drive. The robot Photographs and chases detected geese​. This option is not implemented and applied with Vitis AI, yet.

Cost Effectiveness

We've reduced depreciation costs with our online channel rental service and secured economic efficiency with less management burden. The purchase cost per unit is $5, 500, and the rental fee is $990.

Make GUI workspace

Monitor

The board only supports display port

USB hub

USB and USB hub can be connected with BT mouse and keyboard, it can detect USB flash drive as well.

Note: Other ports are not used.

Current board and display setup run properly, and the setup can use the board without using ssh (we do not need to use serial communication, and ssh was used for installation and debugging for the host and target setup). Using them, it can transfer files directly to the board and use terminal built in the board.

Comparing goose detection (original model vs Vitis AI sample model)

The original detection FPS (Frame Per Second) are less than 4.5 but accuracy is more than 96.3%. When we used ZCU104 board with yolov3 model to test, the models show good FPS near the performance FPS measured before. But when it is tested with 50 test images with yolov3 vcc model, most images are recognized as bird, but the accuracy are very low (average: 54.6%, standard variation: 44.71%). It is expected that the advantages of both cases can be used at the same time if the model is made and used by focusing more on Goose. This is a preliminary validation to detect the target without any specified target detection which is Canadian Geese in our case. As a future work, this can be improved by training with fine tuning.

Out Action Plan

Our BADGER finished driving at an alpine golf course in a small town called Longview, East Texas, and has since been working on an action plan to test prototypes on a PGA-class golf course.

Previous Badger Robot is built as LeTourneau University Senior Design Project with a global team of 12 friends of three nationalities, consisting of a mechanical and electronic, business team, after end of the senior design, the project is being conducted with the robotics group members. In this contest, the team members are listed in this submission contributed to consider the potential usage of Vitis AI and ZCU 104 board and conduct the preliminary validation to apply Vitis AI-ZCU 104 board solution. It was BADGER that solved the conflict between wild animals and people.

Please see our Github readme for more information about our contest work (https://github.com/yeram410/xilinx_contest.git).

Thank you.

Appendix

Contest Information: Adaptive Computing Developer Contest with Xilinxhttps://www.hackster.io/contests/xilinxadaptivecomputing

Forum Resource link:https://forums.xilinx.com/t5/Adaptive-Computing-Challenge/bd-p/ACC_2020

ZCU 104 Evaluation Kit Info.https://www.youtube.com/watch?v=DXiYMplqm2Q

Hardware Configurationhttps://www.xilinx.com/products/boards-and-kits/zcu104.html#hardware

Starting Manualhttps://github.com/Xilinx/ZCU104-reVISION-Getting-StartedLicense GenerationNote: For Vitis AI, there is no need to generate any Licnese

Vivado Design Suite User Guide:http://www.xilinx.com/cgi-bin/docs/rdoc?v=latest;d=iil.pdf;a=ObtainManageLicense

USB-UART driver: CP210x USB to UART bridge

To use Serial Communication, this driver should be installed, and you can get the file from the following site.https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842446/Setup+a+Serial+Console

Booting option from the user-guide

Note: SD1 Boot Mode is mainly used in this project with UART serial communication for debugging purpose.

SD card

Our project tried two approaches 1) Follow the user guide to generate the image 2) download and use the pre-imaged file

Note: after following the instructions, you may build the image; however, you may add and install a few files to make the image fully working with the provided examples in the user guide.

To use Serial Communication, this driver should be installed, and you can get the file from the

Note: Considered the pynq versions, however, in the project, this is not used as we decide to keep using petalinux + Vitis AI.

Note: you may use the following link for checking information about pynq.https://pynq.readthedocs.io/en/latest/getting_started/zcu104_setup.html

Example: Tried with the team’s laptop to run pynq + jupyter

IP configuration

Note: we only use the ethernet

Vethernet

ZCU104 webinar

https://event.on24.com/eventRegistration/console/EventConsoleApollo.jsp?&eventid=2552508&sessionid=1&username=&partnerref=&format=fhvideo1&mobile=&flashsupportedmobiledevice=&helpcenter=&key=12AF8FECC081E16A1AD0DA74931F1585&newConsole=true&nxChe=true&text_language_id=en&playerwidth=748&playerheight=526&eventuserid=343773765&contenttype=A&mediametricsessionid=332153072&mediametricid=3593851&usercd=343773765&mode=launch

ZCU104 Smart Camera Demo Design

https://github.com/Xilinx/Embedded-Reference-Platforms-User-Guide/blob/master/Docs/overview.md#213-zcu104-smart-camera-platform

http://www.pynq.io/ pynqhttps://www.youtube.com/watch?v=emXEmVONk0Q

Setup: Boot from SD Card

AI libraryhttps://www.xilinx.com/support/documentation/sw_manuals/vitis_ai/1_1/ug1354-xilinx-ai-sdk.pdf

https://github.com/Xilinx/Vitis-AI