Published March 31, 2023 © LGPL

Weed Removal & Insecticides/Pesticides Sprayer Robot

Using A.I. to Detect & Remove Weeds Besides the Crop and Spray Insecticides/Pesticides at that point to stop weed from further growing.

AdvancedWork in progressOver 1 day698

Weed Removal & Insecticides/Pesticides Sprayer Robot

Things used in this project

Hardware components

NXP MR-BUGGY3-KIT

NXP RDDRONE-FMUK66

NXP UCANS32K1SIC Development Board

NXP HGD-Telemetry Unit 433 Mhz

Google Coral Camera Module

Orange 2S 7.4V 2200mah Lipo Battery

PCA9685 12 Channel 12 Bit Servo Driver

Generic Robotic Arm with Gripper

USB Camera

SG90 Micro-servo motor

Lipo Charger

Female/Female Jumper Wires

Male/Female Jumper Wires

USB-A to USB-C Adaptor

Software apps and online services

PX4

PX4 QGroundControl

PX4 MAVLink

ROS Robot Operating System

TensorFlow

NXP eIQ Software Toolkit

Pycharm

Hand tools and fabrication machines

Extraction Tool, 6 Piece Screw Extractor & Screwdriver Set

Tape, Double Sided

Story

As per data of the Agriculture Ministry, India had lost agricultural produce worth over $10, 000 million - (Rs 6, 42, 40, 00, 00, 000) more than the Centre's budgetary allocation for agriculture for 2017-18 - between 2003 and 2014 to weeds in 10 major crops in different districts of 18 States. This means agriculture crops worth over $ 910 million (Rs 90, 90, 90, 90, 90, 909) loses annually due to weeds across the country.

And NXP hosted the competition NXP HoverGames3: Land, Sky, Food Supply with NXP so I thought why would not participate and build a project that can help save both Crops and Millions Of Dollars.

So, I proposed a project in which I will use the MR-BUGGY3-KIT and on that I mounted a robotic Arm which is connected to the NAVQPlus Board which can run Weed Detection Model onboard its NPU to detect the weed and extract it with the arm and for an extra mile I will add Insecticides/Pesticides Sprayer so that it prevents from growing it in the future.

Weed Removal Robotic Buggy

Procedure:

1. Follow the below MR-Buggy3 Build Guide to assemble a working prototype.

( https://nxp.gitbook.io/nxp-cup/mr-buggy3-developer-guide/mr-buggy3-build-guide )

2. After that Follow the MR-Buggy3 Software Setup

( https://nxp.gitbook.io/nxp-cup/mr-buggy3-developer-guide/mr-buggy3-software-setup )

3. Make sure to Download and build the beta version (v1.14.0-beta2 at the time of writing) and build according to the given instructions in above link for latest features.

HaveSomeFunWithTheBUGGY 😉

4. Follow the below video for assembling the robotic arm kit

Robotic Arm Assembly

5. Setup the NAVQPlus and connect a Display, Keyboard and Mouse for Development.

Download Pip using Terminal

sudo apt-get install pip

Then download & Install Keras & Tensorflow from their official Websites following the instructions mentioned in the link.

( https://www.tutorialspoint.com/keras/keras_installation.htm )

( https://www.tensorflow.org/install/pip )

Install Colcon using

sudo apt install python3-colcon-common-extension

Install ROS2 (With the latest Image ROS/ROS2 is installed by default if not )

( https://docs.ros.org/en/dashing/Installation/Ubuntu-Install-Debians.html )

6. Following the instructions given in the below link train a custom model on weed using tensorflow ( for better results train on the type of weed commonly found in your area.)

( https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/training.html )

7. Connect the NAVQPlus to the RDDRONE-FMUK66 by following the guide mentioned below

( https://notblackmagic.com/snippets/px4-autopilot/ )

Collaboration:

1.https://discord.com/channels/1014291298812960913/1027691375770218638/1083737455116693564

2. https://discord.com/channels/1014291298812960913/1027691375770218638/1083360693312569414

3. https://discord.com/channels/@me/1084475969072549970/1089519537046691931

https://discord.com/channels/@me/1084475969072549970/1089150799642361866

https://discord.com/channels/@me/1084475969072549970/1088814523680575569

THANKSGIVING:

I would like to thanks igalloway, JingerZeng, NotBlackMagic, mdobrea, bperseghetti, rayv and all the remaining discord channel members for their constant support and help with which I had done this work and will continue to complete the project even after the competition.

import cv2
import numpy as np
from edgetpu.classification.engine import ClassificationEngine
from pynq.lib import Pmod_PWM
from pynq.overlays.base import BaseOverlay

# Initialize the UCANS32K1SIC board and PWM pins
base = BaseOverlay("base.bit")
pwm1 = Pmod_PWM(base.PMODA, 0)
pwm2 = Pmod_PWM(base.PMODA, 1)

# Set the duty cycle and frequency for the PWM pins
pwm1.generate(50, 0.5)
pwm2.generate(50, 0.5)

# Load the Google Coral classification engine
engine = ClassificationEngine('/path/to/model.tflite')

# Open the camera and capture frames
cap = cv2.VideoCapture(0)
while True:
    ret, frame = cap.read()

    # Preprocess the frame for classification
    resized = cv2.resize(frame, (224, 224))
    resized = resized[:, :, ::-1].copy()
    input_data = np.asarray(resized, dtype=np.float32)
    input_data = np.expand_dims(input_data, axis=0)

    # Run the classification on the frame
    results = engine.classify_with_input_tensor(input_data, threshold=0.5, top_k=1)

    # Check if a weed is detected
    if results[0][0][1] == 'weed':
        # Move the servo motors to extract the weed
        pwm1.generate(50, 0.1)
        pwm2.generate(50, 0.9)

    # Display the classification results on the frame
    cv2.putText(frame, results[0][0][1], (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 1,(0, 255, 0), 2, cv2.LINE_AA)
    cv2.imshow('frame', frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the camera and PWM pins
cap.release()
pwm1.stop()
pwm2.stop()
cv2.destroyAllWindows()

import cv2
import numpy as np
import socket

# Set up communication with RDDRONE-FMUK66
UDP_IP = "192.168.1.1"
UDP_PORT = 8000
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

# Create a VideoCapture object for the camera
cap = cv2.VideoCapture(0)

# Check if camera is opened correctly
if not cap.isOpened():
    print("Cannot open camera")
    exit()

while True:
    # Read a frame from the camera
    ret, frame = cap.read()

    # Check if frame is captured correctly
    if not ret:
        print("Can't receive frame (stream end?). Exiting ...")
        break

    # Convert the frame to grayscale and blur it to reduce noise
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    blurred = cv2.GaussianBlur(gray, (5, 5), 0)

    # Threshold the image to create a binary image of the white line
    _, thresh = cv2.threshold(blurred, 200, 255, cv2.THRESH_BINARY)

    # Find contours in the binary image
    contours, _ = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

    # If contours are found, find the contour with the largest area
    if len(contours) > 0:
        c = max(contours, key=cv2.contourArea)

        # Find the centroid of the largest contour
        M = cv2.moments(c)
        cx = int(M['m10'] / M['m00'])
        cy = int(M['m01'] / M['m00'])

        # Draw a circle at the centroid of the largest contour
        cv2.circle(frame, (cx, cy), 5, (0, 0, 255), -1)

        # Generate commands to control the drone to follow the line
        # For example, you could use the centroid coordinates to control the drone's pitch and roll

    # Send commands to stop the drone when a signal is received from RDDRONE-FMUK66
    data, _ = sock.recvfrom(1024)
    if data == b'stop':
        break

    # Display the resulting frame
    cv2.imshow('frame', frame)
    if cv2.waitKey(1) == ord('q'):
        break

# Release the camera and close all windows
cap.release()
cv2.destroyAllWindows()