Created February 14, 2021 © Apache-2.0

Land surveying for crop yield optimization

VARI - Poor people's NDVI. We try to recreate the popular NDVI index without the infrared channel, just the RGB! How bad is it?

IntermediateFull instructions providedOver 1 day75

Bonus Prizes

NXP HoverGames Challenge 2: Help Drones, Help Others During Pandemics

Land surveying for crop yield optimization

Things used in this project

Hardware components

NXP RDDRONE-FMUK66

NXP KIT-HGDRONEK66

Software apps and online services

OpenCV

Microsoft .NET (C#, WPF) for Desktop app and OpenCV wrapper

Ubuntu

Story

Simply put, the target is crop yield optimization by automatically scanning the land and look for stressed areas which need the farmers attention. This might be useful if You are monitoring huge areas and with such land mapping one can see which parts of the field are in bad health (crop diseases, human induced changes, etc). The de-facto standard for this is using an infrared camera and with the infrared channel is calculated the NDVI index, which maps the health of the crop. Hyperspectral imaging improved on that with slicing the bands further into smaller bands, making it even more precise. However, both hyperspectral and infrared cameras are a bit expensive for the ordinary farmer, while RGB cameras are cheap and abundant, like the Google Coral RGB. Although it is well known that without infrared it is not possible to obtain meaningful data, we try the pseudo-NDVI index called VARI (Visible Atmospherically Resistant Index). The formula using RGB is simple: VARI = (Green - Red) / (Green + Red – Blue). Values are then normalized to show a grayscale image.

Apart from the NXP provided drone kit, no additional HW was used. The drone was assembled as per the gitbook with the NAVQ mounted on top. Flight was done on "Position" mode, after calibration on QGroundControl.

On NavQ is running Ubuntu which was tweaked to take pictures via gstreamer command. A simple systemctl service was written which runs a bash script on boot before login to take a picture with the Coral camera every 5 seconds and save it in predetermined folder, executing GStreamer command:

$ gst-launch-1.0 -v v4l2src num-buffers=1 ! jpegenc ! filesink location=capture.jpeg

So no mavlink connection with PX4 flight computer was established, NavQ runs independendly. Afterwards a python: -m http.server 8080 command is executed in the navq to run in a server mode, while from the ubuntu PC the images are copied with wget (also with bash script).

Once the images are extracted, they are processed in a .NET WPF C# desktop application using openCV and standard C# image manipulation. The final results are shown on the GUI , from where screenshots are provided.

The drone in the process of building

The assembly was per the NXP gitbook with some issues, notably the original motors were damaged by using the wrong screws. Mounting the camera was a bit problematic too, for the application a better solution is to mount it underneath instead of in front of the drone. The build quality of the drone is sturdy, survining a couple of hard crushes.

Hovergames drone airborne

There is also a video of flying in "Position" flight mode and landing.

Results are somewhat lacking, as the images taken with the drone suffer from blurring, direct sunlight, badly position camera and vibrations. Last image is a stock photo of the desired target, the same algorithm on a clear image. Despite the lower resolution, results are more convincing. A small speck of black visible on the upper left side is corresponding to the gray/less green patch in the original image, signaling stressed area and possible bad health of the crop.

1 / 4 • Unsatisfying results on Google Coral images

On the following image (not taken with the Hovergames drone) was the actual target of the application. Although invisible for naked eye on first sight, the black pointed patch corresponds with a somewhat gray area in the original image on second inspection.

Demonstration of the desired results on a stock photo with pointed stressed crop patch.

using Emgu.CV.OCR;
using Emgu.CV.UI;
using Emgu.Util;
using Emgu.CV;
using Emgu.CV.Util;
using Emgu.CV.Structure;
using Microsoft.Win32;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
using System.Drawing.Imaging;
using System.Drawing;

namespace hovergames
{
    /// <summary>
    /// Interaction logic for MainWindow.xaml
    /// </summary>
    public partial class MainWindow : Window
    {
        public static Emgu.CV.Image<Rgb, byte> img;
        public static Emgu.CV.Image<Gray, byte> vari;
        public static Bitmap mybmp;

        public MainWindow()
        {
            InitializeComponent();
        }

        private void button_Copy_Click(object sender, RoutedEventArgs e)
        {
            OpenFileDialog opf = new OpenFileDialog();
            if (opf.ShowDialog() == true)
            {
                Mat myimg=CvInvoke.Imread(opf.FileName,Emgu.CV.CvEnum.LoadImageType.Unchanged);//load IMG from file
                mybmp = myimg.Bitmap;
                img = myimg.ToImage<Rgb,byte>();
                BitmapImage bitmap = new BitmapImage();
                bitmap.BeginInit();
                bitmap.UriSource = new Uri(opf.FileName);
                bitmap.EndInit();
                image.Source = bitmap;
            }
        }

        public double scaleBetween(double unscaledNum,double minAllowed,double maxAllowed,double min,double max)
        {
            return (maxAllowed - minAllowed) * (unscaledNum - min) / (max - min) + minAllowed;
        }

        private void button_Copy_Click_1(object sender, RoutedEventArgs e)
        {
            vari = new Image<Gray, byte>(img.Width,img.Height);
            double min = 255;
            double max = 0;
            for (int i=0; i<img.Height;i++)
            {
                for (int j=0;j<img.Width;j++)
                {
                    double delitel = (img[i, j].Green + img[i, j].Red - img[i, j].Blue);
                    double val = (img[i, j].Green - img[i, j].Red) / delitel;
                    if (val > 1)//handle outlayers,extremes
                        val = 1;
                    if (val < 0)
                        val = 0;
                    if (val < min)//keep min and max for scaling afterwards
                        min = val;
                    if (val > max)
                        max = val;
                    vari[i, j] = new Gray(val);//VARI index = (green-red)/(green+red-blue)
                }
            }
            double diff = max - min;
            for (int i = 0; i < img.Height; i++)
            {
                for (int j = 0; j < img.Width; j++)
                {
                    double v = scaleBetween(vari[i, j].Intensity, 0, 255, min, max);
                    vari[i, j] = new Gray(v);
                }
            }
            //bitmap to bitmapSource for GUI show
            var bmp = vari.ToBitmap();
            using (MemoryStream memory = new MemoryStream())
            {
                bmp.Save(memory, ImageFormat.Png);
                memory.Position = 0;
                BitmapImage bitmapImage = new BitmapImage();
                bitmapImage.BeginInit();
                bitmapImage.StreamSource = memory;
                bitmapImage.CacheOption = BitmapCacheOption.OnLoad;
                bitmapImage.EndInit();
                image_Copy.Source = bitmapImage;
            }
        }
    }
}

Credits

Filip Mitrev

1 project • 0 followers

Comments

Awards

Bonus Prizes

NXP HoverGames Challenge 2: Help Drones, Help Others During Pandemics

Land surveying for crop yield optimization