Droop There It Is: Smart Irrigation with Deep Learning

See what's new at Remyx.ai

See an application of our smart irrigation technology in kindbot

Introduction

We must meet the challenges of: climate change, an increasing population, the politics of water-access rights, and the need to produce crops in resource constrained environments, like space. Each scenario demands that we enforce greater constraints on water usage. We must become more efficient in our food production technologies to sustain our civilization.

Deficit Irrigation provides a framework for rationing water resources to produce crops by incorporating information on natural weather patterns and periods of drought tolerance in plant development for irrigation scheduling.

We aim to supplement this strategy by introducing a plant monitor supporting computer vision to detect drought stress and control irrigation with the additional context of transient effects in water use.

Combining this real-time monitoring on an embedded device, feedback from the plant is used to trigger supplemental irrigation by controlling a pump. Our smart irrigation pump offers the capabilities of reduced water consumption in crop production.

Learning to Identify Drought Stress

Typically, drought stress in plants manifest as wilting or droop due to decreased hydrostatic pressure. We train an image classifier to recognize features based on texture and pose, differentiating plants which are healthy from those experiencing drought stress.

To start, we use image search and web scraping to build an image corpus with examples from each class: healthy vs. drought stressed.

Try out a plugin for your favorite web browser to help retrieve image search results. This typically provides a couple hundred usable images, and you will want to manually filter irrelevant results. We tried multiple search engines as results will vary.

To take things further, you might target gardening websites and forums and build a script based on python's request library or even a more powerful scraper using scrapy.

After building a corpus with a couple thousand images or more, fine-tuning a pretrained image classifier is a straight-forward application of resources like tensorflow tutorials, TF-slim, or tensorflow-for-poets.

To localize and get counts of 'droopy objects', instead of training a classifier, consider object detection models like: darknet YOLO, which can be converted to tensorflow for faster inference on embedded hardware using darkflow. Alternatively, use tensorflow's object detection API to both train and run inference.

The benefit of an image classifier is it will be very easy to label data by simply filtering image search results to remove irrelevant images before applying a label related to your query. Additionally, image classifiers run very quickly.

On the other hand, counts and localization offer more information to develop the controller. However, an object detection model will require a more tedious hand labeling of your images, try using labelimg when you are sufficiently caffeinated. Object detectors will also run much more slowly, but that is okay for our application since plant droop will take place over time scales on the order of hours or days, depending on the grow medium.

We choose the latter to measure droop intensity with counts of droopy structures (leaves & limbs), although classifier label confidence may also offer a way to gauge intensity.

A Toy Experiment

To explore the hypothesis that droop detection can be applied to reduce water consumption, we set up a small trial. Needless to say, the scale of our home experiment may drive development but will not offer conclusive results. Rather, we view this as a preliminary test to determine how best to refine our irrigation mechanism.

We test the control of drip irrigation with two groups of tomato plants. Group A receives our experimental irrigation control and Group B is under a conventional, timer-based schedule.

The reasons to choose tomato seedlings are manifold:

• tomatoes are popular, cheap, and readily available in local hardware stores
• tomatoes are a small bushy plant, different than the species we gathered data on, offering an assessment of model generalizibility
• seedling root development, relative to vegetative mass leaves them more susceptible to drought stress, aiding the test of our controller

To precisely measure the volume of water for each irrigation event during the course of our experiment, we calibrate each peristaltic pump to deliver the same dose.

Using a relay strip, we control both pumps through GPIO with a raspberry pi.

All plants will be positioned inside a small grow space to control for ambient conditions like light intensity, air movement/temperature/humidity.

To ensure that wilting from our timer-based control group does not trigger irrigation events, we frame the camera view to avoid Group B.

For timer-based drip irrigation, we run a simple script using the raspbian crontab. We calibrated it by wiring everything together and measuring how many seconds it would take to fill a graduated cylinder up to 5 ml. Pro-tip: calibrate the pumps after they've been connected to the relays, otherwise slight changes in voltage will affect the flow rate.

For our experimental controller, we start with an assumption on the minimum required irrigation frequency which will also be schedule based. Then we supplement this base model triggering additional irrigation when the droop detected is anomalously high.

We waited for 2 days for the substrate to dry out some before giving the seedlings 10 mL of water twice daily.

We use a temperature controller to turn on/off an inline fan venting cooler outdoor air into the warmer grow tent space. A day is 18 hours long under a 315 ceramic metal halide lamp. See the temperatures drop from our target set point during the 6 hour night cycle in the chart below.

Color coding the points and disregarding the night cycles, we see how the controller regulates duty cycles. Here red means the fan is on and blue means the fan is off for a 3 minute cycle

We observe the counts of droop and plot as a time series.

Droop detection inference results can vary from one photo to the next. With simple ideas like low-pass filtering or moving medians, we use smoothing in time to trigger irrigation only when we are sufficiently confident that the plants are experiencing wilting.

Over the first 200 time steps, the count oscillates in time more smoothly. Consider instead, the variance over a rolling window, to see this dispersion more clearly.

After a couple days, the soil became quite dry. We've doubled the irrigation frequency to 10 mL 4 times daily.

Despite calibrating the pumps, we've found that over one week, one pump outpaced the other by roughly 6 tsp of water while using the same irrigation scheduling.

Looking at our time series, a simple idea could be to add an extra irrigation event on days like the 3rd and the 6th. Here, we see an appreciable spike in droop, or signs that the plant is experiencing reduced hydrostatic pressure.

Soil is dry and hard, increasing irrigation frequency to 6 times daily dosing of 10 mL. After a couple weeks in the grow space, the plants are acclimated and we have a baseline for timer-based irrigation frequency for the small plants.

Here is a video of our experiment over 2/1/19 - 2/23/19. We see the drought stress and droop early in the video. Over the middle of the video, the tomato plant hits a growth spurt before slowing due to over-watering. Notice the extra water flowing under the cup and the yellowing.

This still hasn't helped us to use less water than scheduled irrigation. We need to increase irrigation frequency until the plants are thriving and use clues like these to back off in frequency by choosing our irrigation timing with this information. Ultimately, we want to use less water without adversely affecting produce or vegetative development too much.

Analyzing the droop time series, we want to learn the best times to add a non-scheduled irrigation event. Similar to this work in anomaly detection, we attempt to learn the next droop count given a window of previous droop counts. Then at each time step, we can compare the predicted value with the observed count. When the difference is high enough, we will trigger an irrigation event. See our jupyter notebook.

Experimenting with predicting the next time series value, we can experiment with a threshold difference to schedule the irrigation events near times the plant shows exceptional drought stress. The droop count distribution drifts in time and so we should learn in on online fashion.

We take the approach of budgeting a few regularly (say 2-3 daily) scheduled irrigation events while allowing for additional events up to a limit over some time interval (something like: 2 in addition to the schedule and no more than 2 within a given hour).

Discussion

A more productized assembly would look/work like kindbot by controlling a submersible pump with consumer smart plugs via wifi. To control for as many sources of variation as possible in our simple test, we prefer the more precise irrigation delivery available with peristaltic pumps.

By exploring the parameters in this setup, we aim to develop new technology to build a more efficient irrigation pump for greenhouses. In scenarios where electricity is cheaper than water, this approach offers some promise to optimize food production for conserving precious resources.