Monitoring Stress In Plants

At present food waste due to plant stress and disease has a significant impact on crop yield and the sustainability of practices in agriculture. Stress in plants can have significant impact on the health and overall yield of a plant. Early identification of stress in Controlled Environmental Agriculture (CEAs) growth plants will enable growers to adjust and optimize plant growing conditions for maximum crop quality, yield and nutritional value whilst reducing food waste and improving energy efficiency.

At present research focusing on how we can reduce or reverse the impact of stress in CEA-grown crops is limited. Further research is needed to deepen our understanding of the complex signaling networks used by plants to respond to stress in these environments. The early identification of stress-traits are particularly important and understanding ‘how much stress is too much’ will enable growers to adjust and optimize plant growing conditions to combat stress, ensuring maximum crop quality, yield and nutritional value whilst reducing food waste and improving energy efficiency.

To aid this line of research purpose-built chambers which house phenotyping equipment would be a significant asset. However, outsourced equipment of this nature is extremely expensive and can occupy large areas of valuable space in laboratories. Therefore, limiting the accessibility of these types of technology to many research groups and early career researchers where funding and space is limited.

We aim to development of a low-cost CEA chamber with built-in stress monitoring systems to aid the furthering investigation of these key signalling events with the aim of improving the productivity and sustainability of CEA grown crops.

Device development

There are four main features of the device:

• RGB and IR cameras
• Temperature control
• Controllable grow lights
• Datalogging throughout

Each of these features requires its own systems and tech, and each had its own problems to implement.

System overview

The device is made up of two chambers: one in which to grow and monitor plants, and the other to house electronics.

The growth chamber features controllable grow lights, a thermal plate and various sensors (temperature, humidity, light). The plants are grown in petri-dishes on top of a labjack so they can be put in focus of overhead cameras. The door is sealed with closed-cell foam and a magnetic latch.

The electronics chamber houses a Raspberry Pi (RPi), an Arduino, a Peltier heatpump, two cameras viewing the plants and various control circuits and power supplies.

The RPi controls the cameras and also the arduino. The arduino controls the lights and Peltier, and receives data from the sensors, transmitting it back to the RPi when required.

During a test, the temperature and light intensities of different colours can be varied, and data and photos can be logged at specified time intervals throughout (down to every 10 seconds).

As data is gathered, it is moved in real time from local storage to the cloud (Google Drive at the minute). Once on the cloud, images are prepared and run through a premade AraDEEPopsis machine learning model to identify leaf features and other measures.

Datalogging and control

A Raspberry Pi 3B+ was chosen to be the 'central computer'. These are stripped-down low-cost computers that can be used to run custom programs or interface directly to hardware. The RPi models 3 and above also have inbuilt WiFi, so connecting to the cloud did not require any additional hardware. A serial protocal was developed to allow the RPi to generate requests and receive data from the Arduino while running tests.

The Arduino Uno was chosen to interface with the sensors and electronic components as it has superior electronic interfaces to the RPi, and would relieve some of the high-speed processing load (for controlling advanced electronics) from the RPi, without incurring too much additional cost.

Code for the RPi to run tests was written in python 3.9.1. Python code was also written to interface with the Lepton 3.5, and decode the raw output files into JPEG image files. Code for the arduino was written in C++ (including special arduino functions).

During a test, the RPi creates a new set of folders, and repeatedly writes images and data into those folders. The new data being saved is automatically detected by another system, which uploades the new folder-structure and data contained therein to Google Drive (and deletes the data from the RPi once it is uploaded).

Cameras

To monitor the plants as they grew and responded to stimuli, images would be repeatedly taken both in the visible spectrum and in the mid-IR spectrum.

The Raspberry Pi Camera V2 was chosen as a low-cost, high-resolution solution for the visible-light images, as it would easily connect to the RPi and had well documented software.

The Lepton 3.5 was chosen for the IR camera due to its low-cost, adequate resolution (160x120) and telemetry capabilities (it can calculate temperatures based on its pixel values). However, this camera did require an additional circuit-board to operate it: to save cost, the developer breakout board V2 was chosen (as opposed to the more expensive but perhaps simpler-to-use PureThermal board by Groupgets).

These two cameras would be controlled directly by the RPi using the PiCamera module and FLIR provided drivers.

Temperature control

Simple DHT11 sensors were chosen to monitor the temperature and humidity inside the chamber. These sensors are fairly imprecise (±1°C and ±1%), but adequate for monitoring the ambient temperature and humidity.

To reach temperatures of at most ±15°C from ambient, a lightweight Peltier module was chosen due to its compact size and simple operation, and ability to both heat and cool. This means in operation, it will have a 'hot side' and a 'cold side' - which side is which depends on the direction of the current provided. It can therefore heat an object, or if the current is reversed; cool it. A H-bridge (a grid of four electronic switches that can control the direction of DC current) was constructed and used to control the Peltier from the Arduino.

No fans could be used inside the chamber, as circulation might have interfered with the plant responses or disturbed them while pictures were being taken. Therefore to provide cooling, a wide metal sheet (thermal plate) was mounted to the inside roof of the growth chamber which was heated or cooled by the Peltier, thereby heating or cooling the chamber passively. This plate would also act as heatsink for the grow-lights (that would get hot in operation), allowing them to be cooled at the same time as the chamber.

The RPi would dictate what temperature the chamber should be at, but the Arduino would handle controlling the Peltier with a simple hysteretic control algorithm.

Grow lights

The grow lights needed provide light of various wavelengths (colours), each of which that could be controlled independently. LEDs were chosen as they have narrow wavelength outputs and high efficiency. As the chamber would be opaque to block external light, these would have to be powerful enough to provide for the plants completely. Regular hobby LEDs have a power capacity of around 100mW or less - to grow plants would require light sources of around 40W (around 400x more powerful).

It is possible to build an array of 400 small LEDs, which might provide more customisation regarding wavelengths, but the simple alternative was to buy a high-power LED array (or two). To avoid damaging these arrays, they were powered with a constant-current LED driver, which would provide exactly the current the LEDs required.

Each colour LED required the same current, but a different voltage. By choosing an LED driver with a range of output voltages encompassing all the different LEDs a single type of driver could be used for all the LEDs.

Many control schemes were devised to operate each LED colour individually, but ultimately having a seperate dimmable LED driver for each wavelength proved most effective. A dimmable driver can use pulse width modulation (PWM) to control the supplied current (and therefore brightness) based on a separate electronic signal.

The dimmable drivers we required (48V, 700mA) were typically >£20 each. However, a custom driver based on the ILD6070 chip ended up costing ~£4 each. This did require soldering the circuit by hand, but for a small number of drivers this was not too arduous, and had good results.

Nitty-gritty

Lepton interface

To get the Lepton 3.5 working, this guide for the FLIR Breakout board V2 was followed, but to actually get it working required using the kernel version 4.14.71-v7+ rather than the most up-to-date. Additional packages were also required: Bison, Flex and libssl-dev. These can be installed by running the following commands in the RPi terminal.

sudo apt-get install bison

sudo apt-get install flex

sudo apt-get install libssl-dev

A fully functioning OS image for the RPi with all these steps completed should be provided, saving you much of the hassle. To install this, flash the RPi SD card with the image using another computer, just as you would to install Raspbian for the first time.

There does not exist an easy solution to interfacing with the Lepton 3.5 breakout board (at time of writing). Libraries such as PiLepton sadly did not work, despite much prodding; and the FLIR Lepton SDK is very unfriendly, requiring developers to decode low-level protocols for SPI and I2C from technical datasheets.

Fortunately, at the very end of the getting started tutorial, there is a command that allows a user to take n pictures by running a precompiled C(?) program that comes with the drivers. This command is as follows:

./lepton_data_collector -3 -c 50 -o /tmp/capture/frame_

• lepton_data_collector is the name of the program
• -3 tells the program we are using a Lepton 3.x
• -c 50 Tells the program to take 50 pictures
• -o /tmp/capture/frame_ tells the program to save the images as frame_XXXXXX.jpg in the folder /tmp/capture/

The name of the image will always include six extra digits characterising which of the 50 images it is. Using the python os library, this command (or similar) can be run automatically by a program. Such a script was written to run the following command:

./lepton_data_collector -3 -c 1 -o /<data_location>/<img_name>_

which will (in a rather indirect way) take 1 picture (called <img_name>000000.jpg), and put it in a specified folder with a specified name. This indirect method is quite slow, taking roughly 1-2 seconds per image. This therefore limits the developed video-preview function to a rather dismal 0.5-1 fps, which works by repeatedly calling the above command.

Cloud storage and processing

Microsoft Sharepoint was the desired cloud storage solution (due to University affiliations giving large storage size), but the number of administrative and technical hoops to jump through was too great an obstacle. It was more straightforward to create a custom gmail account for the device, and use the complementary 15GB Google Drive storage that accompanies such an account.

When the device runs a test, it will create a new set of folders, uniquely named based on the date/time the first datapoint will be logged. This will typically be of the form YYYY-MM-DD-HH_MM_SS.

An icrontab script watches the Data Folder for any new folders or files to be created, and on creation triggers an rclone command to upload the new file or folder to the Google Drive in the same layout as on the RPi. Any files (.jpg, .png, .txt files) are then deleted from the RPi. Files with the extension .tmp are excluded from this, as data is continually written to a file of this type while a test is carried out. At the end of the test, a copy of the filled-out tmp file is saved as txt file (which is uploaded to the drive), and the tmp file is deleted.

Peltier heatpumps

A Peltier module is a type of heatpump, consisting of a flat grid of semiconductor junctions. When current flows through this grid, heat is transferred from one side of the module to the other. As Peltier modules are not perfect, passing any current through them will always generate additional heating as a byproduct. This means the efficiency of the module for cooling will decrease as more current is provided, as the byproduct heating eventually negates any cooling effect. Consult the Peltier datasheet for optimum current usage - as rules of thumb the cooling power you want should be 0.2x the maximum limit, and operating current should be 0.3x the maximum current limit of the Peltier.

Additionally, rapidly swapping the Peltier from hot to cold will cause swift degradation of the device, similar to repeatedly freezing a banana and then thawing it with a blowtorch. Along these lines, PWM should also be avoided, especially as using the Peltier at maximum current (albeit intermittently) is very inefficient - due to the self-heating effect explained above.

A simple control method would be to heat/cool with the Peltier until it reaches the desired target temperature, and then turn it off. As soon as the temperature falls below the target the Peltier would kick in again and boost it up a smidge, only for it to fall back again. This leads to the Peltier rapidly switching on and off around the target temperature, which is essentially like the PWM we were trying to avoid. To overcome this would be to introduce hysteresis. This means the rules for when to turn on the Peltier are not the same as when to turn off.

The above figures are an artist's rendition of how such control systems may operate. The simple method fluxuates very quickly about the target temperature making tiny adjustments; quickly wearing the Peltier out. The hysteretic method shows much slower oscillation, as the Peltier only turns off at the target temperature, and only back on at the lower boundary. A downside is this results in a broader variation in temperature - but in practice was at most ± 0.5 degrees.

LEDs and LED Drivers

LEDs work by current - each amp passing through an LED produces a certain amount of photons (light). However this current will only flow if you exceed an LED's threshold voltage. This threshold voltage means LEDs exist in two states. Below this voltage, they can be thought of as very large resistors that do very little. Above the threshold voltage they transition to being very small resistors, and if left unchecked will allow very large currents to flow. It is common to use constant-current drivers/power-supplies as they actively control the current they provide. This is as an alternative to the more common constant-voltage driver, such as a battery. The risk of using a constant-voltage driver is that at voltages a little higher than the threshold, most LEDs will draw so much current that they damage themselves. A constant-current driver with the right operating current can automatically find the sweet-spot between threshold and puff-of-smoke, and maintain the required current even if the LED warms up or begins to wear out over time.

Dimmable LED drivers use pulse-width modulation (PWM) to control the brightness of LEDs. Rather than adjust the finely-tuned current or voltages, they simply flicker the lights very fast.

Each cycle has two parts, one where the current flows and the lights are on, and one where no current flows and the lights are off. This typically happens so fast it is not even noticable to the human eye. By adjusting the ratio of these two durations, the light appears dimmer or brighter, as the overall proportion of time the lights stay on changes. This also directly controls the amount of power the lights emit (as power is an average over time), which is what plants care about.

We used an ILD6070 chip to construct custom LED drivers. The current value this chip outputs is specified by user-selected capacitors and inductors. It is somewhat daunting to work through the datasheet to produce these capacitor and inductor values, but Infineon provides suitable documentation to assist.

Post-mortum

Construction

The final system featured cameras and lights installed, but due to space contraints struggled to fit all the electronics in the allotted space. The lab-jack proved unstable, but adequate for crude height adjustment. Diffusers/guards will need to be installed covering the light arrays as the high-voltage terminals were exposued to users.

Guarding would also be required for the cameras, as they are exposed to the elements below. These guards would have to allow visible and/or mid-range IR light to pass through for the cameras to operate correctly.

Datalogging and control

The links between RPi and Arduino were set up and worked well. Logging of data to the Google Drive also worked. The device required a separate monitor, keyboard and mouse to control. Recipe files to run tests needed to be manually created.

Cameras

The Picamera produced clear photos and was easy to use. It suffered from image artefacts due to the PWM control of the lights - it could detect the flickering when an image was taken. It is possible that settings in the camera can be adjusted to rectify this.

The Lepton 3.5 breakout board was very difficult to work with, and ultimately killed the chamber Raspberry Pi by way of faulty cabling. Telemetry (temperature values) were not extracted from the images due to the restricted interface, though with more time could be.

Temperature Control

Temperature control methods worked well. In practice the heatpump was unable to effectively cool the lights at full power, but may be sufficient at <20% operation (~100umol/m^2/s). More testing is required to see what the limits of cooling temperature are.

Considerable development was required to design sufficiently high-amperage H-bridges for use with Peltier devices, a more streamlined though costly approach would be to purchase high power variable-current drivers, though these are not very common.

Grow lights

The lights and custom drivers worked adequately, producing up to 610umol/m^2/s at peak intensity. Heat dissipation became an issue at these intensities without fans present.

Followup

The next iteration of this device is found here: https://www.hackster.io/finnsutcliffe/monitoring-stress-in-plants-2-0-864aa9

Github page for both projects found here: https://github.com/AlexCalderwood/Biomaker_2020