AI based pest detection system for rooftop garden

Summary

Agriculture is one of the pillars of human civilization, but its industrialization has allowed pests to spread like never before. Pests destroy our crops in many ways:

• Eating leaves of crop plants
• Sucking out plant juices
• Boring into the leaves, roots and stems
• Spreading plant pathogens
• Feeding on natural fibers

These actions compromise wood (our most essential building material), contaminate stored crops and accelerate plant decay. As a result, about 40% of food crops fall to pests and pest-caused diseases every year.

This waste contributes to food shortages and harms poor rural communities the most. Of course, they also cause direct harm to humans through bites, stings, diseases and general annoyance.

Our solution

In this project, we are going to use our audio classification model for classifying flying pest or insect. It recognizes three kinds of insect or pest based on their sound such as mosquito, bees and flies. Throughout this project I will describe how to used edge impulse and Nordic thingy:53 to achieve this objective. Edge Impulse is Software as a Service (SaaS) web platform that allows developers to deploy powerful ML models to edge devices without possessing deep knowledge of programming or AI, ML concepts.

Here is a sneak peek of what's about to happen:

• Collecting audio sound for training
• Upload dataset to Edge Impulse to train the model.
• Feature Extraction
• Training of the model
• Deploy to Nordic Thingy:53
• Real-time data evaluation and testing using apps interface

Let’s get started!

Train the Model in Edge Impulse

Edge Impulse is a web-based tool that helps us quickly and easily create AI models that can then be used in all kinds of projects. We can create Machine Learning models in a few simple steps and users can build custom data classifiers with nothing more than a web browser.

As with any deep learning or machine learning task, the first most important task is to prepare the dataset. We prepare the dataset using Nordic Thingy:53 (since it has a microphone for capturing audio data). Capturing real mosquito sound is too much challenging. In this project we used freely available sound from internet source, youtube etc. as our dataset.

A laptop is used for playing those sounds and Nordic thingy:53 along with nRF Edge impulse (Mobile application) are used for capturing and forwarding data to edge impulse platform (web platform).

Edge Impulse account is mandatory for this project. If you don't have any just create one.

Now, we are going to use the nRF Edge Impulse mobile application for collecting and transferring data to the edge impulse.

Open the mobile apps and login to the edge impulse. Create a new project, put a name of the project.

Power up the Nordic Thingy :53 and connect it with your mobile application via Bluetooth.

Navigate to the data tab, click record new data, select category as training and label it as bees or flies or mosquito and choose sensor as microphone.

• Click Record New Data button for collecting sample data
• We need to repeat this process for several times for collecting several sample data.

The data is automatically uploaded to edge impulse (Mobile phone must be connected with Internet).

A short video tutorial is given below,

The data set has Three (03) classes. There are: Bees, Mosquito, and Flies. It would look something like this:

We are collecting very little number of data of each class which really effect the result. Here, around 13 samples of each class are taken for training.

After uploading the audio data for each classes we will be moving on to the splitting process. Any machine learning training procedure involves first splitting the data randomly into two sets. You can do it using the Perform train/test split functionality available in the Edge Impulse. Navigate to Dashboard section and select Perform train/test split. This will automatically split your data between a training class (80%) and a testing class (20%).

Once you have set up all of your classes and are happy with your datasets, it is time to train the model. Navigate to Impulse Design on the left navigation menu.

Select Add a processing block and add Audio (MFCC). MFCC stands for Mel Frequency Cepstral Coefficients. So, it will try to convert audio into some kind of features based on time and frequency characteristics that will help us to do classification. Then select Add learning block and add Classification (Keras). Click Save Impulse.

Then navigate to MFCC. Feature extraction is the process of discovering a value or a group of values that can be used to identify entities like objects, character, person, etc. MFCC is often used for speech recognition tasks.

Click Save parameters and finally press Generate Features.

Next, click on NN Classifier on the navigation menu. We will train a model for 500 epochs and learning rate as 0.005. Click the Start training button. Training the model can take a while. While training is in progress, you'll see a progress bar. Training should take around 5 to 6 minutes or less using the 500 epochs.

A short video is given below for better understanding.

When the training process completes you should have a result like the picture shown below:

Train data was passed through the first 1D-convolutional layer. One-dimensional vector is easy to use for classification purposes, and thereby it was used to flatten the data. The dropout layer was used to reduce the chance of overfitting.

We get a validation accuracy of 64.0%. Increasing the number of training epochs and dataset can potentially increase this accuracy score.

After the training is complete, you can transfer the model to the Nordic Thingy:53 using nRF edge impulse mobile apps.

Deploying the Trained Model to Nordic Thingy53

The given video shows, how we transfer the machine learning model to the Nordic Thingy53 board using nRF Edge Impulse mobile apps.

Testing the model

Use case scenario

The pest detection system trigger the pest control spray or buzzer when it detect any pest or insect.

Future work

We plan to integrate multiple machine learning models to monitor the health of rooftop garden plants.

Reference

• https://docs.edgeimpulse.com/docs/
• https://www.nordicsemi.com/Products/Development-hardware/Nordic-Thingy-53
• https://www.agrilinks.org/post/reducing-health-risks-pests-agriculture