Illegal Logging Detector

Introduction:

The environmental effects of illegal logging include deforestation, the loss of biodiversity and the emission of greenhouse gases. Illegal logging has contributed to conflicts with indigenous and local populations, violence, human rights abuses, corruption, funding of armed conflicts and the worsening of poverty.

"The World Bank estimates that governments worldwide lose between US$ 10 billion and 15 billion each year as a result of illegal logging ." (1)

While other people and countries may think that it only affects the producing countries, long-term climatic, economic, and environmental problems will affect most of the countries in the world. One example is the illegal logging in the Amazon forest, which is projected to lessen rain in northern China and the Midwest US while increasing rain in Northern Europe and Eastern Africa, both with disastrous consequences.

We can find that just in Mexico (my home country) [1] seventy percent of the wood consumed is of illegal origin according to a study carried out by one of the most prestigious universities UNAM (QS ranking # 100).

Read more at:

[1] Illegal logging | WWF (panda.org)

And of course in these last few weeks this has happened:

So, between the illegal aspect and the uprising prices we have to find a solution to this problem.

Most solutions are based on raising awareness, but looking at more dedicated solutions we can find:

• TreeTAG is an emerging smartphone-based supply chain traceability system developed by Earth Observation Systems that tracks the location of logs transported from the forest to the mill.
• Disadvantages: Very complex system that requires authorized personnel to be manipulated.
• Stardust is a dust-like material that can be sprayed onto wood and detected only with a hand-held device.
• Disadvantages: You need to tag manually every tree which is labor intensive and expensive..

Solution:

I’ll create a system that is capable of recognizing, through Machine Learning the sounds generated by falling trees, chainsaws and human voices in protected areas, thus warning that illegal logging may be occurring.

I especially want a system that can help protect forests and the species that inhabit them.

The system, will be easily reproducible, energy efficient and powerful thanks to the ML algorithms that will be implemented combined with the cloud services that we will use for deployment.

With the Infineon IM69D130 PDM digital microphone included in the QuickFeather Development Kit, I will obtain an audio signal which, through SensiML, we can pass through a neural network. That will tell us if the noise of a saw cutting the trees or human voice in the forest.

Displaying the information of the events detected in a simple Web App, together with a map which will indicate the position of the event.

Some of the features that are wanted are:

• Low-power battery consumption (QuickFeather and LoRaWAN).
• High accuracy (thanks to SensiML).
• Easy production at large scale, due to its simplicity.

Backend Diagram:

We will use a TTN LoRaWAN gateway to connect the solution and a ST microcontroller that I happen to have that already have a LoRaWAN shield, that will be joined with the QuickFeather in the following way:

From there I will send the information to the cloud (AWS and SensiML) and from there to my own web application.

Capturing Data:

In this section I will explain how to measure the data from the QuickFeather to the Data capture lab.

Setting up the QuickFeather:

First we have to configure the QuickFeather in data capture mode, in the Capture Project folder I leave the complete project with all the changes made to capture data.

From the official documentation you can see all the steps to setup the development environment. https://github.com/QuickLogic-Corp/qorc-sdk

In case you only need to start capturing the data, I leave the compiled binary so that you can flash your QuickFeather with the program and be able to capture data quickly. Here is a small demo:

In the case of my project, the easiest thing was to record the sound of several chainsaws in order to properly train the model.

NOTE: The captured audios will be in the SensiML Project folder.

Illegal-Logging-Detector/SensiML Project at main · AlexSanch/Illegal-Logging-Detector (github.com)

Labeling Data:

In this case, I did the labeling of the following categories for my model.

The system is capable of detecting the sound of chainsaws, humans and neutral silence, in order to avoid false alarms of the system.

SensiML:

These were the specifications for the data in SensiML.

These were the specs of the model's training.

Here the results of the precision of the model against the data used.

And finally the specifications of the compiled binary that is in the repository.

Testing the Model:

In this case you can see in the video how the model works correctly for the detection of human voice and detection of a Chainsaw. We are doing this test by seeing the QuickFeather serial output.

NOTE: the serial output is at 460800 baud rate, the board B-L072Z-LRWAN1 perfectly reaches these serial frequencies.

Serial Interface:

In order to send the results of the model to the B-L072Z-LRWAN1, we connect the TX pin of the board, which has a serial output at 460800 baud rate to the D2 pin as shown in the diagram.

Finally solder everything in a breadboard to avoid failures when using cables or jumpers.

LoRaWAN Module:

In this section we will explain all the details of the data transmission since we send the QuickFeather data by serial to the B-L072Z-LRWAN1, until it reaches AWS IoT.

Hardware:

The hardware used for this module was a B-L072Z-LRWAN1 and the X-NUCLEO-IKS01A3 sensor combo.

B-L072Z-LRWAN1.

• https://www.st.com/en/evaluation-tools/b-l072z-lrwan1.html
  

X-NUCLEO-IKS01A3.

• https://www.st.com/en/ecosystems/x-nucleo-iks01a3.html

Software:

The board software will be in the Serial2Lora_STM32L0 folder where the Arduino IDE project will be.

Configure the credentials in the TTN creds.h file

(details of these credentials in the TTN section).

static const char *appEui  = "XXXXXXXXXXXXXXXX";
static const char *appKey  = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX";
static const char *devEui  = "XXXXXXXXXXXXXXXX";

Set the correct frequency of LoRaWAN

US915 for North America.

#define REGION_US915

Serial to LoRaWAN Processing

if (Serial1.available()) {
  String temp = Serial1.readString(); // Reading Serial

  // Processing serial read to get classification only

  int checkpoint1 = temp.indexOf(":", 25);
  int checkpoint2 = temp.indexOf(",", 25);
  temp = temp.substring(checkpoint1 + 1, checkpoint2);

  // Append classification in lora message

  my_string = temp + "," + String(int(humidity)) + "," + String(int(temperature)) + "," + String(int(pressure));
  uint8_t payload[my_string.length() + 1];
  my_string.getBytes(payload, my_string.length() + 1);

  // Send to TTN 

  sendResult(payload, my_string.length() + 1);
  delay(1000);
}

TTN:

To communicate the device with TTN, we first need to be in range of the TTN gateways.

Check coverage at: https://www.thethingsnetwork.org/map

If you are in coverage, first we have to create a TTN application as the official documentation says:

https://www.thethingsnetwork.org/docs/applications/add/

Now that we have our app we must register our device to obtain the credentials.

https://www.thethingsnetwork.org/docs/devices/registration/

TTN to AWS IoT:

My first attempt to communicate TTN with AWS IoT was through the official TTN documentation. However, its integration is not updated as of today 05/04/2021.

https://www.thethingsnetwork.org/docs/applications/aws/

In this case I decided to do my own integration through the following AWS services.

TTN MQTT to AWS API Gateway :

Since I want this to be a reproducible integration to any system, I decided to make an integration based on the NodeJS container, this container is in the AWS Integration\Container folder.

Illegal-Logging-Detector/AWS Integration/Container at main · AlexSanch/Illegal-Logging-Detector (github.com)

In a simple way, the container receives all the data that the TTN app receives through the MQTT API and sends everything to its own API as request.

client.on('message', function (topic, message) {
    console.log(message.toString())
    unirest('POST', data.myAPIurl)
        .headers({
            'Content-Type': 'application/json'
        })
        .send(message.toString())
        .end(function (res) {
            if (res.error) throw new Error(res.error);
            console.log(200);
        });
})

To deploy this container on AWS use the ECR service to upload the container to AWS and ECS to deploy it. However, you can deploy it on your computer through Docker without any problem.

NOTE: The code will be fully commented on what it is doing.

AWS API Gateway to AWSLambda :

Once the message reached AWS API Gateway, we have to perform some action with it, for this we deploy a Lambda code in the API to redirect the complete message to AWS IoT.

AWS Lambda to AWS IoT :

The program that is executed when the message arrives from the container is the following. Also the code is in the AWS Integration \ Lambda folder.

import json
import boto3

client = boto3.client("iot-data")

def lambda_handler(event, context):
    response = client.publish(
            topic="ttn/echo",
            qos=1,
            payload=json.dumps(event["body"]))
    return {
        'statusCode': 200,
        'body': json.dumps('TTN Correct!')
    }

All the messages that we send from our device, by TTN to AWS IoT, will reach the topic ttn/echo.

Backend DEMO:

Here is a demonstration of the entire backend running at the same time.

Power Consumption:

For the project it is very important to see the energy consumption of our device, so I decided with the Nordic Power Profiler Kit II, to do an analysis of the mAh consumed by the device.

The complete initialization of the entire device is approximately 40 seconds with an average consumption of 45mA and peaks of 132mA.

However, what interests us is the long-term power consumption, so analyzing the consumption after the initialization, we find a stable zone of 48.7 mAh.

So the LoRaWAN module only adds 10mAh to the consumption of the device.

Web Page Deploy:

The deployment of the web page was done using ReactJS, OpenLayers (Maps) and AWS-SDK for javascript.

Desktop:

Mobile:

Here it is if you want to look at it:

https://illegal-logging-detector.s3.amazonaws.com/index.html

AWS Cognito:

For security, to safely use and consume AWS services, identity pool credentials were implemented with the Cognito service.

The access keys for AWS IoT and Cognito must be placed in the following file.

Webapp/src/components/aws-configuration.js

var awsConfiguration = {
  poolId: "us-east-1:XXXXXXXXXXXXXXX", // 'YourCognitoIdentityPoolId'
  host:"XXXXXXXXXXXXXX-ats.iot.us-east-1.amazonaws.com", // 'YourAwsIoTEndpoint', e.g. 'prefix.iot.us-east-1.amazonaws.com'
  region: "us-east-1" // 'YourAwsRegion', e.g. 'us-east-1'
};
module.exports = awsConfiguration;

AWS IoT Web Socket:

The web page receives the sensor data through AWS IoT as a web socket, so it is important to define within the page, which is the topic that we are going to receive, in this case "ttn/echo" as we could see in the video of backend.

Illegal Logging Detector - Hackster.io

In the following file, put the name of the topic to which you will be subscribed.

Webapp/src/App.js

<IotReciever sub_topics={["ttn/echo"]} callback={this.callBackIoT} />

Decode IN Data :

Because the data we receive from TTN is with a base64 encoding, we need to decode it with the following code in the Web App.

let payload = atob(temp["payload_raw"]).replace(' ', '').split(",")

This performs a conversion like the one we see in the image.

The array is made up of 4 pieces of information:

• Result of the neural network.
• Relative humidity of the environment.
• Temperature in degrees Celsius (converted to Fahrenheit on the page)
• Atmospheric pressure in mmHg.

The finished product:

Epic Final Demo!:

IMDb: "MPAA: Rated R for terror, violence, and disturbing images."

Conclusion

This is one example of what can be accomplished with the technology stack here presented combining a powerful microcontroller in the QuickFeather, a powerful AI application such as SensiML and the cloud. The problem regarding illegal logging is huge, but we can start solving this problem by expanding our "brain" to the physical world in this case of the forest. By having better and intelligent ways to detect problematic areas we can expand our actions to quench it.