MonSand: Monitoring Illegal Sand Mining

Sand is used in numerous applications such as construction (for cement) and microchips (for silicon). But the problem is that as the uses of this material are increasing, the supply is greatly decreasing.

The Problem

Now sand itself is very plentiful. In the Sahara Desert alone, 1.504 septillion grains of sand. But the sand in the Sahara Desert is very different from the sand that goes into the windshield of a Jeep Sahara.

Normal desert sand is smooth and round and doesn't bind together. This means that it can't be utilized in the production of glass used in the windshield of cars or the screens of our phones. Instead, we have to use river sand (wet sand) which is angular and easily sticks together.

We are running out of ways to legally mine wet sand without disturbing the coastline. So. in many third world countries, sand is being supplied to industry through illegal ways that are harmful to environment.

Government has only two ways to solve this problem of illegal sand mining. They can either go like this:

Or they can monitor and stop this environmental problem using MonSand.

The Solution: Overview

Illegal Sand Mining Detection by Vision AI

MonSand can take a photo of illegal sand mining:

And turn it into this:

Illegal Sand Mining Detection by Audio Volume

MonSand can take an audio of illegal sand mining machines operating:

And turn it into this:

Send Data to Google Sheets through Helium LoRa

After the audio and sound inferences are done, MonSand uses the power of Helium to send the data ("machine" or "no machine") to Google Sheets.

Send SMS from Google Sheets

Notify government & authorities of illegal mining via SMS

The Solution: How it works

I used this documentation (https://wiki.seeedstudio.com/Train-Deploy-AI-Model/) to train a YOLOv5s model on the following classes:

• Excavator
• Dump Truck
• Wheel Loader
• Canoe

I then deployed this model on the Wio Terminal + Grove Vision AI Module. I checked if any of the 4 classes were detected. If so, I printed "MACHINE CAMERA DETECTED".

After that, I detected the sound of machines by first recording the volume of the surrounding noise through the Wio Terminal's internal microphone (Seeed tutorial: https://wiki.seeedstudio.com/Wio-Terminal-Mic/). I then used an if-clause in the arduino code that sees if the volume is above a certain threshold. if this is the case, then it prints "MACHINE SOUND DETECTED".

I then use the Grove Lora E5 Module to send the data to Helium as mentioned in this Seeed tutorial: https://wiki.seeedstudio.com/Connecting-to-Helium/

Here is my helium flow:

My decoder function (purple):

function Decoder(bytes, port) {
    var decoded = {};
    
    function transformers(bytes){
      value = bytes[0] * 256 + bytes[1];
      return value;
    }
    
    if (port == 8) {
      decoded.model = transformers(bytes.slice(0,2));
      decoded.confidence = transformers(bytes.slice(2,4));
    }

    // TODO: Transform bytes to decoded payload below
    var decodedPayload = {
      "Number of Machines": decoded.model,
      "Mic Value": decoded.confidence
    };
    // END TODO
  
    return Serialize(decodedPayload)
  }
  
// Generated: do not touch unless your Google Form fields have changed
var field_mapping = {
  "Number of Machines": "entry.405136457",
  "Mic Value": "entry.1518740792"
};
// End Generated

function Serialize(payload) {
var str = [];
for (var key in payload) {
    if (payload.hasOwnProperty(key)) {
    var name = encodeURIComponent(field_mapping[key]);
    var value = encodeURIComponent(payload[key]);
    console.log("NAME ## " + name);
    console.log("VALUE ## " + value)
    str.push(name + "=" + value);
    }
}
return str.join("&");
}
// DO NOT REMOVE: Google Form Function

The Helium Google Sheets integration sends the Vision AI and Sound data to a Sheet from which I have an excel function that checks if sand mining machinery was detected from either a camera or sound and sends sms accordingly.

DIY: How to Build

To recreate monsand, all you have to do is the following steps. I followed these on my Windows 10 but it should work on Mac & Linux.

1. Clone my github repo to your computer: https://github.com/Raunak-Singh-Inventor/monsand

2. Open the monsand_lora_deployment arduino code in Arduino IDE.

3. Install the following libraries:

4. Install the Seeed SAMD Board Package

5. Go to Helium Console and create a device, use the generate_helium_keys script to create the different keys

6. copy the Device EUI, App EUI, & App Key into the respective variables in the arduino code

7. add a function with the function code mentioned above

8. add a google sheets integration

9. connect it as per the flow mentioned above

10. connect the Grove Vision AI module to your PC and load the model-1.uf2 file from the monsand github repo into the mounted drive.

11. Connect the Grove Vision AI module and Grove Lora E5 Module to the Wio Terminal

12. Upload the monsand_lora_deployment arduino code.

13. Follow this tutorial to connect the google sheets: https://wiki.seeedstudio.com/Integrate_into_Google_Sheets_via_Helium/

14. And see the output! :)

15. I used the SheetSMS extension of Google Sheets (https://sheetsms.com/) to text a message whenever illegal sand mining was detected. You can see the automated text message that was sent to my iPhone when it detected illegal sand mining here:

Hope this project will be one step closer to stopping illegal sand mining. Thanks for reading.

Future work:

• Showing sand truckload count for legal sand mining sites
• Building custom application in addition to google sheets integration