Hackster is hosting Hackster Holidays, Ep. 6: Livestream & Giveaway Drawing. Watch previous episodes or stream live on Monday!Stream Hackster Holidays, Ep. 6 on Monday!
Sara Olsson
Published © GPL3+

TinyML and IoT for Conservation Efforts

Beyond historical insights, develop an IoT solution to monitor threatened mammals as elephants with insights based on TinyML in realtime.

IntermediateFull instructions provided16 hours2,339
Top 5 Machine Learning Models
ElephantEdge
TinyML and IoT for Conservation Efforts

Things used in this project

Hardware components

OpenMV Cam M7
OpenMV Cam M7
Optional
×1
Smartphone
Optional
×1

Software apps and online services

Edge Impulse Studio
Edge Impulse Studio
Avnet IoT Connect

Story

Read more

Custom parts and enclosures

Camera Trap model

Along with label.txt file with content:
buffalo
elephant
rhinoceros
zebra

Schematics

OpenMV system

Collar system

Code

iotconnect_upload.py

Python
Simple python script to create MQTT connection to IoTConnect and publish data to simulate a device. This script is indended to be extended to your solution.
import paho.mqtt.client as mqtt
import ssl
import threading
import json

# TO SET
# all found under device - connection info in IoTConnect
hostname = "poc-iotconnect-iothub-eu.azure-devices.net"
usename = ""
password = ""
pub_topic = ""
sub_topic = ""
client_id = "YYY"


# convert into JSON:
# set data "d" to match template properties on IoTConnect 
def generate_payload(test_offset):
      
    # SET cpId to client_id
    json_data = {
      "cpId": "YYY",
      "dtg": "c1431350-6607-40c9-8795-4e9cdbbc87f4",
      "t": "2020-10-23T17:35:44.000Z",
      "mt": 0,
      "sdk": {
        "l": "M_C",
        "v": "2.0",
        "e": "Avnet"
      },
      "d": [
        {
          "id": "esp321",
          "tg": "",
          "dt": "2020-10-21T17:35:44.000Z",
          "d": [
            {
              "Humidity": 13 - test_offset,
              "Temperature": 14,
              "vibration": {
                "x": 11 + test_offset,
                "y": 22 + test_offset,
                "z": 33 + test_offset
              }
            }
          ]
        }
      ]
    }

    # convert into JSON:
    payload = json.dumps(json_data)
    return payload

# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))

    # Subscribing in on_connect() means that if we lose the connection and
    # reconnect then subscriptions will be renewed.
    client.subscribe(sub_topic)

# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
    print(msg.topic+" "+str(msg.payload))

client = mqtt.Client(client_id=client_id)
client.on_connect = on_connect
client.on_message = on_message
client.tls_set(ca_certs=None, certfile=None, keyfile=None, cert_reqs=ssl.CERT_REQUIRED,
    tls_version=ssl.PROTOCOL_TLS, ciphers=None)
client.username_pw_set(usename, password=password)

client.connect(hostname, 8883, 60)

test_offset = 0

def sendSimulated():
    global test_offset
    threading.Timer(5.0, sendSimulated).start()

    payload = generate_payload(test_offset)
    test_offset += 0.5
    
    client.publish(pub_topic, payload=payload, qos=0, retain=False)
    print("publish data")

sendSimulated()

# Blocking call that processes network traffic, dispatches callbacks and
# handles reconnecting.
# Other loop*() functions are available that give a threaded interface and a
# manual interface.
client.loop_forever()

openmv_inference.py

Python
Run image inference on an OpenMV device and send results to IoTConnect.
Don't forget to add your trained.tflite + labels file to the SD card
# Use this script to run a TensorFlow lite image classifier

import sensor, image, time, tf, network, json
from mqtt import MQTTClient

# TO SET
SSID=''
KEY='!'

sensor.reset()
sensor.set_pixformat(sensor.GRAYSCALE)
sensor.set_framesize(sensor.QQVGA)
sensor.skip_frames(time = 2000)

# TO SET
# all found under device - connection info in IoTConnect
hostname = "poc-iotconnect-iothub-eu.azure-devices.net"
username = ""
password = ""
pub_topic = ""
sub_topic = ""
client_id = ""

print("Trying to connect to wifi (may take a while)")
wlan = network.WINC()
wlan.connect(SSID, key=KEY, security=wlan.WPA_PSK)
print(wlan.ifconfig())

# DOCS: https://github.com/pycom/pycom-libraries/blob/master/examples/mqtt/mqtt.py
client = MQTTClient(client_id, hostname, port=8883, user=username, password=password, keepalive=60, ssl=True)
print("connect mqtt")
client.connect()
print("mqtt connected")

clock = time.clock()

net = tf.load("trained.tflite", load_to_fb=True)
labels = [l.rstrip('\n') for l in open("labels.txt")]

def generate_payload(label, score):

    # SET cpId to client_id
    json_data = {
      "cpId": "",
      "dtg": "c1431350-6607-40c9-8795-4e9cdbbc87f4",
      "t": "2020-10-23T17:35:44.000Z",
      "mt": 0,
      "sdk": {
        "l": "M_C",
        "v": "2.0",
        "e": "Avnet"
      },
      "d": [
        {
          "id": "OpenMV1",
          "tg": "",
          "dt": "2020-10-21T17:35:44.000Z",
          "d": [
            {
              "Class": label,
              "Score": score
            }
          ]
        }
      ]
    }

    # convert into JSON:
    payload = json.dumps(json_data)
    return payload

while(True):
    clock.tick()

    # Take a picture and brighten things up for the frontal face detector.
    img = sensor.snapshot().gamma_corr(contrast=1.5)

    # Classify the image and get the class scores list
    scores = net.classify(img)[0].output() # roi=f

    # Find the highest class score and lookup the label for that
    label = labels[scores.index(max(scores))]


    if max(scores) > 0.70:

        score = '%.2f' % max(scores)

        # Draw the label above the box
        if label == 'elephant':
            img.draw_rectangle([10, 10, 140, 100], 255) # adjust to image size
            img.draw_string(15,10, label, 255, mono_space=False)

        summary = label + ' detected. Score: ' + score
        print(summary)

        payload = generate_payload(label, score)
        client.publish(pub_topic, payload)

Credits

Sara Olsson

Sara Olsson

2 projects • 9 followers
System developer with an extra interest in technical solutions for wildlife conservation

Comments

Awards
Top 5 Machine Learning Models
ElephantEdge
Related channels and tags
  • Animals
  • Animals
  • Avnet
  • Edge Impulse
  • Machine Learning & AI
  • Machine Learning & AI
  • OpenMV