Published October 18, 2020 © GPL3+

Elephant Activity Dashboard

This dashboard displays elephant activities and alerts certain types of activities.

IntermediateFull instructions providedOver 2 days422

Things used in this project

Software apps and online services

Edge Impulse Studio

IoTConnect

Node-RED

Story

In this project, I created a dashboard using IoTConnect platform to track elephant activities. Based on a Edge Impulse Studio tutorial, an activity classification model was built. The dashboard shows a time series of activities' classification of one of the elephants. Elephants with high activities will be alerted with red color indication.

Setup IoTConnect

I followed the instructions detailed at Hackster's ElephantEdge contest page. I created 2 templates and 3 devices. The 2 templates are Weather and Activity. One weather device called W1 which is used to receive temperature and humidity sensor data. While the other 2 devices are E1 and E2 which are used to track elephants' activity level and location information.

1 / 3 • List of devices

With IotConnect SDK, I created two apps to ingest data to IoTConnect Platform. One app is a Node.js console app which I modified from the SDK sample program. Another one is a Node-RED app running in the docker container.

1 / 3 • IoTConnect SDK

Setup Edge Impulse

I created an Edge Impulse account and followed the "continuous motion recognition" tutorial to build the activity recognition model. Then I followed the "through WebAssembly (Node.js)" tutorial to build the library so that I can use it in the Node.js environment.

1 / 2 • Edge Impulse project

New IoTConnect Dashboard

With new IoTConnect dashboard functionality, I made a new dashboard to track the elephants' activities and devices' location. The new dashboard shows 3 devices for tracking elephants' activities and 1 device for weather tracking. Since my trial IoTConnect plan allows only 3 active devices, I used 2 devices for activity tracking and 1 device for weather monitoring. The dashboard too shows the elephant activity chart, the devices' location and notification table.

New dashboard with elephants' activities and devices' location

The Demo

The top portion of the dashboard shows the 4 labels: Elephant 1 Activity, Elephant 2 Actviity, Elephant 3 Activity and Temperature. There is an activity chart Elephant 1 below showing the detected activities of Elephant 1. On the right panel are the device location map and notification panel. The activity labels show 4 types of activities. They are (1) minimal activities (resting), (2) low activities (feeding, bathing), (3) medium activities (walking), and (4) high activities (rapid walking). If the activities are of type high activity, the color of the label text will turn red.

The code section below explains how the data (simulated) come about.

1 / 2 • New dashboard

Code

There are 3 important NodeJs codes:

(1) The send_data_w1.js which send weather data of device W1 to IoTConnect. The data are in the form of:

{
uniqueId: 'W1',
time: 2020-10-18T10:25:06.931Z,
data: { temperature: '24.33', humidity: '24.52' }
}.

These values are generated randomly. The temperature values will be displayed on the dashboard.

(2) The send_data_e2.js that send device E2's activities level to IoTConnect. The data are in the form of activity level:

{
uniqueId: 'E2',
time: 2020-10-18T10:29:06.011Z,
data: { latitude: 8, longitude: 38, level: '3' }
}

These activity level values are generated randomly too. The values range from 1 to 4.

(3) The send_data_e1_files.js which send device E1's activities level to IoTConnect. The data have the form as (2) above:

{
uniqueId: 'E1',
time: 2020-10-18T10:40:54.590Z,
data: { latitude: 10, longitude: 10, level: 3 }

}

But the level values are classification values computed from the activity classification model built from the "continuous motion recognition" tutorial. The activities data are raw features recorded from accelerometer in the said tutorial. These values were read from files and fed into the classification model during the demo.

Summary

Obviously, this project uses simulation data in the demo. The next step will be using real life data to build the model and use that to classify the behavior of elephants.

'use strict';

var sdk = require('iotconnect-sdk');

//"Get your ENV and CPID from the portal key vaults module or visit https://help.iotconnect.io SDK section."
var cpId = "CPID from the portal key vaults module";
var uniqueId = "W1";
var env = "Avnet";
var intervalLimit = 60; //Time Interval in Seconds
var iotConnectSDK = "";
var curr_temperature = 25;

setTimeout(() => {
	if(!cpId || !uniqueId){
        if(!cpId)
			console.log("cpId can not be blank");
		if(!uniqueId)
			console.log("uniqueId can not be blank");
        process.exit();
    } else {
		iotConnectSDK = new sdk(cpId, uniqueId, callbackMessage, twinCallbackMessage, env);
		console.log("Initializing...");
	}
}, 500);

var callbackMessage = function callbackMessage(data){
    
    if(data != null && data != undefined && data.ack != undefined && data.cmdType != null)
    {
        if(data.cmdType == '0x01') {
            console.log("\n"+"--- Device Command Received ---");
            console.log(data.ack);
            console.log(data.ackId);
            console.log(data.command);
            console.log(data.uniqueId);
            var obj = {
                "ackId": data.ackId,
                "st": 6,
                "msg": "",
                "childId": ""
            }
            var mt = 5;
            if(data.ackId != null)
			    sendAck(obj, null, mt);
            
        } else if(data.cmdType == '0x02') {
            console.log("\n"+"--- Firmware Command Received ---");
            console.log(data);
            var obj = {
                "ackId": data.ackId,
                "st": 7,
                "msg": "",
                "childId": ""
            }
            var mt = 11;
            if(data.ackId != null)
			    sendAck(obj, null, mt);
        }
    }
    else
    {
        if(data.cmdType == null && data.command == "device_connected") {
            try {
                console.log("Device connected...", new Date());
            } catch (error) {
                console.log("Error while getting attributes :: ",error.message);
            }
        } else {
            //console.log(data);
        }
    }
}

var twinCallbackMessage = function twinCallbackMessage(data){
    console.log("\n--- Twin desired message received ---");
    console.log(data);
}


setTimeout(() => {
    setInterval(() => {
        curr_temperature = nextTemperatureValue(curr_temperature);
        var obj = [
            {
                "uniqueId":uniqueId,
                "time":new Date(),
                "data": {
                    "temperature": curr_temperature.toFixed(2),
                    "humidity": nextHumidityValue() 
                    //"Temp":"11"  // Your Device attributes
                }
            }
        ]
        console.log(obj)
        iotConnectSDK.SendData(obj)
    }, intervalLimit * 1000);
}, 10000);

var updateTwin = function updateTwin(){
    var key = "firmware_version";
    var value = "4.0";
    iotConnectSDK.UpdateTwin(key,value, function (response) {
        console.log("Twin update :: ",response);
    })
}

function disconnectDevice(){
    iotConnectSDK.disconnectDevice(function (response) {
        if(response.status){
            console.log("\nDevice disconnected :: ", new Date());
            process.exit();
        } else {
            console.log("Device disconnection :: ", response);
        }
    })
}

function sendAck(jsonObj, time, mt){
    iotConnectSDK.sendAck(jsonObj, time, mt, function (response) {
        if(response.status){
            console.log("\nCommand acknowledgement sent :: SUCCESS:: ", new Date());
        } else {
            console.log("\nCommand acknowledgement sent :: FAILED :: ", new Date());
        }
    })
}

function nextTemperatureValue(curr) {
  if (curr < 21) curr = 21.0;
  else if (curr > 30) curr += Math.random() - 1.5; 
  else curr += -0.75 + Math.random() * 2.5;
  return curr;
}

function nextHumidityValue() {
  let curr = (Math.random() * (27 - 24) + 24).toFixed(2);
  return curr;
}


function getRandomValue(min, max) {
    return (Math.random() * (max - min) + min).toFixed(0);
}

'use strict';

var sdk = require('iotconnect-sdk');

//"Get your ENV and CPID from the portal key vaults module or visit https://help.iotconnect.io SDK section."
var cpId = "CPID from the portal key vaults module";
var uniqueId = "E2";
var env = "Avnet";
var intervalLimit = 30; //Time Interval in Seconds
var iotConnectSDK = "";

setTimeout(() => {
	if(!cpId || !uniqueId){
        if(!cpId)
			console.log("cpId can not be blank");
		if(!uniqueId)
			console.log("uniqueId can not be blank");
        process.exit();
    } else {
		iotConnectSDK = new sdk(cpId, uniqueId, callbackMessage, twinCallbackMessage, env);
		console.log("Initializing...");
	}
}, 500);

var callbackMessage = function callbackMessage(data){
    
    if(data != null && data != undefined && data.ack != undefined && data.cmdType != null)
    {
        if(data.cmdType == '0x01') {
            console.log("\n"+"--- Device Command Received ---");
            console.log(data.ack);
            console.log(data.ackId);
            console.log(data.command);
            console.log(data.uniqueId);
            var obj = {
                "ackId": data.ackId,
                "st": 6,
                "msg": "",
                "childId": ""
            }
            var mt = 5;
            if(data.ackId != null)
			    sendAck(obj, null, mt);
            
        } else if(data.cmdType == '0x02') {
            console.log("\n"+"--- Firmware Command Received ---");
            console.log(data);
            var obj = {
                "ackId": data.ackId,
                "st": 7,
                "msg": "",
                "childId": ""
            }
            var mt = 11;
            if(data.ackId != null)
			    sendAck(obj, null, mt);
        }
    }
    else
    {
        if(data.cmdType == null && data.command == "device_connected") {
            try {
                console.log("Device connected...", new Date());
            } catch (error) {
                console.log("Error while getting attributes :: ",error.message);
            }
        } else {
            //console.log(data);
        }
    }
}

var twinCallbackMessage = function twinCallbackMessage(data){
    console.log("\n--- Twin desired message received ---");
    console.log(data);
}


setTimeout(() => {
    setInterval(() => {
        var obj = [
            {
                "uniqueId":uniqueId,
                "time":new Date(),
                "data": {
                    "latitude": 8,
                    "longitude": 38,
                    "level": getRandomValue(1, 4)
                    //"Temp":"11"  // Your Device attributes
                }
            }
        ]
        console.log(obj)
        iotConnectSDK.SendData(obj)
    }, intervalLimit * 1000);
}, 10000);

var updateTwin = function updateTwin(){
    var key = "firmware_version";
    var value = "4.0";
    iotConnectSDK.UpdateTwin(key,value, function (response) {
        console.log("Twin update :: ",response);
    })
}

function disconnectDevice(){
    iotConnectSDK.disconnectDevice(function (response) {
        if(response.status){
            console.log("\nDevice disconnected :: ", new Date());
            process.exit();
        } else {
            console.log("Device disconnection :: ", response);
        }
    })
}

function sendAck(jsonObj, time, mt){
    iotConnectSDK.sendAck(jsonObj, time, mt, function (response) {
        if(response.status){
            console.log("\nCommand acknowledgement sent :: SUCCESS:: ", new Date());
        } else {
            console.log("\nCommand acknowledgement sent :: FAILED :: ", new Date());
        }
    })
}

function getRandomValue(min, max) {
    return (Math.random() * (max - min) + min).toFixed(0);
}

'use strict';

const Module = require('./edge-impulse/edge-impulse-standalone');

const path = require('path');
const fs = require('fs');

const dirPath = path.join(__dirname, 'data');
const files = fs.readdirSync(dirPath);

console.log(files);

// Classifier module
let classifierInitialized = false;
Module.onRuntimeInitialized = function() {
    classifierInitialized = true;
};

class EdgeImpulseClassifier {
    //_initialized = false;

    init() {
        if (classifierInitialized === true) return Promise.resolve();

        return new Promise((resolve) => {
            Module.onRuntimeInitialized = () => {
                resolve();
                classifierInitialized = true;
            };
        });
    }

    classify(rawData, debug = false) {
        if (!classifierInitialized) throw new Error('Module is not initialized');

        const obj = this._arrayToHeap(rawData);
        let ret = Module.run_classifier(obj.buffer.byteOffset, rawData.length, debug);
        Module._free(obj.ptr);

        if (ret.result !== 0) {
            throw new Error('Classification failed (err code: ' + ret.result + ')');
        }

        let jsResult = {
            anomaly: ret.anomaly,
            results: []
        };

        for (let cx = 0; cx < ret.classification.size(); cx++) {
            let c = ret.classification.get(cx);
            jsResult.results.push({ label: c.label, value: c.value });
        }

        return jsResult;
    }

    _arrayToHeap(data) {
        let typedArray = new Float32Array(data);
        let numBytes = typedArray.length * typedArray.BYTES_PER_ELEMENT;
        let ptr = Module._malloc(numBytes);
        let heapBytes = new Uint8Array(Module.HEAPU8.buffer, ptr, numBytes);
        heapBytes.set(new Uint8Array(typedArray.buffer));
        return { ptr: ptr, buffer: heapBytes };
    }
}


// Initialize the classifier, and invoke with the argument passed in
var classifier = new EdgeImpulseClassifier();
var level;

function classify(i) {

let data = fs.readFileSync(path.join(dirPath, files[i]), 'utf8');

classifier.init().then(() => {
    //let result = classifier.classify(process.argv[2].trim().split(',').map(n => Number(n)));
    let result = classifier.classify(data.trim().split(',').map(n => Number(n)));

    console.log(result);

    console.log(...result.results.map(x => x.value));
    
    var a = result.results;
    var k = a.reduce((j, x, i, a) => x.value > a[j].value ? i : j, 0);
    console.log(k, a[k].label);

    level = k + 1;

}).catch(err => {
    console.error('Failed to initialize classifier', err);
});
}


///classify(3);
///classify(0);

//////

var sdk = require('iotconnect-sdk');

//"Get your ENV and CPID from the portal key vaults module or visit https://help.iotconnect.io SDK section."
var cpId = "CPID from the portal key vaults module";
var uniqueId = "E1";
var env = "Avnet";
var intervalLimit = 30; //Time Interval in Seconds
var iotConnectSDK = "";
var i = 0;

setTimeout(() => {
	if(!cpId || !uniqueId){
        if(!cpId)
			console.log("cpId can not be blank");
		if(!uniqueId)
			console.log("uniqueId can not be blank");
        process.exit();
    } else {
		iotConnectSDK = new sdk(cpId, uniqueId, callbackMessage, twinCallbackMessage, env);
		console.log("Initializing...");
	}
}, 500);

var callbackMessage = function callbackMessage(data){
    
    if(data != null && data != undefined && data.ack != undefined && data.cmdType != null)
    {
        if(data.cmdType == '0x01') {
            console.log("\n"+"--- Device Command Received ---");
            console.log(data.ack);
            console.log(data.ackId);
            console.log(data.command);
            console.log(data.uniqueId);
            var obj = {
                "ackId": data.ackId,
                "st": 6,
                "msg": "",
                "childId": ""
            }
            var mt = 5;
            if(data.ackId != null)
			    sendAck(obj, null, mt);
            
        } else if(data.cmdType == '0x02') {
            console.log("\n"+"--- Firmware Command Received ---");
            console.log(data);
            var obj = {
                "ackId": data.ackId,
                "st": 7,
                "msg": "",
                "childId": ""
            }
            var mt = 11;
            if(data.ackId != null)
			    sendAck(obj, null, mt);
        }
    }
    else
    {
        if(data.cmdType == null && data.command == "device_connected") {
            try {
                console.log("Device connected...", new Date());
            } catch (error) {
                console.log("Error while getting attributes :: ",error.message);
            }
        } else {
            //console.log(data);
        }
    }
}

var twinCallbackMessage = function twinCallbackMessage(data){
    console.log("\n--- Twin desired message received ---");
    console.log(data);
}


setTimeout(() => {
    setInterval(async () => {
        await classify(i);  i = (i+1) % files.length;
        console.log('level>>> ', level);
        console.log('i>>> ', i);
        var obj = [
            {
                "uniqueId":uniqueId,
                "time":new Date(),
                "data": {
                    "latitude": 10,
                    "longitude": 10,
                    //"level": getRandomValue(1, 4)
                    "level": level 
                    //"Temp":"11"  // Your Device attributes
                }
            }
        ]
        console.log(obj)
        iotConnectSDK.SendData(obj)
    }, intervalLimit * 1000);
}, 10000);

var updateTwin = function updateTwin(){
    var key = "firmware_version";
    var value = "4.0";
    iotConnectSDK.UpdateTwin(key,value, function (response) {
        console.log("Twin update :: ",response);
    })
}

function disconnectDevice(){
    iotConnectSDK.disconnectDevice(function (response) {
        if(response.status){
            console.log("\nDevice disconnected :: ", new Date());
            process.exit();
        } else {
            console.log("Device disconnection :: ", response);
        }
    })
}

function sendAck(jsonObj, time, mt){
    iotConnectSDK.sendAck(jsonObj, time, mt, function (response) {
        if(response.status){
            console.log("\nCommand acknowledgement sent :: SUCCESS:: ", new Date());
        } else {
            console.log("\nCommand acknowledgement sent :: FAILED :: ", new Date());
        }
    })
}

function getRandomValue(min, max) {
    return (Math.random() * (max - min) + min).toFixed(0);
}