EleTect - TinyML and IoT Based Smart Wildlife Tracker

Problem Statement:

Man’s relationship with nature has changed drastically over the last few centuries. For thousands of years we have lived in harmony with nature, taking only what was needed to survive. As humanity grew in numbers and as our consumption of resources steadily increased, we needed to innovate and efficiently use the scarce resources at our disposal.

The rate that we consume these resources has rapidly increased since the 20th century. Technology has allowed humanity to work less and gain more for the same amount of time and energy. Unfortunately our society no longer only takes from nature what it requires, but instead over consumes and depletes the already limited resources. The International Union for Conservation of Nature (IUCN) estimates that half of the 5, 491 known mammal species populations are declining, with 1, 131 classified as endangered, threatened or vulnerable.

“The threat to elephants is greater than it’s ever been. Particularly because the economies of Asian countries, especially China, have grown exponentially. Ivory is part of Chinese culture and history – it’s a commodity that indicates a certain status. If we’re serious about saving a species as important and as symbolic as the elephant, then we’ve got to bite the bullet and say, ‘We don’t need ivory.’ It’s complete and utter nonsense to say, ‘We need it.'” – Richard Leakey

One example of this is the declining elephant population in Africa, primarily due to the large demand for their tusks. As is the story with many other species of wildlife, humanity is decimating the remaining population in pursuit of profit. There is still a strong demand for ivory on the international market, especially in China, where the price of ivory peaked at $2100 per kg in 2014. The demand for ivory is fuelled by the association of wealth and status, and the ascribed medical benefits it is believed to have. Governments are stepping in and imposing strong sanctions on the open trade of ivory, but there is still a strong belief among many potential buyers that the global elephant population is stable and not at risk of extinction.

Contrary to that belief, the Ivory Demand In China report released by WildAid reports that at least 65% of all African forest elephants were poached between 2002 and 2013. One way to fight this ignorance and intent to buy ivory is to gather factual information that can be used to educate consumers. Awareness is spreading, but there is a lot of uncertainty regarding the size, distribution and trend of the elephant populations in Africa.

A lot of the countries in Africa have outdated, inaccurate or non-existent census data, making it impossible to accurately estimate the state of the elephant population. The Great Elephant Census (GEC) is a philanthropic effort by Paul G. Allen to do the first-ever pan-African survey of savanna elephants. The project was started in 2013 and was mostly completed by 2016, and aimed to provide accurate data to support the growing concerns of elephant population decline across the African continent. As we will see below, counting elephants is both time consuming and a costly exercise. The purpose of this thesis is to look at an alternative approach to the one employed by the GEC and use technology to reduce the cost and increase the frequency of such a census. The GEC found that the overall elephant population declined by 30% between 2007 and 2014, with a current 8% per year accelerating population decline. The GEC surveyed 18 countries and counted 352, 271 elephants.

This number represents at least 93% of elephants in these countries. The majority of the elephants were counted in legally protected areas, though a high number of poached carcasses were still found in these protected zones. There is a large cost associated with an endeavour as ambitious as this. The GEC accumulated 9, 700 hours of flight time, with an average flight lasting nearly three hours. The census was conducted with a crew of 286 members using 81 airplanes, assisted by 90 scientists and six NGO partners. The survey planes flew a total of 463, 000 km to sample 24% (218, 238 km2 ) of the total ecosystem area. The GEC had a $7 million budget donated by Paul G. Allen to sustain the initiative over its three year life.

Country-by-country findings attempted to estimate population trends using the carcass ratio, defined as the number of carcasses counted against the total elephants counted in the region. Due to the difficulty of spotting and identifying a carcass (above the initial difficulty of spotting a living elephant), the carcass ratio is considered an underestimation of the true carcass count. The results show an overall carcass ratio of 11.9% (1 in 10 elephants counted were found dead), which indicated overall population decline on the continent.

Countries such as Cameroon and Mozambique showed the highest carcass ratios, indicating a high rate of population decline and is a cause for concern. Countries such as South Africa and Uganda showed stable and growing populations, which can be attributed to the efforts made to protect their respective elephant populations. The GEC has provided invaluable information on the current state of affairs. We now have a better understanding of the overall situation, as well as country specific trends. We can identify the patterns and areas that require immediate attention, but we do not have a viable means to track the impact that these changes will have. Given the massive time and cost associated with the project, it seems infeasible to repeat the census on a regular basis.

Hundreds of visitors in Mana Pools are able to encounter elephants up close every year, guided by experts who know how to read the animals’ behaviour. The trust that elephants have for humans is exceptional. Yet in most of Africa elephants and humans are not on such good terms.

Accurate estimates suggest that there were 12 million elephants in the early 1900s. Today there are only 350, 000, which includes both savanna elephants (Loxodonta africana) and forest elephants (Loxodonta cyclotis).

That’s a 97% decline in a century.

And the killing is accelerating. Poachers are currently shooting elephants at a rate of about 100 per day, or about 30, 000 every year. In other words, about 10% of the population is being wiped out ever year. More elephants are being killed than being born.

‘Poachers’ is a euphemistic term for what is actually a highly organised network of criminals. In East, Central and West Africa, terrorist groups like Lord’s Resistance Army and Al Shabab are slaughtering elephants in their hundreds, using helicopters, AK47s, rocket-propelled grenades and night-vision goggles. There is big money in ivory, mostly in China, and it funds the criminal networks handsomely.

According to National Geographic, a pair of ivory chopsticks can sell for several thousand dollars, and carved tusks can sell for hundreds of thousands of dollars.

While southern Africa’s elephant population on the whole is mostly stable, the rest of the continent has seen a catastrophic decline. Tanzania has lost 60% of its elephants in the last five years, down to just 44, 000. Mozambique has fewer than 10, 000 elephants in the whole country, a loss of 48% in the last five years. The forest elephants of Central and West Africa – a separate, but closely related species to the savannah elephant – are crashing too. About 65% of the total number of forest elephants have been killed in the last 12 years.

Elephants of most African countries are currently listed on Appendix 1 of CITES, the international organisation that administers the trade in endangered animal species and their body parts. According to CITES, “Appendix I lists species that are the most endangered. They are threatened with extinction and CITES prohibits international trade in specimens of these species”. However, elephants from South Africa, Namibia, Botswana and Zimbabwe are currently listed on Appendix II.

According to CITES, “Appendix II lists species that are not necessarily now threatened with extinction but that may become so unless trade is closely controlled.” Until 1989, all elephants in Africa were listed on Appendix II, which allowed restricted trade in ivory. But as a result of the trade, poaching increased dramatically, and in 1989 all elephants were listed on Appendix I, effectively banning ivory trade. The price of ivory immediately fell, and poaching declined quickly.

For eight years the ban remained in place and several countries – like Kenya, Gabon, Chad, Mozambique and Zambia – burnt some or all of their ivory stockpiles. Elephant populations recovered to such an extent that in 1997 Botswana, Zimbabwe and Namibia proposed to list their elephants once again on Appendix II.

The ban on ivory trade for these countries was lifted, and in 2000 the ban was also lifted for South African elephants, but only included trade in leather goods, not ivory.

The lifting on the ban of ivory sales from the Southern African countries stimulated demand for consumer ivory in Asia, and immediately caused poaching to increase. In 1999, Japan bought 55 tons of ivory legally, and in 2008, China bought 73 tons, and actively promoted the trade in ivory carvings.

Statistics:

Over a two-year period, using standardized data collection and validation methods, the GEC accurately determined the number and distribution of the great majority of African savanna elephants and provides a baseline for future surveys and trend analyses. Final results show:

• Savanna elephant populations declined by 30 percent (equal to 144, 000 elephants) between 2007 and 2014.[2]
• The current rate of decline is 8 percent per year, primarily due to poaching. The rate of decline accelerated from 2007 to 2014.
• 352, 271 elephants were counted in the 18 countries surveyed. This figure represents at least 93 percent of savanna elephants in these countries.
• Eighty-four percent of the population surveyed was sighted in legally protected areas while 16 percent were in unprotected areas. However, high numbers of elephant carcasses were discovered in many protected areas, indicating that elephants are struggling both inside and outside parks.

Elephant population trends in Africa, in Great Elephant Census (GEC) landscapes over the past ten years based on GEC data and comparable previous surveys. Red landscapes declining >5 percent per year, orange declining 2-5 percent per year, yellow declining or increasing <2 percent per year, pale green increasing 2-5 percent per year, dark green increasing >5 percent per year. Graphic: Great Elephant Census

Chase, the founder of Elephants Without Borders (EWB), is the lead scientist of the Great Elephant Census, (GEC) an ambitious project to count all of Africa’s savannah elephants — from the air. Before the GEC, total elephant numbers were largely guesswork. But over the past two years, 90 scientists and 286 crew have taken to the air above 18 African countries, flying the equivalent of the distance to the moon — and a quarter of the way back — in almost 10, 000 hours.

Prior to European colonization, scientists believe that Africa may have held as many as 20 million elephants; by 1979 only 1.3 million remained — and the census reveals that things have gotten far worse.

According to the GEC, released Thursday in the open-access journal PeerJ, Africa’s savannah elephant population has been devastated, with just 352, 271 animals in the countries surveyed — far lower than previous estimates. In seven years between 2007 and 2014, numbers plummeted by at least 30%, or 144, 000 elephants

Estimated trends in Africa elephant populations for Great Elephant Census (GEC) study areas with historical data available, 1995–2014. Results are based on 1, 000 Monte Carlo replicates. Dark shaded area indicates ±1 SD; light shaded area indicates 95 percent confidence interval. Tick marks on x-axis indicate dates of data points used in model; dates are perturbed slightly to prevent overlap.

At least two-thirds of the continental savanna elephant population figures reported in IUCN’s 2016 African Elephant Status Report (AESR) come from the GEC results. The report will be published in advance of the 17th meeting of the Conference of the Parties to CITES (CoP17) in Johannesburg, South Africa. This updated AESR will be used to shape major policy decisions about the future of elephants.

Deeply troubled by what the GEC revealed, Mr. Allen has made elephant conservation a top priority and is working with individuals, organizations and governments from around the world to ensure that future generations will be able to see elephants with their eyes and not just read about them in books. New initiatives include the launch of a forest elephant census in regions where poaching is thought to have devastated herds and adequate population data is critically needed. Vulcan has also developed a visual data platform that provides real-time intelligence for protected management areas. The new system has been deployed as a pilot program in Kenya.

Estimated trends in Africa elephant populations in the Great Elephant Census (GEC) study areas, by country for sites with historical data available. Results are based on 1, 000 Monte Carlo replicates for each country. Dark shaded area indicates ±1 SD; light shaded area indicates 95 percent confidence interval. Tick marks on x-axis indicate dates of data points used in model; dates may be perturbed slightly to prevent overlap.

African elephants (Loxodonta africana) are imperiled by poaching and habitat loss. Despite global attention to the plight of elephants, their population sizes and trends are uncertain or unknown over much of Africa. To conserve this iconic species, conservationists need timely, accurate data on elephant populations. For this purpose, the Great Elephant Census (GEC), the first continent-wide, standardized survey of African savannah elephants are reported. Estimation of a population of 352, 271 savannah elephants on study sites in 18 countries was done, representing approximately 93% of all savannah elephants in those countries. Elephant populations in survey areas with historical data decreased by an estimated 144, 000 from 2007 to 2014, and populations are currently shrinking by 8% per year continent-wide, primarily due to poaching. Though 84% of elephants occurred in protected areas, many protected areas had carcass ratios that indicated high levels of elephant mortality. Results of the GEC show the necessity of action to end the African elephants’ downward trajectory by preventing poaching and protecting habitat.

Introduction:

Wildlife population monitoring is crucial to general conservation, sustainable wildlife use and managing human wildlife interactions. Aerial surveying is an effective tool for monitoring populations within large areas. However, such surveys are often expensive and arduous to conduct. Most aerial surveys still rely on the same methods devised five decades ago. Since then, digital cameras have become more affordable and computer vision software has advanced to such a state that those aerial survey methods can be augmented with these technologies to reduce costs and risk to human life while improving results.

Chase, the founder of Elephants Without Borders (EWB), is the lead scientist of the Great Elephant Census, (GEC) an ambitious project to count all of Africa’s savannah elephants — from the air. Before the GEC, total elephant numbers were largely guesswork. But over the past two years, 90 scientists and 286 crew have taken to the air above 18 African countries, flying the equivalent of the distance to the moon — and a quarter of the way back — in almost 10, 000 hours.

Counting and collecting Elephant data from Aerial Imaging is a time consuming task and real time monitoring of each elephant individually is not possible. Aerial Imaging of elephants can't be carried out over a large surface of area and cannot monitor each elephant habitat individually and hence lot of essential data to be recorded is missed out. Aerial Imaging is also costly and needs manual data monitoring as well. A lot of factors including increasing noise of the image due to increase of aerial altitude and biological factors degrade the accuracy of computer vision models and hence hyperspectral imaging becomes important to capture such data. Hyperspectral imaging increases the cost of the imaging by a significant amount and in this way capturing accurate metrics of elephant becomes extremely hard. Aerial Imaging is useful while counting and collecting data of number of elephants particularly useful while carrying out population census.

Conclusion:

After taking into considerations these above aspects and the problem situation caused due to the traditional and manual method of elephant monitoring, Hackster.io and Smart Parks came together with leading technology and conservation partners to build a smart wildlife tracker that will be attached on Elephants via Elephant Collars. These Collars are capable of capturing real time elephant movements with positioning of each individual elephant using GPS calibration in real time. Besides Geo location of each individual elephant, the elephant collars can be useful in capturing Image data of the nearby surroundings and similarly extract features in those images such as monitoring the distance of the elephant group near them and also identifying threats to them such as predators and human encroachment. The collar can also include audio models to identify the Elephant behavior and can identify elephant mood swings from aggressive to calm. An Accelerometer model can be deployed to understand Elephant movement and physical behavior in detail from raging of an elephant to sleeping of an elephant. The brief deployments of the models to monitor elephant activity are:

Vision Sensor Data Models include:

• Poaching andPredatorRisk Monitoring: In this solution, a vision model is designed on EdgeImpulse to monitor the risk of human encroachment in Elephant habitats or Elephant movements in human agricultural grounds. This model analyses whether the camera module deployed on the collar detects an image in the range or not. Similarly, this model is implemented in three different image module variants including thermal, infrared and RGB. Besides Human, Elephant conflict, this model monitors conflict between elephant and lions as well ( In this case, lions because upon researching and finding out elephant predators, the only result was Lions). This model detects Lions as well as bees and forest fires in thermal imaging as well as RGB.
• Elephant Seed Distribution Contribution: This model may not be as effective as the other models to track elephant behavior accurately, but this model definitely helps in estimating when an elephant contributes to Seed Distribution. This is a Vision Model that consists of 4 classes including custard apple, banana, sausage plant pods( These are the fruits wherein elephants contribute to their distribution ), and the third class includes Elephant Dung which indicates the area where the seed has been distributed.
• Density of Elephant Population Detection: This is an infrared Model to Detect the density of Elephant Population. This model consists of 3 classes which include Single Elephant, Multiple Elephant, and Multiple Obstructing Elephant. On taking a glance through the dataset, we observe that the density of population of elephants increases ascendingly starting from single elephant, to multiple elephants to multiple obstructing Elephants. This model helps to analyze the surrounding population of the Elephant Group
• Distance from Nearest Elephant Group Estimation: This infrared model contains 5 classes with relating distance of the elephants nearby with respect to the observing elephant. This model is used to detect the distance of the Elephant group from the Elephant who is capturing the data. The classes include 5 different distances which are (0-5)m, (5-10)m, (10-15)m, (15-20)m, (20-25)m
• Angle of Movement of Elephants: This infrared Model of Elephants is used to monitor the view of the surrounding Elephants with respect to the observing Elephants. This model is not necessary and important but provides an in depth information of the behavior of the other member elephants of the herd. The model includes three classes including rear view, side view and front view.

Accelerometer Data Models Include:

• Accelerometer Model to predict and classify common Elephant behavior: This model includes three classes to track elephant movement behavior on a 3 axis accelerometer. The model is trained to classify samples from aggressive behavior to standing and still behavior. The eight classes include aggressive behavior, standing, falling of an elephant, foraging of an elephant, reproductive behavior, shaking of body, sleeping and walking.

Audio Data Models Include:

• Musth Data Model: This Audio model detects and classifies musth data in Elephants which include musth in Males as well as Females. The model consists of two classes namely musth-male and musth-female used to identify the aggressive sound captured by the different genders during the event of musth.
• Audio Data Model 1: This model consists of 5 classes which are the most produced call types of an elephant. This model consists of the class which cover up 70% of the call times produced by Elephant. These classes are alert, bark, roar, rumble and trumpet. These are generic calls and relate to the organ they are produced from including trunk calls and laryngeal calls.
• Audio Data Model 2: This model is based on the audio captured on the classification basis of elephant sentiments and their mood. The first model was based on the audio created from the type of organ and Amplitude. This model is based on the Mood and social activities portrayed by the Elephant. It consists of 3 classes, social, play and assurance.
• Amplitude Data Model to identify distance of Elephants of the herd: This is simple Audio Data Model consisting of all the classes in Audio Model 1 and 2 and is classified together in 2 classes with a class of data with high Amplitude and the other with low amplitude both consisting of added Noise in the background. The 2 classes are near and far with respect to the Amplitude of the data.
• Audio Data to predict Threats to Elephants: This model consists of Audio data of classes which are threats to Elephants. The classes are Lions, Humans, bees and Guns. The data of the Human class consists of instances of Human-Elephant conflict and the Noise that Humans usually create in a conflict with Elephants. This model also consists of Audio Data of Guns which refers to the Poaching activity in Elephant habitat.

Creating TinyML Models:

Camera Vision Models:

First Model - Poaching and Predator Risk Monitoring:

In this solution, a vision model is designed on EdgeImpulse to monitor the risk of human encroachment in Elephant habitats or Elephant movements in human agricultural grounds. This is designed based on the Images captured during the encroachment. Each time an elephant comes near a human, the human is taken in as the image source in the model. This model is initially designed as a thermal model which differentiates the sources easily. The thermal data of each class is different and hence classifying data in the model becomes more accurate. This model also consists other Classes which might be Danger for Elephant. Other classes which are included in the model are to detect whether the Elephant is near an area where a wildfire is present or has started. This can help in moving Elephants to a safer area as well as detect the region where the Wildfires have been activated. The third class is of a lion which is the major predator for Elephants. This model detects Lions from all views that is front, rear and side. This can help to create an early warning system for forest rangers to help save elephants from predators like Elephants. The fourth class includes bees. Elephants are scared of bees even though bees cannot penetrate through Elephants skin. This is because Elephant brain has three times more number of neurons than a human brain. This makes the receptors even more sensitive and hence they are afraid of bees.

Data Acquisition for the Model:

Thanks to Flir Imaging to make thermal data acquisition possible for the model. I have used thermal images of all the classes to make this model.

This is the data for wildfires where the most heat can be seen in the epicentre of the fire and slowly it decreases in the periphery. Many more images of the same Wildfire has been taken which differentiates it from other classes and various angles of the wildfire has been considered for better accuracy.

This is the thermal image of a bee. The range of temperature here is a bit different with the bee being cooler than the background which is the honeycomb

The other data is collected for classes like Elephant, Lions and humans. Here variety of thermal images of humans are taken. Out of these many of them include humans holding a gun in his hand. This is used to classify a poacher much more easily. All the images have a constant thermal colour scale. The data for lions is taken with adults as well as cubs to generate alerts irrespective of the age range of lions.

This data class includes Thermal Elephants to detect Elephants nearby of the same herd. Taken from various angles, the model performs highly accurate in real world scenarios while capturing elephants even with other background data or different thermal temperature patterns.

Training the Model:

To begin training the model, initially an impulse design has to be created. By default, the first parameter for the impulse is of Image data because here the data uploaded consists of images. The second parameter also consists of Input axes as images. Here the data is processed before the initial model training starts. This helps in better separation of the classes. In the third parameter, rather than anamoly detection which is useful to predict abnormal behavior in data, here I have used Transfer Learning for Images to Fine tune a pre-trained image model.

Since this model is a thermal model, the Image Parameter for Colour depth has been selected as RGB. For every image data, there are Raw features of the image as well as the processed features. These features completely depend on the data type and vary for each image data. The processed features for different data in the same class are similar.

After selecting the Parameter of the colour depth as RGB, I proceeded over to Generating the features of the model. For each class, I have taken around 100 images which rounds up to total of 534 items in the training set. This shows the Feature Explorer of the model where areas with high density of a single class have been found. These classes densities are spread through the visualization layers which indicates a diverse dataset for each class.

Using the different Angles of Visualization layers of the model, here is the different views of the generated features of the model. This allows us to view the sample projections from different angles.

Using the hover view, I disabled the features for other classes and viewed each samples individually to have a deeper view of the placements of these features on the generated graph. Each Class has a different region where the data has been plotted.

Finally using Transfer Learning, I decided to train the model. Certain parameters had to be set for the model wherein the training cycles/iterations were set to 30. The learning rate has been set to 0.0005 to ensure the model neither overfits nor underfits.

During the first test, I selected the MobileNetV2 0.35 - A pre-trained multi-layer convolutional network designed to efficiently classify images. The model is around 687K in size with default settings and optimizations. It will perform optimally with a 96x96 input resolution and supports both RGB and grayscale.

Finally after training the model, I got decent results which provides an accurate real world classification. In the quantized (int8) model format, the loss drips down to 0.25.

In the unoptimized (float32) model, the loss dips down to 0.00 and the accuracy shoots up to 100% with the Actual label and predicted label for each class in the confusion matrix to be the same.

Later after seeing the inferencing time which is quite high for a microcontroller, I decided to train the Model on MobileNetV2 0.05 (final layer: 10 neurons, 0.1 dropout) to reduce the inferencing time further. After training the model, I got better accuracy than the initial model, higher than expected.

The results for the quantized as well as the unoptimized remain the same except for the inferencing time and peak RAM usage that is almost the same in both the models. The model yielded a 100% accuracy which is the best that the model could achieve.

Live Classifying data to check model accuracy:

Here, I have taken a thermal image of a lion. This model classifies this image and gives a 97% confidence rating that the image is a lion. Since this was connected to a mobile phone for live classification, while capturing the image, the saturation has increased. Despite this, the classification is accurate.

In this live classification data as well, the saturation of the image is high. This image contains more than one instance of human in the image. Even after this, the model is able to capture one instance of a human from the image and classify the image as human with 100% accuracy.

Deploying the Model:

In this, I have created a library for OpenMV which includes the tflite model and the classes text file of the classes. OpenMV cam supports running multiple models simultaneously.

Github Link: Thermal OpenMV model

This is a side by side comparison of an EON optimized model and a non EON optimized model. This clearly shows that the EON optimized model uses much less RAM and ROM as compared to the unoptimized model. This optimization feature is a newly released feature and allows to make inferencing on boards smoother and easier even with Image models.

Besides the OpenMV model, I have created a C++ library as well to be integrated with most of the ARM based hardwares easily using Mbed OS.

Github: Thermal C++ model

Finally I have also created an Arduino Library for the same to be implemented in many of the ESP32 microcontrollers as well as Arduino Portena H7 cam shield.

Github: Thermal Arduino Model

Second Model - SeedDistribution Detection:

This model may not be as effective as the other models to track complete elephant behavior, but this model definitely helps in estimating when an elephant contributes to Seed Distribution. This is a Vision Model that consists of 4 classes including custard apple, banana, sausage plant pods ( These are the fruits wherein elephants contribute to their distribution ), and the third class includes Elephant Dung which indicates the area where the seed has been deposited and includes the area where the plant will grow. Elephants are major contributors to Seed Distribution and can carry seeds for larger distances as compared to other animals.

The African savanna elephant holds the prize for largest living terrestrial animal, and now it apparently just set another land record: the longest distance mover of seeds. The pachyderms can transport seeds up to 65 kilometers, according to a study of elephant dung in South Africa. That’s 30 times farther than savanna birds take seeds, and it indicates that elephants play a significant role in maintaining the genetic diversity of trees on the savanna.

elephants can eat larger fruit than many other species, such as the meter-long seed pods of the sausage tree. And then there’s the sheer volume: Each elephant may deposit nearly 3200 seeds a day, estimates Joseph Dudley, an ecologist with Leidos, a science and engineering company in Germantown, Maryland, who studied savanna elephants in Zimbabwe in the 1990s.

Data Acquisition:

This model is created on a set of RGB images which is the most accurate form of classification in terms of fruits. for this the most important factor for classification is colour, followed by the shape of the fruit. The models is trained to differentiate between four classes which include sausage plant pods, custard apple, bananas and elephant dung.

This is an example of the class banana which includes images of bunches of bananas because that is the only form in which bananas are available in any forest unlike the individual ones.

This image shows the data collected for the other three classes which are sausage pods, elephant dung and custard apple. EdgeImpulse models perform better with data which contain a single image of the object and not many instances of the same object.

Creating An Impulse:

While training the model, a fixed dimension of all images are required. Since 96 x 96 is the standard size for most models, I have selected this dimension as the standard dimension. Here, the resize method is set to squash to squash all the images and include the complete image in the dataset. The learning block is set to Transfer Learning (Images) to Fine tune a pre-trained image model on your data. Good performance even with relatively small image datasets.

Output features: 4 (banana, custard_apple, elephant_dung, sausage-pod)

Generating Features for the Model:

This shows the generated features of the model. The four classes are on either vertices of the visualization dimension. This is the classification factor for all the classes.

These are individual plots of all the classes of the model. While the plot of the dataset for the class banana is present on the back of the graph, the plot of the custard apple is present on the front of the graph. Since this plot is visible from only one side of the plane, the fore and back of the graph aren't visible.

Finally using Transfer Learning, I decided to train the model. Certain parameters had to be set for the model wherein the training cycles/iterations were set to 17. The learning rate has been set to 0.001 to ensure the model neither overfits nor underfits and performs better with a small dataset.

After Training the model, the desired outputs were received. This model is trained using MobileNetV2 0.05 (final layer: 10 neurons, 0.1 dropout) to ensure that the model does not become oversized for the microcontroller.

The quantised as well as the unoptimized model yielded the same accuracy with a higher inferencing time, RAM and ROM usage. The quantised model has an inferencing time of 355ms and peak RAM usage of nearly 265K.

Here, a bunch of bananas have been taken in live classification and the model classifies it with a confidence score of 95%.

Here, the image taken is of a sausage pod. The model can detect and classify nearly 3 sausage pods of the image with 95% accuracy. The image also has a lot of brightness yet, the model is able to detect and classify.

Deployment:

This is a side by side comparison of an EON compiled model vs an uncompiled model. The differences in RAM and ROM usage are clearly visible which makes the model lighter to be deployed on a low sized microcontroller.

Finally the models are ready for deployment in three forms which include a C++ library, Arduino library and an OpenMV library.

Github Arduino Library: Eletect Arduino

Github OpenMV library (Includes a.tflite model, classes.txt): EleTect OpenMV

Github C++ Library: EleTect C++

We can further calculate the Distance between the detected fruit and the Elephant dung using GPS Algorithms to detect the distance of the seed distribution, Elephants have contributed to.

I have created a simple algorithm to calculate the distance between the fruit detected in the model and the Elephant dung detected. The distance between the fruit detected and the Elephant dung indicates the distance covered by the Elephant contributing to Seed Distribution.

#include <TinyGPS++.h>
#include <SoftwareSerial.h>
lat = (gps.location.lat(), 6);
lng = (gps.location.lng(), 6);
static const int RXPin = 4, TXPin = 3;
static const uint32_t GPSBaud = 9600;
// The TinyGPS++ object
TinyGPSPlus gps;
// The serial connection to the GPS device
SoftwareSerial ss(RXPin, TXPin);
void setup(){
 Serial.begin(9600);
 ss.begin(GPSBaud);
}
void loop(){
 // This sketch displays information every time a new sentence is correctly encoded.
 while (ss.available() > 0){
   gps.encode(ss.read());
   if (gps.location.isUpdated()){
     Serial.print("Latitude= "); 
     Serial.print(lat);
     Serial.print(" Longitude= "); 
     Serial.println(lng);
   if (banana_score > 0.5) {
    fruitlat = (gps.location.lng(), 6);
    fruitlng = (gps.location.lng(), 6);
   if (custard_apple_score > 0.5) {
    fruitlat = (gps.location.lng(), 6);
    fruitlng = (gps.location.lng(), 6);
   if (sausage-pod_score > 0.5) {
    fruitlat = (gps.location.lng(), 6);
    fruitlng = (gps.location.lng(), 6);
   if (elephant_dung_score > 0.5) {
    dunglat = (gps.location.lng(), 6);
    dunglng = (gps.location.lng(), 6);

   }
 }
}

This script simply records the GPS location whenever a fruit or elephant dung is noted and the distance between thos two can later be printed on the IoTConnect GPS block.

Elephant Population Density Detection:

This is an infrared Model to Detect the density of Elephant Population. This model consists of 3 classes which include Single Elephant, Multiple Elephant, and Multiple Obstructing Elephant. On taking a glance through the dataset, we observe that the density of population of elephants increases ascendingly starting from single elephant, to multiple elephants to multiple obstructing Elephants. This model helps to analyze the surrounding population of the Elephant Group.

Along with Analyzing an Elephants behavior, analyzing the surrounding population is also necessary. The Elephant population density model helps in analyzing the surrounding population and similarly detecting whether the Elephant is with it's herd or not. Also to analyze whether the herd has many elephants or few number of elephants.

Data Acquisition:

This is an infrared image for multiple obstructing Elephants. The image resolution of the image is not that high because these images were captured on regular infrared modules and not high end Infrared Cameras. This makes the model robust for real world scenarios and when the collar will actually be deployed on an Elephant.

This is the data for all the three classes. In total there are 2013 items. This dataset contains Multiple Obstructing Elephant images on the left, Multiple Separate Elephant images in the middle, and single Elephants in the right.

Training the model:

While training the model, a fixed dimension of all images are required. Since 96 x 96 is the standard size for most models, I have selected this dimension as the standard dimension. Here, the resize method is set to squash to squash all the images and include the complete image in the dataset. The learning block is set to Transfer Learning (Images) to Fine tune a pre-trained image model on your data. Good performance even with relatively small image datasets. Just like other Image models, this impulse generates several output features respective to the classes: multiple-separate, multiple-together, single-elephant

Generating Features for the model:

Unlike other models, which have features of a certain class grouped together, this model consists of groups of features of a class spread across the plot with many features clumped together at a certain group. This clearly represents that many images of a same class has a certain angle of the data capture, certain ambience and colour saturation. Similarly data of rear, front and back angles is grouped at different locations. To get a better view of the classes, we can view each class plot individually.

These images show that a lot of data is accumulated at some places and is sparse at most. The areas of the plot where the data is dense shows that many images of a class contain same features or same angle of the image. Most of the dense data is located in the class multiple-together

Finally using Transfer Learning, I decided to train the model. Certain parameters had to be set for the model wherein the training cycles/iterations were set to 17. The learning rate has been set to 0.0005 to ensure the model neither overfits nor underfits and performs better with a small dataset.

As seen this image, I have used the MobileNetV2 0.05 (final layer: 10 neurons, 0.1 dropout), to ensure that the model is compatible enough to be deployed on the microcontroller. Comparing all the models, the V2 0.05 model yields a high accuracy along with a low RAM and ROM usage. Finally after training the model, a decent accuracy was obtained to be found as follows:

The quantised model yields an accuracy of 85.6% and a loss of 0.70 while the unoptimised model yields an accuracy of 91.1% and loss of 0.30. Along with the increasing accuracy of the model, it is also necessary to dip the loss below 1.00.

Testing Accuracy of the Model:

Here, I have used Test Data to test the model accuracy. This test data is not trained in the model and this indicates that the model performs good with out of the box data.

There were a few instances where the model gave an uncertain output but overall the model performed well with the fed test data.

Deployment:

Finally the models are ready for deployment in three forms which include a C++ library, Arduino library and an OpenMV library.

Before Deploying the model, I used the EON Compiler to reduce the RAM and ROM usage and overall reduce the size. The accuracy of the model remains the change irrespective of the optimizer.

The deployed models can be found on Github:

Github Arduino Library: Eletect Arduino

Github OpenMV library (Includes a.tflite model, classes.txt): EleTect OpenMV

Github C++ Library: EleTect C++

Distance from Nearest Elephant Group EstimationModel:

This infrared model contains 5 classes with relating distance of the elephants nearby with respect to the observing elephant. This model is used to detect the distance of the Elephant group from the Elephant who is capturing the data. The classes include 5 different distances which are (0-5)m, (5-10)m, (10-15)m, (15-20)m, (20-25)m. Estimation of the distance of the other Elephants from the observing elephants is necessary so that the Elephant does not get separated away from it's herd.

This is the dataset used to train the model with a set of 5 classes. Out of these, examples of data of the four classes are shown in this image. As the distance of the Elephants is near the source, the brighter pixels are more in density while once the distance of the population increases, the density of darker pixels increases and the brighter pixels are centered in the middle of the image.

These are some instances of the uploaded data in the model. Each image represents some class of the model totally consisting of 5 classes.

Creating an Impulse:

While training the model, a fixed dimension of all images are required. Since 96 x 96 is the standard size for most models, I have selected this dimension as the standard dimension. Here, the resize method is set to squash to squash all the images and include the complete image in the dataset. The learning block is set to Transfer Learning (Images) to Fine tune a pre-trained image model on your data. Good performance even with relatively small image datasets. Just like other Image models, this impulse generates several output features respective to the classes: 0-5, 10-15, 15-20, 20-25, 5-10

A lot of features of the same class are grouped together at a particular area in the plot and a lot of these groups are collectively found throughout the plot. A detailed view of all the individual groups of the classes can be seen here:

This image shows the individual plots of the classes of the model. The plots have clear difference in the features of the classes. The first class has most of the features centered in the plot while the fourth class has the features shifted towards the right. This feature explorer includes 2322 items generated from 2322 images.

Training the Neural Network Classifier:

Finally using Transfer Learning, I decided to train the model. Certain parameters had to be set for the model wherein the training cycles/iterations were set to 17. The learning rate has been set to 0.0005 to ensure the model neither overfits nor underfits and performs better with a small dataset.

After Training the model, the desired outputs were received. This model is trained using MobileNetV2 0.05 (final layer: 10 neurons, 0.1 dropout) to ensure that the model does not become oversized for the microcontroller.

In this model, the quantised (int8) model receives an accuracy of 89.5%. This is accurate enough even though there some bit of difference in the expected output and actual output in one of the class. This class is the 5-10m class. Here, the actual label is 5-10m but the predicted label is 0-5m. Apart from this class, other classes perform really well and predict expected outcomes.

This is the unoptimised (float32) model which yields an accuracy of 95.7%. Here, the model predicts accurately even between the 5-10m and 0-5m class. Hence, the model has expected and predicted outputs.

Testing Accuracy of the model:

After using the test data to do live classification, I got an accuracy of 84.62%. This shows that the model performs good with test data with real life data as well. Some instances were classified as uncertain but most of them were classified as per the expected classes.

Deployment:

Finally the models are ready for deployment in three forms which include a C++ library, Arduino library and an OpenMV library.

Before Deploying the model, I used the EON Compiler to reduce the RAM and ROM usage and overall reduce the size. The accuracy of the model remains the change irrespective of the optimizer.

The deployed models can be found on Github:

Github Arduino Library: Eletect Arduino

Github OpenMV library (Includes a.tflite model, classes.txt): EleTect OpenMV

Github C++ Library: EleTect C++

Angle of Movement of ElephantsModel:

This infrared Model of Elephants is used to monitor the view of the surrounding Elephants with respect to the observing Elephants. This model is not necessary and important but provides an in depth information of the behavior of the other member elephants of the herd. The model includes three classes including rear view, side view and front view. This also helps in estimation of the direction of movement of the Elephant herd. This also helps in estimation of the movement of the herd with respect to the observing Elephant.

Data Acquisition:

This is an example image of an Elephant from the side view. Such three classes are present which are Rear, Front, and Back.

This shows an example of the data collected for the three classes, front, back and side. Similarly 1417 instances of these images are collected. These images offer a clear discrimination of the dataset between the classes. It is an infrared model and hence is scalable and deployable easily on microcontrollers with low cost infrared cameras.

This is the data used to train the model. This shows an example dataset of all the three classes.

Creating an Impulse:

While training the model, a fixed dimension of all images are required. Since 96 x 96 is the standard size for most models, I have selected this dimension as the standard dimension. Here, the resize method is set to squash to squash all the images and include the complete image in the dataset. The learning block is set to Transfer Learning (Images) to Fine tune a pre-trained image model on your data. Good performance even with relatively small image datasets. Just like other Image models, this impulse generates several output features respective to the classes: back, front, side

Unlike other models, which have features of a certain class grouped together, this model consists of groups of features of a class spread across the plot with many features clumped together at a certain group. This clearly represents that many images of a same class has a certain angle of the data capture, certain ambience and colour saturation. Similarly data of rear, front and back angles is grouped at different locations. To get a better view of the classes, we can view each class plot individually.

This shows the individual plot of all the classes of the model, front, back and rear. The features of the class back in this angle of the feature explorer contains the features towards the front of the plot while the features of the class front are present towards the back of the plot from this angle of visualization.

Training the Neural Network:

Finally using Transfer Learning, I decided to train the model. Certain parameters had to be set for the model wherein the training cycles/iterations were set to 20. The learning rate has been set to 0.0005 to ensure the model neither overfits nor underfits and performs better with a small dataset.

After Training the model, the desired outputs were received. This model is trained using MobileNetV2 0.05 (final layer: 10 neurons, 0.1 dropout) to ensure that the model does not become oversized for the microcontroller. Here, I have enabled Data Augmentation which randomly transforms data while training.

These are the results after training the model for 20 iterations. The quantised (int8) model yielded an accuracy of 86.3%. The loss dropped to 0.64 with average inferencing time of 352 ms which is optimum for capturing data on an elephant collar.

The unoptimised model yielded an accuracy of 94.0% and accuracy slurped down to 0.23. Even though the inferencing time is 704ms, this is enough for a microcontroller on an elephant collar.

Finally to test the model, I used the uploaded test data from the respective classes to test it. After testing the model, I got an accuracy of 89.47%. This model performed well with unknown data as well as with data which consisted of two elephants in the same frame. Finally I deployed the model in three formats:

Before deploying the model, I ensured that the EON compiler is enabled and this helped in reducing the RAM and ROM usage on the microcontroller. The average accuracy of the Quantised model was 94.74% with the test data.

Deployment:

Finally the models are ready for deployment in three forms which include a C++ library, Arduino library and an OpenMV library.

The deployed models can be found on Github:

Github Arduino Library: Eletect Arduino

Github OpenMV library (Includes a.tflite model, classes.txt): EleTect OpenMV

Github C++ Library: EleTect C++

Accelerometer Data Models

Accelerometer Model to predict and classify common Elephant behavior: This model includes three classes to track elephant movement behavior on a 3 axis accelerometer. The model is trained to classify samples from aggressive behavior to standing and still behavior. The eight classes include aggressive behavior, standing, falling of an elephant, foraging of an elephant, reproductive behavior, shaking of body, sleeping and walking.

Males have a sexually active period, called musth, that lasts for months, while females are receptive for just a few days. During the period of musth, males search for females while advertising their heightened sexual and aggressive state with particular behaviors, secretions, and a pulsating musth-rumble, which is given while waving one ear at a time. An estrous female secretes from her temporal glands, urinates, and makes a series of powerful estrous-rumbles after mating, to attract the attention of any distant higher-ranking males, as heard here. Family members add their voices by giving a cacophony of calls in a mating-pandemonium.

Aggressive Behavior:

A combined highly aggressive form of Pushing and Tusking results in Ramming, in which an individual lowers its head with its trunk curled under (to expose tusks) and rushes toward another, goring or poking it with its tusks, breaking the skin and potentially causing serious wounds. Males, particularly those in musth, take physical contact to an extreme form, rushing at one another and attempting to gore each other with their tusks. Dueling males tusk and push one another head to head, attempting to lock tusks and leverage their opponent over or maneuver him into position where he can be gored.

Standing and Sleeping Behavior:

In the accelerometer model, apart from aggressive classes, I also includes Standing and Sleeping class. Standing class includes a small deflection in axes throughout the data capture. Sleeping class includes data with no deflection and difference in axes from the standing class. This is the differentiating factor between the standing and the sleeping class.

Falling of an Elephant:

This shows a shot of a baby elephant falling down many times. This is an instance where the Elephant isn't hurt and falls down while playing. But in many cases when an elephant is tied from it's legs, it falls down similarly. Even when an elephant is hurt on it's legs, it keeps falling down frequently. Frequent recording of the same Elephant falling down is an indicative measure that the Elephant is hurt. The model shows a great deflection when the Elephant falls down suddenly.

Elephant Shaking gesture:

An elephant expresses irritation or impatience with an individual or circumstance with an abrupt shaking of the head, or Head- Shaking, which causes the ears to flap sharply and dust to fly. Alternatively, an elephant may frighten a predator (e.g., lions, humans), an irritant (e.g., egrets, ground- hornbills, warthogs), or an elephant adversary with a Forward- Trunk- Swing, stepping or lunging forward and swinging or tossing the trunk in its direction, while simultaneously blowing out through the trunk.

This is an example of an elephant shaking. When continuously "shaking" data is forwarded to the dashboard, it can be concluded that there is a predator nearby or whether the Elephant is impatient or irritated.

Foraging and Walking of an Elephant:

The study suggests that, even at fast speeds (up to 15 mph/24 km/h), it might seem to the casual observer that elephants don’t run. Their footfall pattern remains the same as that in walking, and never do all four feet leave the ground at the same time – a hallmark of running. But an elephant’s centre of mass appears to bounce at high speeds, which meets the biomechanical definition of running.

Kinematically, one thing that distinguishes walking from running is the footfall pattern. Typical quadrupeds use a walk at slow speeds, a trot at medium speeds and a gallop at fast speeds – and each gait has a different footfall sequence/pattern. But elephants are weird because no matter how fast they go, their footfall pattern doesn’t change. They use a walking footfall pattern even at top speed. That pattern has the left hind foot moving first, followed by a brief pause, after which the left front foot moves. Then there’s a long pause, after which the same thing happens on the right side.

Usually foraging behavior of an Elephant is recorded when a predator is in sight or when a human drives an Elephant away from the Agricultural field. These are the instances when an Elephant is observed running.

Walking of an Elephant is the most frequent behavior showcased by an Elephant. In most instances whether, logistic movement of an elephant or travelling from one place to another, walking is an indicative measure that the behavior of the Elephant is decent and there is no predator or danger in range.

Reproductive behaviorof an Elephant:

Elephants of all ages and sexes secrete a fluid called temporin out of this orifice. Males, however, enter a “musth period, ” during which they secrete a fluid differing in viscosity from the fluid secreted when they are not in musth. Serum testosterone during musth is higher than in a nonmusth elephant, and the animal’s behaviour is erratic; they are uncontrollable, sometimes even by their own handlers (mahouts). Musth is the time for establishing reproductive hierarchy, which can differ from the usual social hierarchy in that a male in musth outranks nonmusth males. In the wild, males are usually at their prime physical state during musth and ordinarily do most of the breeding.

The orientation of the device was as follows: the x-axis registered sway (lateral, i.e. left−right acceleration), the y-axis registered surge (anterior− posterior, i.e. front−back acceleration), and the z-axis registered heave (dorsal− ventral, i.e. up−down acceleration

Data Acquisition:

Accurate Accelerometer data of Elephants possessing different gestures in real world environment was not available, hence, I had to simulate these gestures after a research on these gestures with respect to 10different herds of elephants. Apart from the uploaded videos of Elephants showing these behaviors, some research papers helped me get the right data simulated: Paper1, Paper2, Paper3. Even without the availability of real world data, these simulated datasets can perform really well in real world environment. The main reason behind this is that EdgeImpulse Model captures the difference in the change of the movement of the axes. Irrespective of altitude of the capture, the data is recorded. In order to capture a gesture "shake" of an elephant, the model captures the deflection of the accelerometer from the initial axes it was present at. If the accelerometer is capturing data of an object being elevated with no interruptions to the elevation of the object, the recorded data is same as when the object is stationary. Hence, simulation of the Elephant behavior is nearly the same as the gesture possessed by the Elephant in real world environment. Later, this model can be improved in case real world Data of Elephant gestures is made available.

Aggressive behavior data:

This is the gesture possessed by an Elephant when it gets too aggressive in cases of Pushing or tusking. In short there two modes of anger possessed by the Elephant, in the form of foraging and shaking. This leads to change in all three axes of the accelerometer from the initial state. This type of behavior is characterized when there is nearly equal amount of deflection in the X, Y and Z axis of the accelerometer.

Sleeping Behavior Data:

This is the data recorded when an Elephant is sleeping. When an Elephant is sleeping, there is no change in the the deflection of any axis. This was concluded after researching and confirming seeing behavior of an Elephant while sleeping in 5 uploaded Wildlife captured Video datasets. The Elephant also Sleeps on either of the sides, left or ight. The data captures in both these sides is different but since there is a small difference, it is included in the same class. The Amplitude of the recorded data in sleeping left and right is the exact opposite. While the X axis shows an Amplitude of the negative of 15 in Sleeping right, It shows an absolute Amplitude of 15 while sleeping left. Some details have to be noted while capturing these data to ensure the data performs well in Wildlife.

Falling Behavior Data:

This type of data is usually recorded when an elephant is walking and suddenly falls down. Usually this happens when an Elephant is injured and keeps falling down. This type of data when recorded many times, can be concluded that the Elephant is injured. In the graph of the recorded falling data, it is seen that there is continuous data recorded of Walking and suddenly the data deflects, noting that the Elephant has fallen down. This data interval is of 5s. Towards the end of the recorded data, it can be seen that there is a deflection from the initial recurrent recordings.

Standing Gesture Data:

This the gesture portrayed by an Elephant when an Elephant is still and standing. This data has very less deflection and this shows that the Elephant is still and standing at a position for a while. After observing Elephant behavior for a while, it was concluded that Elephants usually possess this type of a behavior when they are eating, feeding or standing in a herd. While drinking water or bathing, they do not possess this behavior. Drinking water type of behavior is included in the "shaking"class.

Walking Gesture Data:

This type of data is captured when an Elephant is walking slowly. Most of the usual travel data of an Elephant captured is from this class. Whenever an Elephant travels from one area to another or even during any usual walks, this gesture is captures. The Peaks in data is observed due to the usual humping of an elephant. After observing casual and usual walks of an Elephant, Humping was prominent during walking of an Elephant. Compared to foraging, the Amplitude of the recorded Data is low. The Amplitude of the Z Axis is in the range 15-20. Also, the interval of the data is 1000ms. Interval of data is also an important fact that classifies this data.

Foraging Gesture Data:

This type of gesture is possessed when an Elephant is running extremely fast and the humping is subsequently in the same pace. The interval between two successful Peaks of Z axis is nearly 400ms. The Amplitude of the peak is nearly 40, double as compared to walking. When an Elephant is running or rather foraging, the interval between two successful peaks of Amplitude is really small. Compared to Walking, the intensity of the recorded data Amplitude is also nearly double. This is because the Elephant is running at an extremely high speed. After observing the behavior of Elephant and comparing this behavior to Walking data, the recorded data could be simulated easily.

Shake Gesture Data:

This is the gesture portrayed by an Elephant when an Elephant is irritated or impatient and when the Elephant tries to drive away a predator, e.g- Humans, Lions. This is not such an Aggressive class of data, and the Amplitude of the highest peak is also not comparatively high. The interval between two peaks is 1000ms which means that the captured data is not so frequent.

Reproductive Gesture Data:

Musth males carry their heads well above their shoulders with exaggerated rigidity, tuck in their jaw, and stiffen their ears. These Head- High, Chin- In, Ears- Tense postures are elements of a dominant musth male’s swagger, or Musth- Walk, and characterize all of their behavior and interaction. Taking into consideration these gestures after observing and reading Research papers, I simulated this gesture. In this gesture it is prominently observed that the gesture of the Elephant head is below it's shoulders, while with increasing time, the Amplitude of these axes changes, and indicates the head of elephant risen above the shoulders.

Creating An Impulse:

In the Input block, I have selected the Time Series Data block: Operates on time series sensor data like vibration or audio data. Lets you slice up data into windows. In the processing blocks, I have selected Spectral Analysis: with input axes- X, Y and Z axis, Great for analyzing repetitive motion, such as data from accelerometers. Extracts the frequency and power characteristics of a signal over time. In the learning block, I have selected Neural Network Keras: Learns patterns from data, and can apply these to new data. Great for categorizing movement or recognizing audio.

Generating Features:

In the Digital Signal Processing plots, one of the plots is of Frequency domains: The Frequency Domain refers to the analytic space in which mathematical functions or signals are conveyed in terms of frequency, rather than time. For example, where a time-domain graph may display changes over time, a frequency-domain graph displays how much of the signal is present among each given frequency band.

Generating Features:

After collecting all the necessary data and setting all the required parameters, I generated features for the plot with classes: aggressive, falling, foraging, musth, shake, sleeping, standing, walking. In case of the accelerometer model, the window length is of 1000ms and window size increase of 80ms. After this, the total training data collected is 4716.

In this image, we can see the individual plot of all the classes. Over here, we can clearly conclude the generated features for all the classes. In the mixed plot, it was quite clear where all the classes are placed and all the classes had a factor of differentiation.

Training the Model:

Certain parameters are set before the training begins. The iterations required for training was 30 and was sufficient enough for the model. Learning rate was set to 0.0005 so that the model performs well with the data. Minimum confidence rating was set to 0.70. After training the model, I received expected results.

After training the model, I received an accuracy of 96% and the loss dipped dwon to 0.13. Most of the expected and the predicted outcomes were the same and matched accurately. The RAM and ROM usage is quite low in case of the Accelerometer model and can be deployed on any microcontroller.

Deployement:

Finally the models are ready for deployment in three forms which include a C++ library, Arduino library and an STM32 CubeAI library.

The deployed models can be found on Github:

Github Arduino Library: Eletect Arduino

Github STM32 CubeAI library ): EleTect STM32

Github C++ Library: EleTect C++

Audio Data Models:

Musth Data Model: This Audio model detects and classifies musth data in Elephants which include musth in Males as well as Females. The model consists of two classes namely musth-male and musth-female used to identify the aggressive sound captured by the different genders during the event of musth.

Male Musth: Young males sometimes exhibit sexual behavior toward one another. For example, a male may chase another male with an erection much as he would a female. A male with an erection may be observed to reach over the back of an- other male, and even mount him, much as he would with an estrous female (Male- Male- Chase, Male- Male- Reach-Over, Male- Male- Mounting)

Female Musth: Musth males often adopt a non- threatening stance or Casual- Walk when approaching a group of females, while in the presence of females, or when moving through a group testing females. This walking (or standing) with the trunk draped over a tusk in a relaxed and “casual” manner signals friendly intent

Source: Paper1

This data model is used to capture audio data from Elephants during musth, and similarly distinguish between male and female musth.

The dataset has been adopted from elephantvoices.org

Data Acquisition:

The audio data in this image is from the class musth-female and contains two examples of the recorded data of 24s and 16s. A pattern in this data is clearly visible; peaking of the Amplitude two times in the data. The msuth-female class includes this pattern of Audio.

The Audio Data from the class male-musth data shows frequent peaks in recorded Amplitude. This makes the data different from the female musth class. The examples of Audio samples in this image are of 8seconds.

Creating an Impulse:

Since this data is a time series data, the input block is Time Series data which Operates on time series sensor data like vibration or audio data and Lets you slice up data into windows.

The processing block was set to MFCC which performs great with this dataset, to identify and discriminate between two similar type of Audio signals produced by Elephants. The MFCC block performs better than MFE block with this dataset.

MFCC block Extracts features from audio signals using Mel Frequency Cepstral Coefficients.

The learning block was selected as Neural Network (Keras) which Learns patterns from data, and can apply these to new data. Great for categorizing movement or recognizing audio.

Generating Features:

Before generating features, certain parameters were set to train the model accurately. EdgeImpulse has a standard rate at which the features are set to, bu they can be altered in case the Dataset requires different parameters.

Finally, I generated features for the model and the features for each class were distinct and had their features grouped together in a spherical plot.

Since the data collected for musth male was slightly less that musth female, the features generated are a bit less. However, the individual plots clearly showcase that the features are grouped away from each other.

Training the Model:

Here, I have set the training cycles to 50, which was good enough to train my model with the given data. The learning rate I set to 0.0005 which is sort of a default parameter for most of the Audio data models to ensure the model does not overfit and the minimum confidence rating is set to 0.80 so that the model is trained to the most accuracy to perform in real world environments. EdgeImpulse also has recently launched a feature for Audio models to allow data augmentation. This randomly transforms data during training and allows to run more training cycles without the model overfitting. Here, I have added some noise to each spectrogram so that the model learns with random dataset.

The preset Architecture I have chosen is 1D convolutional which is suitable for general purpose dataset.

After training the model, I got an accuracy of 80% which seemed to be good enough with a dataset of musth distinguishing between male and female musth Audio. The loss dipped down to 0.46 which is good enough, and the inferencing time is 22ms, so the data can be captured almost continuously with very less time for classification.

Deployement:

Finally the models are ready for deployment in three forms which include a C++ library, Arduino library and for ST IoT Discovery Kit firmware.

The deployed models can be found on Github:

Github Arduino Library: Eletect Arduino

Github ST IoT Discovery Kit(.bin): EleTect ST IoT

Github C++ Library: EleTect C++

2)Audio Data Model for Trunk and Laryngeal calls:

This model consists of 5 classes which are the most produced call types of an elephant. This model consists of the class which cover up 70% of the call times produced by Elephant. These classes are alert, bark, roar, rumble and trumpet. These are generic calls and relate to the organ they are produced from including trunk calls and laryngeal calls.

What does Alert Call mean:

A new study suggests that elephants make a specific rumble in response to the danger of humans that’s distinct from calls warning of bees or other threats.

African elephants make a specific alarm call in response to the danger of humans, according to a new study of wild elephants in Kenya.

Researchers from Oxford University, Save the Elephants, and Disney’s Animal Kingdom carried out a series of audio experiments in which recordings of the voices of the Samburu, a local tribe from North Kenya, were played to resting elephants. The elephants quickly reacted, becoming more vigilant and running away from the sound whilst emitting a distinctive low rumble. This rumble indicates that the Elephant has reported danger in sight and that the human is the danger.

What does the Trumpet Call mean:

Elephants like to let loose and play around. They run around in a loose, floppy way, heads down, trunks swinging, and tails up as they make pulsating play-trumpets and nasal play-trumpets. On several occasions elephants have mock-charged her car while trumpeting and then playfully pretended to trip and fall down.

What does Elephant Rumble mean:

If a calf is denied access to the breast, as heard in this recording, he may escalate his demand by combining a cry with a rumble, making a cry-rumble. When that doesn’t work, the calf's call crescendos to a roar. When an elephant calf is hungry, he makes a begging-rumble and walks parallel to his mother, lifting his trunk so he can suckle milk from her breasts. Usually mothers will stop moving and place a foreleg forward to let their calves feed.

What does Elephant roar mean:

Elephants use soft rumbling sounds to alert the extended family to Maasai warriors, or they may emit a commanding roar or trumpet-blast to intimidate a lion lurking in the bush. In this recording, a lion attacks an elephant calf. The calf screams, and immediately his mother and other adult females rush to him. They use mobbing tactics, making powerful rumbles with a roaring quality to scare off the lion. In the midst of this commotion, an alarmed juvenile trumpets and an adult gives a threatening trumpet-blast.

Data Acquisition:

This shows the captured raw data from an example of each of the class. In the Alert Class, the peaks in the recorded data is too frequent. In the Bark Class, the Amplitude of the recorded data is less. In the Roar class, there aren't many peaks but the recorded data has a very small or nearly negligible interval between data. In the Rumble class, The intensity of the recorded data is initially high and thee intensity decreases later. In the Trumpet class, the Amplitude peaaks at one point and then the intensity of the data decreases later.

Generating Features:

Before training the model, certain parameters have to be set. EdgeImpulse by default sets these parameters, and only need to be altered if there is any specific change in your dataset.

Finally, I went ahead and generated features for the model. In the features generated for each of the classes, the plot does not seem to be extremely distinguishable from others, this is because, the features in each spectrogram of the class are diverse, but they have their own spot in the plot. No two features of the model, are grouped together which belong to different labels. During classification of this data, it can be observed that the model performs good enough, because the unique feature allotted to each spectrogram matches the respective class accurately.

In this model, I have set the number of training cycles to 100 for a good performance. The learning rate of the model is set to 0.005, here because the dataset might be a bit non differentiable if the model learns slowly. So in order to make the model learn a bit faster, I opted for 0.005 learning rate. Similarly, the preset Architecture used is 1D convolutional which is the best architecture for my Dataset.

After training the model, I got an accuracy of 67.2% and loss of 1.13 which might be a little low than expected, but after the contest, I will keep improving my model with a larger dataset and ensure that the dataset contains noise to perform better in Real environment. Even after this, the inferencing time was small, only 4ms, and the RAM usage is 5.3K and can easily be deployed on a microcontroller.

Deployment:

Finally the models are ready for deployment in three forms which include a C++ library, Arduino library and for ST IoT Discovery Kit firmware.

The deployed models can be found on Github:

Github Arduino Library: Eletect Arduino

Github ST IoT Discovery Kit(.bin): EleTect ST IoT

Github C++ Library: EleTect C++

Audio Data Model 2:

This model is based on the audio captured on the classification basis of elephant sentiments and their mood. The first model was based on the audio created from the type of organ and Amplitude. This model is based on the Mood and social activities portrayed by the Elephant. It consists of 3 classes, social, play and assurance. I will call this model2.

Similar to this model, another model with a bit different classes is created which I will be showing along with this model. It consists of four classes, alone, anger, bond and play. I will call this model3

When does an Elephant possess Anger mood:

An elephant may threaten another simply by a resolute re-orienting of its body to gaze at an opponent, or by purposeful, directed walking toward another. Sometimes just one step toward another is enough to cause a reaction. A Turn - Toward and an Advance- Toward may be associated with other aggressive postures.

When does an Elephant possess Social behavior:

Elephants engage in numerous forms of low intensity and more exuberant displays of affiliative behavior that function to reinforce social bonds between family and bond-group members. An elephant approaching another family member may rub (Social- Rub) its head or side against the other individual. In the context of greeting and reassuring calves an individual may reach its trunk toward or into the mouth of another.

During many affiliative or socially exciting interactions, elephants raise their heads and lift their ears. The Head- Raising and Ear- Lifting posture is most commonly seen when elephants greet infants or, as in this illustration, their close associates. Note the posture of the female being approached by the long- tusked matriarch who is vocalizing with her Mouth- Open. The young female raises her head and lifts her ears while reaching her trunk back toward the approaching older female.

When does an Elephant possessAssurance behavior:

Using the trunk to touch, or Caress, another in a protective, reassuring, or comforting manner is a common behavior in an elephant family. While adult females or juvenile females most often direct caresses toward infants and calves, adults and juveniles may also Caress one another. Caresses come in many forms: wrapping a trunk over the back and around the belly of a calf or over calf’s shoulder and under its neck, often touching its mouth; reaching out to touch the genitals, temporal glands, face, legs, mouth, or trunk of another individual. A gentle rumble often accompanies a Caress. Adult and juvenile females are often observed to use the body, feet, trunk, or tail to shepherd, check for the presence of, gently guide, or assist an infant or calf to safety.

Data AcquisitionModel2:

This is the collected data from three classes, play, social and assurance. The data is usually distinguishable from other collected data.

Data Acquisition Model 3:

This is the data collected from four classes: play, alone, bond and anger. From the collected data, most of the data is easily differentiable from other data. In the Anger class, the call time period is less, and also the Amplitude of the data is less.

Model2 Impulse:

In this impulse, the input block is a time series block. The processing block selected here is Mel Filterbank Energy Block which Extracts a spectrogram from audio signals using Mel-filterbank energy features, great for non-voice audio.

The learning block here is Neural Network Keras block : Learns patterns from data, and can apply these to new data. Great for categorizing movement or recognizing audio.

Generating Features:

Finally after setting some parameters, I went ahead to generate features for the model.

In this model, the generated spectrogram corelating to the data is a bit different than MFCC block.

Finally after Generating features, I got a spindle shaped plot with all the features grouped in the spindle. Usually the features generated for MFE blocks are similar to the on shown here.

This shows an individual plot of all the classes in the spindle shaped graph.

Creating an Impulse Model3:

In this impulse, the input block is a time series block. The processing block selected here is MFCC which Extracts features from audio signals using Mel Frequency Cepstral Coefficients, great for human voice. In this model, MFCC proved to be a better choice for my dataset.

The learning block here is Neural Network Keras block : Learns patterns from data, and can apply these to new data. Great for categorizing movement or recognizing audio.

After setting up certain parameters which by default are required by the model, I went ahead to generating features for the model.

This shows the features generated for the classes of the model. With the features for bond and alone in the centre of the plot, the features for play and anger are towards the opposite corners in the plot.

This is the individual plot of the model, where the features can be studied in detail.

Training the Model2:

Before training the model, I set the Number of training cycles to 150 and minimum confidence rating to 0.005 which can make the model learn good even in small datasets. After the contest ends, I will be adding more data to the model to make the Wildlife tracker much more robust and accurate. Due to time constraints, I will be working with the available dataset.

The accuracy of the model is lower than expected but the loss of the model has dipped to 0.88 which means the model could perform better with a better and larger dataset.

Training the Model3:

In this model, I have set the number of epochs to 100; sufficient enough for the model and learning rate to 0.005. The minimum confidence rating is 0.8 and the preset Architecture is 1D Convolutional.

The accuracy obtained was fair enough - 72.4%. The loss dipped to 0.79 and the inferencing time was just 4ms, easy to deploy on any microcontroller.

Deploying Model 2 & 3:

The ST IoT Discovery Kit (also known as the B-L475E-IOT01A) is a development board with a Cortex-M4 microcontroller, MEMS motion sensors, a microphone and WiFi - and it's fully supported by Edge Impulse. This board is best suited for accelerometer models, Audio models, temperature models and barometer models. Apart from computer vision, this board is the best choice to deploy the models.

Model 2:

Finally the models are ready for deployment in three forms which include a C++ library, Arduino library and for ST IoT Discovery Kit firmware.

The deployed models can be found on Github:

Github Arduino Library: Eletect Arduino

Github ST IoT Discovery Kit(.bin): EleTect ST IoT

Github C++ Library: EleTect C++

Model 3:

Finally the models are ready for deployment in three forms which include a C++ library, Arduino library and for ST IoT Discovery Kit firmware.

The deployed models can be found on Github:

Github Arduino Library: Eletect Arduino

Github ST IoT Discovery Kit(.bin): EleTect ST IoT

Github C++ Library: EleTect C++

The dataset for all the models will be uploaded on Github by November 4th. So keep a check on all three repositories of EleTect on Github.

Audio DataModel to predict Threats to Elephants:

This model consists of Audio data of classes which are threats to Elephants. The classes are Lions, Humans, bees and Guns. The data of the Human class consists of instances of Human-Elephant conflict and the Noise that Humans usually create in a conflict with Elephants. This model also consists of Audio Data of Guns which refers to the Poaching activity in Elephant habitat.

in the Human Audio data collected, the data is of a nearby tribal village in Africa which lives close to Elephant habitat and the Audio consists of Voices of people trying to drive away the Elephant from the Farm. The Data of Guns includes the shooting of the Humans when an Elephant approaches the farm. The shooting has been done in Air to drive the Elephant away. The Data of bees includes the buzzing of bees near an Elephant. Here, I have removed the noise of Elephant so that the Audio data of bees can be better captured and classified. The Data for Lions includes roaring of lions when they try to attack an Elephants. This model includes the data for the roar when the Lion tries to attack the Elephant as well as when the Elephant tries to drive away the Lion. The roars have nearly the same Amplitude but different frequency. These audios are a major threat to Elephants and if captured, the park rangers can be sent alerts via IoTConnect Dashboards.

This is an example of the data captured when Humans try to drive away and attack the Elephant to prevent it from entering the farm.

This is an example of buzzing of bees near Elephants. The noise of Elephant trumpeting in the background has been removed.

This is an example data of the Gun shoot fired in the air to drive away Elephants from the farm. The Amplitude of this data is 30000 which is really loud. The data was recorded frequently, hence, I used split data to split the samples of the gun shoot.

This is the data of roaring of a lion from a distance of about 10m. Capturing data of roaring of a lion extremely near an Elephant was difficult, hence the Amplitude of the recorded data is 20000.

I have used MFE Block to train the model because the model includes data of bees and guns and hence, MFE Block helps train the model better than MFCC.

These were the features generated for the model, The features seem to be arranged in a spindle shaped- which is prominent in most of MFE models.

The features generated for each class is also spindle shaped and is arranged together.

Training the Model:

Before training the Epochs was set to 100, which was good enough for the model to train accurately. The learning rate was set to 0.005 which is default for all MFE models and the minimum confidence rating was set to 0.80.

The accuracy obtained was 80.6% and the loss obtained was 1.03. The inferencing time was 4ms which is good enough to be deployed on any microcontroller.

The ST IoT Discovery Kit (also known as the B-L475E-IOT01A) is a development board with a Cortex-M4 microcontroller, MEMS motion sensors, a microphone and WiFi - and it's fully supported by Edge Impulse. This board is best suited for accelerometer models, Audio models, temperature models and barometer models. Apart from computer vision, this board is the best choice to deploy the models.

Finally the models are ready for deployment in three forms which include a C++ library, Arduino library and for ST IoT Discovery Kit firmware.

The deployed models can be found on Github:

Github Arduino Library: Eletect Arduino

Github ST IoT Discovery Kit(.bin): EleTect ST IoT

Github C++ Library: EleTect C++

Amplitude Data Model to identify distance of Elephants of the herd:

This is simple Audio Data Model consisting of all the classes in Audio Model 1 and 2 and is classified together in 2 classes with a class of data with high Amplitude and the other with low amplitude both consisting of added Noise in the background. The 2 classes are near and far with respect to the Amplitude of the data.

This is a pretty simple model and does the work of detecting whether the Elephant herd is near or far. If the herd detected is too far, then the park rangers can be alerted that the Elephant has lost it's herd and can be relocated safely.

This is an example of the data recorded when the Elephant is near to the herd. The Amplitude of the recorded data is also high and the data is recorded frequently. A certain patter is followed in capturing the raw data so that Elephant voice is differentiable from other sound. This data was recorded when the herd was close to each other.

This is the data recorded when the Elephant is far from the herd. The Amplitude of the data is less and the call is herd faintly. This was recording from a Video Dataset where an Elephant far away from it's herd is trying to call it's herd for help.

After setting certain parameters, I went ahead with Generating features for the model. I have used MFCC blocks to train the model because it performs better with Elephant voice dataset.

These are the features generated in the model. The features for the class near were clustered near the right while the features for class far were clustered towards the upper left.

A closer view of the generated features for individual plots can be seen here.

Training the model:

I trained the model to 40epochs, which was good enough with my small dataset( Will be training the model with a larger dataset soon!!). I set the learing rate to 0.005 which was optimum for my dataset and also set the minimu confidence rating to 0.60.

I finally got an accuracy of 90.3% with loss of 0.22. The inferencing time was just 1ms, pretty fast on a microcontroller. The issue is that that the model does not have a large dataset to perform really well in real world scenarios. Hence, I will be adding a larger dataset with total collected data of 30minutes so that the model performs really well.

Deploying Model:

The ST IoT Discovery Kit (also known as the B-L475E-IOT01A) is a development board with a Cortex-M4 microcontroller, MEMS motion sensors, a microphone and WiFi - and it's fully supported by Edge Impulse. This board is best suited for accelerometer models, Audio models, temperature models and barometer models. Apart from computer vision, this board is the best choice to deploy the models.

Finally the models are ready for deployment in three forms which include a C++ library, Arduino library and for ST IoT Discovery Kit firmware.

The deployed models can be found on Github:

Github Arduino Library: Eletect Arduino

Github ST IoT Discovery Kit(.bin): EleTect ST IoT

Github C++ Library: EleTect C++

The dataset to all the models will be uploaded on Github by 4th November so keep an eye on the three repositories. Dataset is provided so that you get a better understanding on what the model is trained on.

Other EdgeImpulse Models to be built:

The three main categories of the models have been covered and the Accelerometer, Audio and Vision models helped in Assessment of Elephants and predators. But having data about the Environmental and biological factors is also important. For this purpose, I will be further building 3 more models to assess environmental factors and detect environment conditions for Elephants. These models will be temperature models, humidity models, barometer models and in the near future will aim to build gas sensor models. These models will be uploaded by 20th November. To build these models, I will be using The ST IoT Discovery Kit (also known as the B-L475E-IOT01A) is a development board with a Cortex-M4 microcontroller, MEMS motion sensors, a microphone and WiFi - and it's fully supported by Edge Impulse. This board is best suited for accelerometer models, Audio models, temperature models and barometer models.

The data for uploading these models is set to 20th November because the ordered board from Element14 will take around 5-7 days to be shipped. Since the ElephantEdge is a contest for building Wildife trackers for Animals, these trackers need to be evaluated properly before they can be deployed. For this purpose, the models also need to be altered as per the trackers requirements. Hence, I will be building these models beyond the contest submission dates.

The ST IoT Discovery Kit includes all of the sensors to deploy these models as well as assess environmental data.

The SeeedStduio Wio Terminal includes Gas sensors to further analyse the environment to detect forest fires, dead elephants or other toxic chemicals in the environment. This can help to assess each region's air quality and whether this area is safe for an Elephant habitat or not. It can also help in early detection of fires and Elephants can be evacuated early. But this board is not available on Element14 and hence, currently, it is not possible for me to build the Gas Model. But in the near future, to improve the tracker, I will be building this model.

A glimpse of the 3 models to be built using ST IoT Discovery Kit:

Temperature model:

In order to detect certain environmental conditions, the temperature senor is real important. In cases where there is sudden increase of temperature, the Machine Learning model can help detect these instances easily. Sudden rise in temperature refers to wildfires. In cases of sudden drops in temperature, it indicates that the Elephant is having a bath or drinking water. This helps in assessing Elephant behavior with other biological factors.

This is one of the instances when there is sudden rise in temperature.

This is one of the instances when there is sudden decrease in temperature.

Anomaly detection in Edge Impulse helps in evaluating other drastic changes in temperature.

This type of a processing block can be used to train models to detect Temperature data.

Humidity Model:

The Humidity model can be used to detect the humidity in the environment. If an Elephant goes swimming in water, the humidity rises suddenly, which indicates the Elephant has gone for a bath. In dry conditions, the humidity drops significantly, which can be used to say whether an Elephant is in a region with moisture or in a dry region. Dry regions are indicators of drought and Elephants can be evacuated from there immediately.

Barometer model:

Barometer model usually helps in measuring pressure of a region. But in this case, sudden drop in pressure indicates sudden drop in altitude. Over here, the Elephants fall down suddenly in water which leads to sudden drop and then sudden increase of pressure. In such cases, it can be said that the Elephant has fallen down in a waterfall. Similarly, It can be used to measure the Altitude of the Elephant herd. Whether they are on top of a hill, or not.

Gas Sensor Model:

This can be used in various cases like, to detect a wildfire in a region by detecting the level of CO, C2HSCH, VOC and NO2 with the help of multichannel gas sensor on Seeedstudio wio terminal. It can also help detect an Elephants dead body by measuring the released toxins from the decayed body.

An example gas model used to classify beer looks like this.

Than you for viewing my project.

Updates coming soon: 4th November- Dataset on Github

20th November- 3 ML models including temperature, humidity and barometer models.

21st November- Advanced Algorithms for GPS tracking to track whether an Elephant is dead, or sleeping, calculate velocity of Elephant and rest time.

Project Explanation: