Real-time Food Quality Prediction

With each passing year, the issue of food waste becomes more acute for the environment. A recent Food Waste Index Report by the United Nations Environment Program (UNEP) showed that, on average, consumers waste almost a billion tons of food per year (or 17 percent of all food purchased): https://www.unep.org/resources/report/unep-food-waste-index-report-2021

The fact that people produce more food than they consume has significant negative consequences. For example, an estimated 8-10% of global greenhouse gas emissions come from unused food. On the contrary, reducing food waste will help to reduce greenhouse gas emissions and global pollution, as well as increase food availability for countries that suffer from hunger.

This situation suggests that in the near future, we will need to focus not on scaling food production, but on timely quality control so that fresh products can be sold and consumed. To fix the current situation, humanity will need smarter user-friendly technologies that can help them monitor product quality in real-time.

In this piece, I’ll explain an easy way to check food quality that can be implemented in an average store, and even in your own fridge. And the best part - it’s not rocket science at all!

Introduction:

Recently, I conducted a simple experiment, and I would like to share it with you, as I strongly believe that such practical solutions can make a great difference in solving global problems. Baby steps on the way to global good.

My idea is to use the Tiny Machine Learning approach to forecast whether food is fresh or spoiled based on the data from gas sensors. I conducted my experiment with the use of 7 gas sensors.

In my tutorial, you will learn how you can automatically create a super tiny machine learning model, embed it into a sensor’s microcontroller, and check food quality with it.

So let’s get it started!

Procedure:

Step 1: Create a TinyML model with Neuton

Create a new solution “Food Quality” on the Neuton platform, and upload the training dataset containing signals for food quality, labeled for two classes (fresh and spoiled). My dataset contained 784 rows.

Then, select the target (Label) and target metric (Accuracy), also enabling the Tiny Machine Learning mode. Additionally, select the 8-bit depth for calculations without float data types and click “Start Training”.

The model will be ready in several minutes. Next, download the model.

Step 2: Create the microcontroller’s firmware

Download an example: https://github.com/Neuton-tinyML/arduino-example

Project Description

The project contains:

• code for receiving a dataset via USB-UART serial port,
• prediction fulfillment,
• results indication,
• code for measuring prediction time.

The main sketch file “arduino-tiny-ml-neuton.ino” has functions for processing data packets.

The main process goes on in the user_app.c file:

static NeuralNet neuralNet = { 0 };
extern const unsigned char model_bin[];
extern const unsigned int model_bin_len;
uint8_t app_init()
{
return (ERR_NO_ERROR != CalculatorInit(&neuralNet, NULL));
}
inline Err CalculatorOnInit(NeuralNet* neuralNet)
{
memUsage += sizeof(*neuralNet);
app_reset();
timer_init();
return CalculatorLoadFromMemory(neuralNet, model_bin, model_bin_len, 0);
}

Here, create an object NeuralNet and call a function for loading the model located in the file model.c

CalculatorLoadFromMemory(neuralNet, model_bin, model_bin_len, 0);

The model is now ready to make predictions. For this, you need to call the CalculatorRunInference function by transferring a float array of size neuralNet.inputsDim to it.

The last value is BIAS and should be 1.

inline float* app_run_inference(float* sample, uint32_t size_in, uint32_t *size_out)
{
if (!sample || !size_out)
return NULL;
if (size_in / sizeof(float) != app_inputs_size())
return NULL;
*size_out = sizeof(float) * neuralNet.outputsDim;
if (app.reverseByteOrder)
Reverse4BytesValuesBuffer(sample, app_inputs_size());
return CalculatorRunInference(&neuralNet, sample);
}

When performing a prediction, three callback functions are called: CalculatorOnInferenceStart before and CalculatorOnInferenceEnd after the prediction, as well as CalculatorOnInferenceResult with the prediction result.

In the example, I used these functions to measure the prediction time.

An array with class probabilities is passed to the function with the result of the prediction, with the size neuralNet.outputsDim. Here, find the class with the highest probability, and if the probability is > 0.5, turn on the LED (green for class 0 and red for class 1).

inline void CalculatorOnInferenceResult(NeuralNet* neuralNet, float* result)
{
if (neuralNet->taskType == TASK_BINARY_CLASSIFICATION && neuralNet->outputsDim >= 2)
{
float* value = result[0] >= result[1] ? &result[0] : &result[1];
if (*value > 0.5)
{
if (value == &result[0])
{
led_green(1);
led_red(0);
}
else
{
led_green(0);
led_red(1);
}
}
else
{
led_green(0);
led_red(0);
}
}
}

Step 3: Copy the downloaded model to the sketch

Copy the model file model.c from the model archive to MCU firmware.

Step 4: Compile the sketch and upload it to the board

Now, everything is ready for sketch compilation. I used a program to send data from the computer to MCU and display the prediction results (it emulates sensor data and sends data to MCU).

To perform the prediction, download the utility: https://github.com/Neuton-tinyML/dataset-uploader

Depending on your OS, use the appropriate file in the bin folder.

You need to specify two parameters for the utility: USB port and dataset file.

Sample:

uploader -d./food_quality_binary_test_spoiled.csv -s /dev/cu.usbmodem14411101

The utility reads a CSV file and sends the samples line by line to the microcontroller. Then, it outputs the results as a CSV file to the stdout stream. After sending all the samples, the utility requests a report that contains the prediction time and the amount of memory consumed.

Step 5: Check how the embedded model functions

Create two CSV files, containing one line each, with data corresponding to two classes: fresh and spoiled.

Then, send each of them to the microcontroller and see the result of the prediction

In this case, the food stays fresh, as the predicted class is zero, which means “fresh food”. The probability of zero is very high - 100% percent. The prediction was made in 3844 microseconds with 199 kB of Flash memory usage and 136 B of RAM usage. Also, you can see that the green LED is on, which signifies a good outcome.

Here are the results for another row of data. In this case, we see that the model predicted that the food is spoiled, as the predicted class is one, which indicates “spoiled food”. The prediction was also made very fast, in 3848 microseconds, with the same 199 kB of Flash memory usage and 136 kB of RAM usage. In this case, you can see the red LED, indicating that the food is spoiled.

Conclusion:

This experiment proves that in just 5 simple steps, you can create a working smart device that, despite its tiny size, can be of great help in monitoring food quality. I am absolutely sure that such technologies can help us make our planet a cleaner and healthier place