ML model Optimization with Edge Impulse on RzBoard

Intro

In this project, we will develop machine learning models, train them, and optimize them on data collected using the same sensor we will use in deployment.

Prerequisite

The first category of people this project is for, are people who are familiar with running software on edge-embedded devices; and would like to learn how to apply machine learning model development practices, to deploy machine learning models on edge devices.

The second category is the people who took at least an introductory course in machine learning; and want to bridge into deploying machine learning models on edge devices. For reference follow the tutorial on training and deploying ML models on RzBoard with Edge Impulse.

System setup

Verify that the USB to serial cable is connected properly.

RXD is RzBoard's Receiver, which means the transmitting cable to Rzboard goes there. The Same for TXD which is RzBoard's Transmitter to the other device.

The RED cable connector should not be connected.

RZBoard boot up with a Windows host

Open Tera Term and select the serial port associated with the USB-Serial adaptor then click OK.

In the Tera Term console select Setup from the top menu and then Serial Port. Change the speed to 115200 then click New Setting.

In the RZBoard, press and hold the S1 button to power on the RZ/V2L. Verify that the U-boot/Linux boot messages display via serial cable.

.
.
.
Poky (Yocto Project Reference Distro) 3.1.14 rzboard ttySC0
rzboard login: root
Password: avnet

After booting connect the RzBoad ethernet cable to your router for an internet connection. you will need the internet to set up edge impulse for RzBoad.

For a Linux host connect via serial connection

• Open a console and run

bn@nc:~$ sudo chmod 666 /dev/ttyUSB0
bn@nc:~$ cu -s 115200 -l /dev/ttyUSB0 --parity none --nostop
Connected.

• Open another console on Ubuntu PC and change RTS/CTS flow-control option:

bn@nc:~$ stty -F /dev/ttyUSB0 -crtscts

On the RZBoard, press and hold the S1 button to power on the RZ/V2L.Verify that the U-boot/Linux boot messages display via serial cable.

• Return to the first console then log in:

bn@nc:~$ cu -s 115200 -l /dev/ttyUSB0 --parity none --nostop
Connected.
.
.
.
Poky (Yocto Project Reference Distro) 3.1.14 rzboard ttySC0
rzboard login: root
Password: avnet

In the Console (Linux / Tera Term), ping your PC to ensure that the RzBoard is properly talking to your PC.

Connecting to Edge Impulse

Software Requirements

• Edge Impulse
• Updated RZBoard OS

Log in to Edge Impulse, click on Create Project, and provide a name for your project.

With all software set and hardware set up, connect your USB camera to your RzBoard and run:

root@rzboard:~# edge-impulse-linux
Edge Impulse Linux client v1.4.2

? Select a microphone (or run this command with --disable-micro
phone to skip selection) USB-Audio - HD Pro Webcam C920
[SER] Using microphone hw:1,0
[SER] Using camera Video Capture 4 starting...
[SER] Connected to camera
[WS ] Connecting to wss://remote-mgmt.edgeimpulse.com
[WS ] Connected to wss://remote-mgmt.edgeimpulse.com
? What name do you want to give this device? RzBoard
[WS ] Device "RzBoard" is now connected to project "bngaboav-project-1"
[WS ] Go to https://studio.edgeimpulse.com/studio/185802/acquisition/training to build your machine learning model!

This will start a wizard that will ask you to log in and choose an Edge Impulse project. If you want to switch projects run the command with --clean.

Verifying that your device is connected

That's all! Your device is now connected to Edge Impulse. To verify this, go to your Edge Impulse project, and click Devices. The device will be listed here.

Data correction

In this project, we will be optimizing the machine Learning model for a classification task. We will build a classifier with four classes: Rock, Paper, Scissors, and Background. We will collect around 100 images for each class, 80 for training and 20 for testing. Each split contains the same number of samples from each class: this is a balanced multiclass classification problem, which means classification accuracy will be an appropriate measure of success.

We will be collecting data using the Webcam connected to the RZBoard, via the Edge Impulse for Linux CLI tools that are installed on RZBoard.

Go to the "Data acquisition" tab on Edge Impulse and choose Connect a device to start building your dataset. In our case will be choosing the RZBoard we connected earlier. A new tab or window will open, with the ability to take pictures by clicking the "Capture" button. set the label for the Image you are collecting. Images collected will be automatically added to your dataset.

Impulse Design

Developing a model that does better than a baseline

In this phase, the objective is to establish a model with sufficient predictive capacity, surpassing a naive benchmark. For the MNIST data set classification example, which involves classifying digits, a model demonstrates this capability if it attains an accuracy exceeding 10%, indicating it performs better than random guessing across ten classes. In the context of the IMDB dataset, used for sentiment analysis, a model is considered to have predictive capacity if it achieves an accuracy higher than 50%, surpassing the baseline of random binary classification. For the specific scenario being discussed, a model would need to exceed an accuracy of 25% to be considered as having sufficient statistical efficacy.

Default Configuration

Edge Impulse default configuration includes a drop-out layer; we will remove this layer in order to show later the effect of this layer.

Scaling up: developing a model that overfits

Once you’ve obtained a model that has statistical power, the question becomes, is your model sufficiently powerful? Does it have enough layers and parameters to properly model the problem at hand? For example, a neural network comprising a single hidden layer with merely two neurons might exhibit some level of predictive capability on the MNIST dataset but would fall short of adequately addressing the task. It's crucial to navigate the perennial trade-off in machine learning between optimization and generalization, aiming for a model that precisely balances underfitting with overfitting, and undercapacity with overcapacity. Identifying this equilibrium necessitates initially surpassing it.

To figure out how big a model you’ll need, you must develop a model that overfits. This is fairly easy:

Make the layers bigger (Increase the number of filters)

Add layers

From adding the dense layer of 512 nodes, we see that the loss increases.

Overfitting and underfitting

During the training phase, it's common to observe a model's performance on the test set reach its zenith within a few epochs, followed by a subsequent decline, indicative of the model commencing to overfit to the training dataset. Overfitting is a ubiquitous phenomenon across all machine-learning tasks. Acquiring proficiency in mitigating overfitting is pivotal for achieving expertise in machine learning.

The bias which is the difference between average model prediction (across data sets) and the target, vises the variance of models (across data sets) for a given point.

In machine learning, a core challenge lies in balancing optimization with generalization. Optimization is the act of fine-tuning a model to achieve peak performance on its training dataset—essentially the 'learning' aspect of machine learning. Generalization, on the other hand, measures the model's efficacy on unseen data. The ultimate aim is to enhance generalization, but you don’t control generalization; adjustments are made based on the model's training data performance.

Initially, during training, there's a positive correlation between optimization and generalization: improvements in training loss often parallel enhancements in test data performance, indicating an underfitting scenario where the model has yet to capture all pertinent patterns in the training set. However, beyond a certain point, generalization gains plateau, and further training can lead to a decline in test performance as the model begins to overfit, learning training-specific patterns that do not generalize well to new data.

To mitigate overfitting and achieve better generalization, expanding the training dataset is often the most effective strategy, particularly since a more extensive dataset promotes more robust generalization. In domains like computer vision, data augmentation is a widely adopted technique to artificially enlarge the training set. When obtaining more data is not feasible.

Another approach involves restricting the amount of information the model can store or imposing constraints on the stored information, compelling the model to prioritize learning the most significant patterns, which are more likely to generalize well.

This strategy, known as regularization, aims to develop a model that neither underfits nor overfits, striking an optimal balance between the two.

Because we have relatively few training samples (400), overfitting will be our number one concern. Let’s review some of the most common regularization techniques

Regularizing your model and tuning your hyperparameters

This phase is the most time-intensive: iterative cycles of model refinement, training, and evaluation on the validation set (distinct from the test set at this stage) will be conducted until the model achieves its optimal performance.

These are some of the techniques that can help mitigate overfitting:

• Add dropout.
• Try different architectures: add or remove layers.
• Try different hyperparameters (such as the number of units per layer or the learning rate) to find the optimal configuration.

Reducing the network’s size

The simplest way to prevent overfitting is to reduce the size of the model: the number of learnable parameters in the model (which is determined by the number of layers and the number of units per layer). In deep learning, the number of learnable parameters in a model is often referred to as the model’s capacity. Intuitively, a model with more parameters has more memorization capacity and therefore can easily learn a perfect dictionary-like mapping between training samples and their targets—a mapping without any generalization power.

For instance, a model with 500, 000 binary parameters could easily be made to learn the class of every digit in the MNIST training set. We’d need only 10 binary parameters for each of the 50, 000 digits.

However, such a model would be useless for classifying new digit samples. Always keep this in mind: deep learning models tend to be good at fitting to the training data, but the real challenge is generalization, not fitting.

On the other hand, if the network has limited memorization resources, it won’t be able to learn this mapping as easily; thus, to minimize its loss, it will have to resort to learning compressed representations that have predictive power regarding the targets—precisely the type of representations we’re interested in.

At the same time, keep in mind that you should use models that have enough parameters that they don’t underfit: your model shouldn’t be starved for memorization resources. There is a compromise to be found between too much capacity and not enough capacity.

Unfortunately, there is no magical formula to determine the right number of layers or the right size for each layer. You must evaluate an array of different architectures (on your test set) to find the correct model size for your data. The general workflow to find an appropriate model size is to start with relatively few layers and parameters, and increase the size of the layers or add new layers until you see diminishing returns concerning test loss.

One basic strategy to counter overfitting involves downsizing the model, specifically the quantity of learnable parameters it contains, which hinges on the number of layers and units within each layer. This measure of learnable parameters, often called the model's capacity in deep learning, correlates directly with its ability to memorize. Essentially, a model with a higher count of parameters has a greater capacity to memorize, enabling it to learn a precise, dictionary-style correlation between training samples and their targets—a mapping without any generalization power.

Take, for example, a hypothetical model equipped with 500,000 binary parameters could theoretically memorize the category of every digit in the MNIST dataset using just 10 parameters for each of the 50,000 digits. However, such a model would falter when tasked with recognizing new digit instances, underscoring a pivotal notion: the true hurdle in deep learning isn't merely fitting the training data but achieving generalization beyond it.

Conversely, a model constrained by limited memorization capability cannot easily learn direct mappings; it is compelled to adopt more generalized, compressed representations that hold predictive value, which is exactly what we aim for. Yet, it's crucial to ensure the model has enough parameters that they don’t underfit. Striking a balance between excessive and inadequate capacity is key.

Unfortunately, there is no magical formula for finding the optimal mode with the right layer count and dimensions. The process typically involves starting with relatively few layers and parameters and increasing the size of the layers or adding new layers until you see diminishing returns concerning test loss.

Training results

Test results

Adding dropout

Dropout is a widely used and highly effective technique for regularizing neural networks. It involves randomly dropping out (setting to zero) a portion of the output features in a layer during the training phase.

The proportion of features deactivated is known as the dropout rate, typically set between 0.2 and 0.5. During testing, dropout is not applied; instead, the outputs of the layer are scaled down by the dropout rate to compensate for the increased number of active units compared to the training phase.

In summary, to combat overfitting in neural networks, the most prevalent strategies include acquiring more training data, reducing the network's size, and implementing dropout.

Training results

Test results

Training for more epochs

Training for 25 epochs

Training for 40 epochs

Test results

Deploying back to the device

To run your impulse locally, just connect to your Renesas RzBoard and run:

root@rzboard:~# edge-impulse-linux-runner --clean
Edge Impulse Linux runner v1.4.2
? What is your user name or e-mail address (edgeimpulse.com)?

This will automatically compile your model with full hardware acceleration and download the model to your RzBoard, and then start classifying.

Or you can download an eim model locally with the following command.

root@rzboard:~# edge-impulse-linux-runner --clean --download rock-paper-scissors-rzb.eim
Edge Impulse Linux runner v1.4.2
? What is your user name or e-mail address (edgeimpulse.com)? b
ernard.ngabonziza@avnet.com
? What is your password? [hidden]
[RUN] Downloading model...
[BLD] Created build job with ID 6317518
.
.
.
[BLD] Building binary OK
[RUN] Downloading model OK
[RUN] Stored model in /home/root/rock-paper-scissors-rzb.eim
root@rzboard:~#

Then use it with the above runner as follows:

root@rzboard:~# edge-impulse-linux-runner --model-file rock-paper-scissors-rzb.eim
[RUN] Starting the image classifier for Ngabonziza / bngaboav-project-1 (v1)
[RUN] Parameters image size 96x96 px (4 channels) classes [ 'paper', 'rock', 'scissors' , 'backgroung']
[RUN] Using camera Video Capture 4 starting...
[RUN] Connected to camera

Want to see a feed of the camera and live classification in your browser? Go to http://10.42.0.92:4912

classifyRes 4ms. { paper: '0.3870', rock: '0.6113', scissors: '0.0018', backgroung: '0.0018' }
classifyRes 1ms. { paper: '0.4084', rock: '0.5894', scissors: '0.0022', backgroung: '0.0018' }
classifyRes 1ms. { paper: '0.3691', rock: '0.6289', scissors: '0.0019', backgroung: '0.0018' }
classifyRes 1ms. { paper: '0.3760', rock: '0.6221', scissors: '0.0021', backgroung: '0.0018' }

You will see the model inferencing results in the terminal also we stream the results to the local network. This allows you to see the model's output in real-time in your web browser.

Open the URL shown when you start the runner.

In the Linux host start a browser and in the browser type:

http://10.42.0.92:4912 # This is specific to your RzBoard's IP-Address.

And you will see both the camera feed and the classification results.

Results