Build ML model on Edge Impulse and run it on MaaXBoard-8ULP

Intro

• First, we will show how to set up a shared connection between the MaaXBoard 8ULP board and the host computer, which is necessary for transferring files and debugging. The board can be connected to the host via USB, Ethernet, or Wi-Fi.
• Then, how to run a web-based WebAssembly application on the board. Impulses can be deployed as a WebAssembly library. This packages all your signal processing blocks, configuration, and learning blocks up into a single package. You can include this package in web pages or as part of your Node.js application. This allows you to run your impulse locally, without any compilation. In this tutorial, you'll export an impulse, and build a Node.js application to classify sensor data.

What is WebAssembly?

WebAssembly is a new type of code that can be run in modern web browsers and provides new features and major gains in performance. It is not primarily intended to be written by hand, rather it is designed to be an effective compilation target for source languages like C, C++, Rust, etc.

A web-based WebAssembly application consists of a web server that serves the WebAssembly module and an HTML file that loads and executes it in the browser. This would allow us to develop web applications that include machine learning modules developed with Edge Impulse.

Hardware setup

Boot from eMMC

Set SW1 Boot Switch on MaaXBoard-8ULP to Internal Boot Mode.

Connect the two USB type C ports to your PC, one is for serial console (J4), and another is for power supply(J7).

When the system boots up, LED1 blinks blue.

Connect Debug UARTs, Boot up and Login

MaaXBoard 8ULP has 4 UARTs converted from the USB 2.0 device port (J4). The first two are reserved, the third UART is used for A35 debug, the last UART is used for M33 debug.

The following is an example, when connect J4 of MaaXBoard 8ULP to the PC USB.

Windows Host

View the Device Manager on the PC, there are four USB Serial Ports, the first two (COM25&COM26) are reserved, the third (COM27) is A35 Debug Interface, the last one (COM28) is M33 Debug Interface (These COM numbers can be different on your system, look at the four new COM number that appear when you connect the debug cable).

Login

Yocto system supports the following login methods: login directly, login from debug serial, login from SSH.

• login directly

Connect screen and keyboard to MaaXBoard 8ULP. When the system boots up, it will run the Weston Wayland Desktop Environment directly, click Wayland Terminal to operate the board.

• login from debug serial

Install the Serial Communication software (e.g. PUTTY), select the corresponding port number (Obtained from Device Manager refer to 2.3 Debug UARTS), baud rate as 115200, data bits as 8, stop bits as 1, parity as none.

When the system boots up, the serial terminal of A35 debug interface will print the following information:

NXP i.MX Release Distro 6.1-langdale maaxboard8ulp ttyLP1 
maaxboard8ulp login:root

Enter username as “root”, login without password.

When the system boots up, the serial terminal of M33 debug interface will print the following information:

AD entered Active mode
Handle Peer Core Reboot
RPMSG String Echo FreeRTOS RTOS API Demo (v2.13.1-g40ab8545a1a3) ...
Handle Peer Core Linkup
Nameservice sent, ready for incoming messages...

Linux Host

After you connect the debug USB cable to the host, check to see USB Serial Ports available by typing in the following: ls /dev/ttyUSB ( don't press the Enter key), and hit the tab key.

bn@nc20:~$ ls /dev/ttyUSB
ttyUSB0  ttyUSB1  ttyUSB2  ttyUSB3

• In the first console run

bn@nc20:~$ sudo chmod 666 /dev/ttyUSB2
bn@nc20:~$ cu -s 115200 -l /dev/ttyUSB2 --parity none --nostop
Connected.

• Open another console on Ubuntu PC and change RTS/CTS flow-control option:

bn@nc20:~$ stty -F /dev/ttyUSB2 -crtscts

Power up the MaaXBoard 8ULP.

Verify that the U-boot/Linux boot messages display via serial cable.

• Return to the first console then log in:

NXP i.MX Release Distro 6.1-langdale maaxboard8ulp ttyLP1 
maaxboard8ulp login:root

Software setup

Prerequisites

Install Node.js - to build the application.

Check whether node.js is installed in your board

root@maaxboard-8ulp:~# node -v

Connect to the internet and deploy the Impulse

With Windows Host, connect the ethernet cable to the home router.

If you are working with a Linux Host, connect the MaaXBoard 8ULP ethernet to your Host ethernet port.

Set a Shared IP Address for Ubuntu PC's Ethernet Interface

For the Ethernet communication between the host PC and MaaXBoard 8ULP, Edit wired Connections through a Linux wired connection settings and set the ipv4's connection method to "shared". Alternatively, Network-manager offers a built-in interactive console to achieve the same.

Open a new terminal and set up a shared connection using these commands:

bn@nc20:~$ sudo nmcli connection edit type ethernet

nmcli> set ipv4.method shared
nmcli> save
nmcli> quit

Check the shared ip_address.

On the host:

bn@nc20:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp89s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 88:ae:dd:0a:cf:b6 brd ff:ff:ff:ff:ff:ff
    inet 10.42.0.1/24 brd 10.42.0.255 scope global noprefixroute enp89s0
       valid_lft forever preferred_lft forever
    inet6 fe80::f7da:fbc1:9412:5a4e/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

On MaaXBoard 8ULP:

root@maaxboard-8ulp:~# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.42.0.196  netmask 255.255.255.0  broadcast 10.42.0.255
        inet6 fe80::c40a:8dff:fec6:6bd3  prefixlen 64  scopeid 0x20<link>
        ether c6:0a:8d:c6:6b:d3  txqueuelen 1000  (Ethernet)
        RX packets 342  bytes 55576 (54.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 260  bytes 19845 (19.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 93  bytes 7578 (7.4 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 93  bytes 7578 (7.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Deploying your impulse

On your Host, head over to your Edge Impulse project, and go to Deployment. From here you can create the full library which contains the impulse and all external required libraries. Select WebAssembly and then click Build to create the library. Download and unzip the.zip file.

Linux Host

Unzip the WebAssembly library and upload the folders into MaaXBoard 8ULP using these commands:

bn@nc20:~$ sudo su
root@nc20:~# cd /home/bn/Downloads/ 
root@nc20:/home/bn/Downloads# unzip embeddedcvonline-wasm-v11.zip -d wasm/
Archive:  embeddedcvonline-wasm-v11.zip
   creating: wasm/browser/
  inflating: wasm/browser/README.md  
  inflating: wasm/browser/index.html  
  inflating: wasm/browser/run-impulse.js  
  inflating: wasm/browser/server.py  
  inflating: wasm/browser/edge-impulse-standalone.js  
  inflating: wasm/browser/edge-impulse-standalone.wasm  
   creating: wasm/node/
  inflating: wasm/node/README.md     
  inflating: wasm/node/run-impulse.js  
  inflating: wasm/node/edge-impulse-standalone.js  
  inflating: wasm/node/edge-impulse-standalone.wasm  
root@nc20:/home/bn/Downloads# ls wasm/
browser  node

Now use remote copy to copy the files to MaaXBoard 8ULP

root@nc20:/home/bn/Downloads# scp -r wasm root@10.42.0.196:~/
The authenticity of host '10.42.0.196 (10.42.0.196)' can't be established.
ECDSA key fingerprint is SHA256:RD2/oAJslXyYRVe7NVPJwJtYnILgcI70fE+j3B6sqFs.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '10.42.0.196' (ECDSA) to the list of known hosts.
edge-impulse-standalone.wasm                  100% 7183KB  11.2MB/s   00:00    
README.md                                     100%  840   543.4KB/s   00:00    
run-impulse.js                                100% 4253     2.4MB/s   00:00    
edge-impulse-standalone.js                    100%  170KB   9.8MB/s   00:00    
edge-impulse-standalone.wasm                  100% 7183KB  11.2MB/s   00:00    
README.md                                     100%  318   200.3KB/s   00:00    
index.html                                    100% 1534   926.7KB/s   00:00    
run-impulse.js                                100% 3337     2.2MB/s   00:00    
edge-impulse-standalone.js                    100%  170KB  10.0MB/s   00:00    
server.py                                     100%  604   369.3KB/s   00:00    
root@nc20:/home/bn/Downloads#

Check if the files are in MaaXBoard 8ULP

root@maaxboard-8ulp:~# ls wasm/                                                 
browser  node                                                                   
root@maaxboard-8ulp:~#

Running the impulse

With the project ready it's time to verify that the application works. Head back to the studio and click on Live Classification. Then load a validation sample, and click on a row under 'Detailed result'.

To verify that the local application classifies the same, we need the raw features for this timestamp. To do so click on the 'Copy to clipboard' button next to 'Raw features'. This will copy the raw values from this validation file, before any signal processing or inferencing happened.

Then invoke the local application by passing these features as an argument to the application.

On MaaXBoard 8ULP run:

root@maaxboard-8ulp:~# node wasm/node/run-impulse.js "0xfdfdfd, 0xfdfdfd, 0xfdfdfd, 0xfdfdfd, 0xfdfdfd, ..."

This will run the signal processing pipeline, and then classify the output:

Running inference for Bernard Ngabonziza / EmbeddedCVOnline (version 11)
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
{
  anomaly: 0,
  results: [
    { label: 'paper', value: 3.763475087126267e-10 },
    { label: 'rock', value: 1 },                                                
    { label: 'scissors', value: 6.991365900923738e-14 }                         
  ]                                                                             
}                                                                               
root@maaxboard-8ulp:~#

Which matches the values we just saw in the studio. You now have your impulse running locally!

Run the Web browser application.

Prerequisites

• Python 3 - to run a web server that serves the MIME type of the .wasm file correctly.

The web browser application adds two new files. An HTML file called index.html, where you'll call the inferencing engine with the features, and a server file called server.py.

Running the impulse

On MaaXBoard 8ULP run the the server application via these commands:

root@maaxboard-8ulp:~# cd wasm/browser/ 
root@maaxboard-8ulp:~/wasm/browser# python3 server.py
Running at http://localhost:8082

In the Linux host start a browser and type:

http://10.42.0.196:8082

Note that we replaced localhostwith the MaaXBoard 8ULP's IP-Address shared from the Host.

Running inference

Paste in the features (when prompted unresponsive, press wait) that we got earlier then hit Run inference.

Our impulse runs in our web browser application!