Edge Impulse's Ingestion Service on Embedded Linux Part 1

This first installment in using Edge Impulse's Ingestion Service on embedded Linux serves to demonstrate how to build the C SDK into an embedded Linux image using PetaLinux and deploy it on a Xilinx Zynq-based development FPGA board.

The Edge Impulse C SDK is a header-only library written in C99 that is portable. It serves as a method for collecting data on embedded devices that will ultimately be streamed to Edge Impulse's online IDE, Edge Impulse Studio, for machine learning applications.

For this project I chose Avnet's MiniZed board for its reliable Wi-Fi and Bluetooth 4.1 connection capabilities, ARM-based processor due to the C SDK's Mbed TLS dependencies, and its available I/O interfaces for connecting sensors to to gather data from. Its 8GB of eMMC flash is a good size to allow for a larger sized filesystem and/or interim storage of sensor data.

Create PetaLinux Project

Start by creating a top level project folder and change directories into it:

mkdir -p ./edge-impulse-projects
cd ./edge-impulse-projects

In a previous project, I covered how to generate BSPs for Avnet boards such as the MiniZed. After creating the project directory, I copied over the BSP file for simplicity to keep everything all in one place (this is just a personal preference of mine because I've overly obsessed with my file organization).

Using this PetaLinux BSP for the MiniZed, create a new project and change directories into it:

petalinux-create -t project -s ./minized_emmc_enhanced_2019_2.bsp
cd ./minized_emmc_enhanced_2019_2/

Import Hardware Platform from Vivado

The BSP project also includes the hardware platform generated by Vivado necessary for building a PetaLinux project on. For the MiniZed, it's located under /<project directory>/edge-impulse-projects/minized_emmc_enhanced_2019_2/hardware/MINIZED_2019_2

Import the hardware into the PetaLinux project and open the hardware configuration editor with the following command:

petalinux-config --get-hw-description ./hardware/MINIZED_2019_2/

Configure the hardware to update the MiniZed's configuration to use its 8GB external flash for storing the root filesystem in the ext4 format, as well as update the device node name for the filesystem. TFTP boot is also not needed in this project.

Under Image Packaging Configuration, update Root Filesystem type to EXT (SD/eMMC/QSPI/SATA/USB). Change Device node of SD device from /dev/mmcblk0p2 to /dev/mmcblk1p2, and clear out Root filesystem formats to only include ext4. Finally, uncheck the option to Copy final images to tftpboot.

Save and close the hardware configuration editor after making these changes.

Run an initial build on the project with the MiniZed hardware now configured:

petalinux-build

Create Custom Recipe

Through some trial and error, I found that the best way to port over the Edge Impulse C SDK into embedded Linux was to add it as its own custom recipe versus adding it as a user-defined application under the application recipe in the PetaLinux project.

To create a custom recipe in PetaLinux's Yocto-based filed structure, start by adding a directory in /<project directory>/edge-impulse-projects/minized_emmc_enhanced_2019_2/project-spec/meta-user/ titled recipe-<desired recipe name>. Special characters and underscores are not allowable in recipe names.

mkdir -p ./edge-impulse-projects/minized_emmc_enhanced_2019_2/project-spec/meta-user/recipes-edgeimpulse

Within the new custom recipe directory, a subdirectory is needed for the specific element being added since a recipe can contain multiple elements (for example, the apps recipe can contain multiple apps).

Create a directory for the Edge Impulse C SDK:

mkdir -p ./edge-impulse-projects/minized_emmc_enhanced_2019_2/project-spec/meta-user/recipes-edgeimpulse/ei-c-sdk

Finally a bitbake needs to be created for the C SDK to tell PetaLinux how to build it into the filesystem. Using your file editor of choice create a new bitbake file. The bitbake file must be titled the same as the directory it is within. Thus I named both the directory and the bitbake file ei-c-sdk.

cat > ./edge-impulse-projects/minized_emmc_enhanced_2019_2/project-spec/meta-user/recipes-edgeimpulse/ei-c-sdk/ei-c-sdk.bb

The bitbake I wrote for this simply clones the C SDK source code from Edge Impulse's Github repository and initializes the submodules for the Mbed TLS and QCBOR dependencies, then installs it in the root directory of the embedded Linux image.

The do_configure_prepend function simply follows the installation directions specified in the README of the C SDK source code Github repository and the do_install function handles installing the resulting files in the root directory of the embedded Linux image.

I did find that a specific revision of the Github repository must be specified for this particular repository (usually I can just get away with specifying the master branch). I'm guessing that's due to the specific Mbed and QCBOR repository version dependencies.

Bitbake code:

SUMMARY = "Edge Impulse C SDK"
DESCRIPTION = "Edge Impulse C SDK source code"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"

#FILESEXTRAPATHS_prepend := "${THISDIR}/files:"

SRC_URI =  "git://github.com/edgeimpulse/ingestion-sdk-c.git;protocol=git;branch=master;rev=0431e2bdb29a9ee7deafa9a038704fa0af000f87 \
"

S = "${WORKDIR}/git"
RDEPENDS_${PN} += "bash"

do_configure_prepend() {
    cd ${S}
    git config --global http.sslverify "false"
    git submodule update --init --recursive
    make
}

do_install() { 
    install -d ${D}/home/root/
    cp -r ${S}/* ${D}/home/root/
}

FILES_${PN} = " \
    /home/root/* \	
"

Finally, for the root filesystem to be aware that this new Edge Impulse recipe is now an option to be included in the next build, specify it in the user root filesystem configuration file: /<project directory>/edge-impulse-projects/minized_emmc_enhanced_2019_2/project-spec/meta-user/conf/user-rootfsconfig as shown in the last two lines below:

# Add modules, apps, and packages not in the default rootfs config

# Local to project
CONFIG_peekpoke
CONFIG_gpio-demo

# Minized specific customizations
CONFIG_i2csensor
CONFIG_dialog-control
CONFIG_get-gpio-offsets
CONFIG_minized-misc
CONFIG_minized-wireless

# LIS2DS12 Accelerometer driver module
CONFIG_lis2ds

# Extra packages
CONFIG_iperf3
CONFIG_nano
CONFIG_hostapd
CONFIG_iw
CONFIG_bluez5-doc
CONFIG_pulseaudio-module-bluetooth-discover
CONFIG_pulseaudio-module-bluetooth-policy
CONFIG_pulseaudio-module-bluez5-discover
CONFIG_pulseaudio-module-bluez5-device
CONFIG_pulseaudio-module-switch-on-connect

# Edge Impulse C SDK
CONFIG_ei-c-sdk

With the EI custom recipe completed, it's ready for its stand alone build before we build the root filesystem as a whole:

petalinux-build -c ei-c-sdk

Build the Root Filesystem

The root filesystem in the BSP for the MiniZed already contains most of the packages needed for compatibility with Edge Impulse such as SSH and build-essential. To cover the remaining packages needed such as gstreamer I found that enabling all of the PetaLinux package groups for the root filesystem using the the root filesystem configuration editor in PetaLinux covered all the bases.

I've personally found this is the easiest way to create an embedded Linux containing pretty much anything you might when you're not in a pinch to cut down on the amount of space the root filesystem fits into. Only enable the base package group in each category, this project doesn't need any of the -dev or -dbg versions of the package groups.

To access the root filesystem configuration editor in the PetaLinux project:

petalinux-config -c rootfs

Also enable the EI C SDK user package generated from the EI custom recipe to be included in the next build of the root filesystem.

Then build the root filesystem:

petalinux-build -c rootfs

The first time this version of the root filesystem is built will take quite a while (also depending on your host PC specs).

Build PetaLinux Project

With the EI custom recipe and the root filesystem having each had their own build run at this point, it's time to build the PetaLinux project as a whole:

petalinux-build

Again, the first time this is run, it will take a while.

Package Boot Binary

After the PetaLinux project has been built, the boot binary image needs to be generated:

petalinux-package --boot --fsbl <project directory>/images/linux/zynq_fsbl.elf --fpga <project directory>/images/linux/system.bit --u-boot

Upload Image to MiniZed

To upload a new root filesystem to the MiniZed that's an ext4 type booted from the eMMC of the MiniZed, we first have to revert back to the initramfs root filesystem so that the ext4 partition of the eMMC is not locked by the filesystem running on it.

Since this PetaLinux project was built using the eMMC boot BSP for the MiniZed, the kernel and boot binary for the initramfs root filesystem image can be found in the pre-built directory of the project. The kernel is image.ub and the initramfs boot binary is BOOT_EMMC.BIN

Using a USB flash drive (you will need a second USB cable plugged into the AUX Power micro-USB port of the MiniZed to provide power to its USB Type A port), copy the initramfs kernel & BOOT_EMMC.BIN to /run/media/mmcblk1p1 and write the BOOT_EMMC.BIN to the MiniZed's QSPI flash. Then reboot the board into the initramfs system.

root@MiniZed:~# cp /run/media/sda1/BOOT_EMMC.BIN /run/media/mmcblk1p1
root@MiniZed:~# cp /run/media/sda1/image.ub /run/media/mmcblk1p1
root@MiniZed:~# flashcp /run/media/mmcblk1p1/BOOT.BIN /dev/mtd0
root@MiniZed:~# reboot

Modify the wpa_supplicant.conf living in the /run/media/mmcblk1p1 directory with your network's ESSID and password and reboot again. Once rebooted, run the wi-fi setup script to connect the MiniZed to your wireless network:

root@MiniZed:~# wifi.sh

As I outlined previously, the initramfs image doesn't have the necessary libraries for handling ext4 so we have to copy them over manually (I covered how to generate these library files in a past project here). I found that using a secure copy straight from my PC to the MiniZed is the best method of doing this:

scp -p ./lib/libcom_err.so     root@192.168.1.xxx:/lib
scp -p ./lib/libcom_err.so.2   root@192.168.1.xxx:/lib
scp -p ./lib/libcom_err.so.2.1 root@192.168.1.xxx:/lib
scp -p ./lib/libe2p.so         root@192.168.1.xxx:/lib
scp -p ./lib/libe2p.so.2       root@192.168.1.xxx:/lib
scp -p ./lib/libe2p.so.2.3     root@192.168.1.xxx:/lib
scp -p ./lib/libext2fs.so      root@192.168.1.xxx:/lib
scp -p ./lib/libext2fs.so.2    root@192.168.1.xxx:/lib
scp -p ./lib/libext2fs.so.2.4  root@192.168.1.xxx:/lib

scp -p ./sbin/mke2fs.e2fsprogs root@192.168.1.xxx:/sbin
scp -p ./sbin/mkfs.ext4        root@192.168.1.xxx:/sbin

After the eMMC is partitioned & formatted the first time out of the box, you don't need to do it again unless it gets corrupted. In my case, since I'm updating the ext4 filesystem, I don't need to reformat the partition of the eMMC it's on (/dev/mmcblk1p2). This means I can skip straight to mounting it to a local directory and erasing the old ext4 root filesystem on it (if you haven't previously formatted the MiniZed's eMMC, follow the first part of my past project here).

root@MiniZed:~# mkdir /mnt/emmc_rootfs
root@MiniZed:~# mount /dev/mmcblk1p2 /mnt/emmc_rootfs
root@MiniZed:~# rm -r /mnt/emmc_rootfs/*

Copy over the new rootfs.ext4 from the USB flash drive and mount a temporary drive to the second partition to extract it into for the extra space. Copy the root filesystem files back into the main drive of the eMMC's second partition then unmount and delete the other temporary drive directory.

root@MiniZed:~# cp /run/media/sda1/<new_linux>/rootfs.ext4 /mnt/emmc_rootfs
root@MiniZed:~# mkdir /mnt/ext4
root@MiniZed:~# mount /mnt/emmc_rootfs/rootfs.ext4 /mnt/ext4/ -o loop
root@MiniZed:~# cp -rf /mnt/ext4/* /mnt/emmc_rootfs/
root@MiniZed:~# umount /mnt/ext4/
root@MiniZed:~# rm /mnt/emmc_rootfs/rootfs.ext4

Copy the new kernel and boot binary for the new ext4 root filesystem to the first partition of the eMMC and flash the QSPI with the new boot binary.

root@MiniZed:~# cp /run/media/sda1/<new_linux>/image.ub /run/media/mmcblk1p1
root@MiniZed:~# cp /run/media/sda1/<new_linux>/BOOT.BIN /run/media/mmcblk1p1
root@MiniZed:~# flashcp /run/media/mmcblk1p1/BOOT.BIN /dev/mtd0

Finally, fully power cycle the board by sending the shutdown command then using the Reset button (SW2) on the board to power it back on:

root@MiniZed:~# shutdown -h now

Again after rebooting the MiniZed, run the wi-fi setup script to connect the MiniZed to your wireless network:

root@MiniZed:~# wifi.sh

Try EI Ingestion Service Test Application

Finally, to test the validity of the C SDK install, run the example application provided:

root@MiniZed:~# cd ./ingestion-sdk-c
root@MiniZed:~# ./ingestion-sdk-example
Encoded file:
a3 69 70 72 6f 74 65 63 74 65 64 a2 63 76 65 72 62 76 31 63 61 6c 67 65 48 53 32 35 36 69 73 69 67 6e 61 74 75 72 65 78 40 66 30 62 63 39 34 65 65 36 31 36 66 39 30 32 61 35 64 62 63 65 32 61 33 32 66 33 39 34 36 38 37 33 38 33 31 61 37 34 36 62 38 30 31 39 62 62 33 62 33 65 62 30 38 35 30 61 39 62 31 37 30 34 38 67 70 61 79 6c 6f 61 64 a5 6b 64 65 76 69 63 65 5f 6e 61 6d 65 71 61 63 3a 38 37 3a 61 33 3a 30 61 3a 32 64 3a 31 62 6b 64 65 76 69 63 65 5f 74 79 70 65 73 44 49 53 43 4f 2d 4c 34 37 35 56 47 2d 49 4f 54 30 31 41 6b 69 6e 74 65 72 76 61 6c 5f 6d 73 f9 49 00 67 73 65 6e 73 6f 72 73 83 a2 64 6e 61 6d 65 64 61 63 63 58 65 75 6e 69 74 73 64 6d 2f 73 32 a2 64 6e 61 6d 65 64 61 63 63 59 65 75 6e 69 74 73 64 6d 2f 73 32 a2 64 6e 61 6d 65 64 61 63 63 5a 65 75 6e 69 74 73 64 6d 2f 73 32 66 76 61 6c 75 65 73 9f 83 fa c1 1c f5 c3 fa 3c f5 c2 8f fa 3f 9a e1 48 83 fa c1 1d 47 ae fa 3d 23 d7 0a fa 3f a3 d7 0a 83 fa c1 11 eb 85 fa 3c f5 c2 8f fa 3f 9d 70 a4 83 fa c1 12 3d 71 fa 3c 23 d7 0a f9 3d 00 ff

Stay tuned for Part 2 covering how to get the MiniZed connected to Edge Impulse Studio!