Generating an Embedded Linux Image for the ZynqberryZero

The Zynqberry series boards from Trenz Electronic are fun little Xilinx Zynq-based FPGA development boards that adopt the popular form factors from their microcontroller counter parts, the Raspberry Pi boards. As you can guess from the name, the ZynqberryZero takes on the form factor of the Raspberry Pi Zero. And while I've covered how to create a hardware design and bare-metal application for the the ZynqberryZero earlier this month, it's not really an apples-to-apples comparison between the ZynqberryZero and the Raspberry Pi Zero without having an embedded Linux image to run on the ZynqberryZero.

The ZynqberryZero boasts the same hardware peripherals for the embedded Linux image to take advantage of including a mico-USB OTG port, mini-HDMI port, MIPI/CSI camera connector, and the 40-pin GPIO with standard Raspberry Pi pinout for SPI, I2C, GPIO, 5V power, and ground.

It is worth noting that while the base Raspbian image for the Raspberry Pi Zero is Debian-based, the PetaLinux tool set generates lightweight Red Hat images that uses RPM (Red hat Package Manager). So keep that in mind if you're trying to migrate/replicate any of your Raspberry Pi Zero projects on the ZynqberryZero.

Create PetaLinux Project

First things first, source the PetaLinux tools in your environment from the command line:

~$ source /tools/Xilinx/PetaLinux/2019.2/settings.sh

Then change directories into the desired folder you'd like the PetaLinux project to be in. I personally like to put the PetaLinux project in the top-level directory of the Vivado project that I created the hardware design for the board in. So in my case, my Vivado project for the ZynqberryZero is located in zynqberry_zero_prj in my home folder:

~$ cd ./zynqberry_zero_prj/

Next create the PetaLinux project with the following command. Pass project for the type flag and zynq for the template flag. Feel free to pass as desired name for the project name however.

~/zynqberry_zero_prj$ petalinux-create --type project --template zynq --name zynqberry_zero_os

Change directories into the project directory created by the last command:

~/zynqberry_zero_prj$ cd ./zynqberry_zero_os/

Import Hardware Design

The first thing that needs to be done in a new PetaLinux project is to import the appropriate hardware platform exported from Vivado for the target FPGA development board.

In this case, that would be the hardware platform exported in the last step of my previous project post covering how to generate the base hardware design for the ZynqberryZero located in the top level directory of the Vivado project that I've also created the PetaLinux project in.

~/zynqberry_zero_prj/zynqberry_zero_os/$ petalinux-config --get-hw-description ../

After importing the hardware platform, the system hardware configuration editor will automatically launch:

Configure the system hardware for the embedded Linux image for the ZynqberryZero to to look for the boot binary in the QSPI flash memory of the ZynqberryZero and the device tree on the SD card.

Under Subsystem AUTO Hardware settings > Advanced bootable images storage Settings, change image storage media to primary flash for boot image settings and change image storage media to primary sd for dtb image settings.

To be more closely matched functionality-wise to the Raspberry Pi Zero, I chose to change the root filesystem type from initramfs to extended (EXT).

The initramfs filesystem is writeable, but not persistent. Thus any changes are lost at each reboot because it is simply a complete set of directories as a normal root filesystem that is compressed into a single cpio archive loaded by the kernel at boot. The extended filesystem is the traditional filesystem type most think of. In this case it is ext4 in PetaLinux.

Change the root filesystem type from INITRAMFS to EXT. And uncheck the option for TFTP boot if you don't plan to boot your ZynqberryZero over a network connection.

Exit and save the system hardware configuration editor.

Linux Kernel

Next configure kernel in the PetaLinux project by launching the configuration editor:

~/zynqberry_zero_prj/zynqberry_zero_os/$ petalinux-config -c kernel

Use the search function in the kernel configuration editor by pressing the / key at any time. Verify/modify the kernel to configure the following kernel modules:

CONFIG_MII=y
CONFIG_XILINX_GMII2RGMII=y
CONFIG_USB_USBNET=y
CONFIG_USB_NET_AX8817X=y
CONFIG_USB_NET_AX88179_178A=y
CONFIG_USB_NET_CDCETHER=y
# CONFIG_USB_NET_CDC_EEM is not set
CONFIG_USB_NET_CDC_NCM=y
# CONFIG_USB_NET_HUAWEI_CDC_NCM is not set
# CONFIG_USB_NET_CDC_MBIM is not set
# CONFIG_USB_NET_DM9601 is not set
# CONFIG_USB_NET_SR9700 is not set
# CONFIG_USB_NET_SR9800 is not set
# CONFIG_USB_NET_SMSC75XX is not set
CONFIG_USB_NET_SMSC95XX=y
# CONFIG_USB_NET_GL620A is not set
CONFIG_USB_NET_NET1080=y
# CONFIG_USB_NET_PLUSB is not set
# CONFIG_USB_NET_MCS7830 is not set
# CONFIG_USB_NET_RNDIS_HOST is not set
CONFIG_USB_NET_CDC_SUBSET_ENABLE=y
CONFIG_USB_NET_CDC_SUBSET=y
# CONFIG_USB_ALI_M5632 is not set
# CONFIG_USB_AN2720 is not set
CONFIG_USB_BELKIN=y
CONFIG_USB_ARMLINUX=y
# CONFIG_USB_EPSON2888 is not set
# CONFIG_USB_KC2190 is not set
CONFIG_USB_NET_ZAURUS=y
# CONFIG_USB_NET_CX82310_ETH is not set
# CONFIG_USB_NET_KALMIA is not set
# CONFIG_USB_NET_QMI_WWAN is not set
# CONFIG_USB_NET_INT51X1 is not set
# CONFIG_USB_SIERRA_NET is not set
# CONFIG_USB_VL600 is not set
# CONFIG_USB_NET_CH9200 is not set
CONFIG_FB_SIMPLE=y
# CONFIG_FRAMEBUFFER_CONSOLE is not set
CONFIG_SND_SIMPLE_CARD_UTILS=y
CONFIG_SND_SIMPLE_CARD=y
CONFIG_USBIP_CORE=y
# CONFIG_USBIP_VHCI_HCD is not set
# CONFIG_USBIP_HOST is not set
# CONFIG_USBIP_VUDC is not set
# CONFIG_USBIP_DEBUG is not set

This enables the kernel drivers for the Ethernet controller/USB hub chip, the LAN9514. Exit and save the system hardware configuration editor for the kernel.

Root Filesystem

Configure root filesystem by launching its respective configuration editor in PetaLinux:

~/zynqberry_zero_prj/zynqberry_zero_os/$ petalinux-config -c rootfs

Enable the following filesystem packages (again you can use the search function in the kernel configuration editor by pressing the / key at any time to find exactly where each packages are in the menu):

i2c-tools
alsa-plugins
alsa-lib-dev
libasound
alsa-conf-base
alsa-conf
alsa-utils
alsa-utils-aplay
busybox-httpd

Next, add the custom applications for the ZynqberryZero:

~/zynqberry_zero_prj/zynqberry_zero_os$ petalinux-create -t apps --name fbgrab --enable
~/zynqberry_zero_prj/zynqberry_zero_os$ petalinux-create -t apps --name ffmpeg --enable
~/zynqberry_zero_prj/zynqberry_zero_os$ petalinux-create -t apps --name i2cpick --enable
~/zynqberry_zero_prj/zynqberry_zero_os$ petalinux-create -t apps --name rpicam --enable
~/zynqberry_zero_prj/zynqberry_zero_os$ petalinux-create -t apps --name startup --enable
~/zynqberry_zero_prj/zynqberry_zero_os$ petalinux-create -t apps --name webfwu --enable

The source files for these custom applications are available in the Trenz example design, so simply copy them over:

Delete everything generated by PetaLinux within the fbgrab directory and the copy over contents of /zbzerodemo1/os/petalinux/project-spec/meta-user/recipes-apps/fbgrab from the Trenz example design (which is just the bitbake file).

Also delete everything generated by PetaLinux within the ffmpeg directory and the copy over contents of /zbzerodemo1/os/petalinux/project-spec/meta-user/recipes-apps/ffmpeg from the Trenz example design (the ffmpeg folder with source files and the bitbake file).

For i2cpick, rpicam, startup, and webfwu, copy over the contents of their respective files directories from the Trenz example design (/zbzerodemo1/os/petalinux/project-spec/meta-user/recipes-apps/<app name>/files) to the files directory within the PetaLinux project (/zynqberry_zero_prj/zynqberry_zero_os/project-spec/meta-user/recipes-apps/<app name>/files)

For some reason the TE audio codec kernel module was missing from the ZynqberryZero example design, so I copied it from my main Zynqberry PetaLinux project.

And added the following line to user-rootfsconfig in /zynqberry_zero_prj/zynqberry_zero_os/project-spec/meta-user/conf for the root filesystem configuration editor to pick it up so I could go back and enable it under modules (run petalinux-config -c rootfs again).

CONFIG_te-audio-codec

Finally, copy over recipes-core recipe folder from the Trenz example design (/zbzerodemo1/os/petalinux/project-spec/meta-user/recipes-core) to the meta-user folder within the PetaLinux project: /zynqberry_zero_prj/zynqberry_zero_os/project-spec/meta-user/.

This recipe contains the basic TCP/IP networking initialization scripts and configuration files which are the high level tools for configuring network interfaces in a Linux image. The ZynqberryZero's Ethernet controller/USB hub chip, the LAN9514, allows for it to be connected to a network via a microUSB to Ethernet adapter such as this one:

Thus you'll notice that the network interface is established as an Ethernet port (eth0) in the interfaces file within the recipe.

Device Tree

In order for the kernel to be able to see what hardware is available in the system, the proper nodes need to be added to the device tree that correlate with the block design in Vivado.

Add the following device tree nodes to system-user.dtsi in /zynqberry_zero_prj/zynqberry_zero_os/project-spec/meta-user/recipes-bsp/device-tree/files

/include/ "system-conf.dtsi" 
/{
};

/{
    #address-cells = <1>;
    #size-cells = <1>;
  
    reserved-memory {
        #address-cells = <1>;
        #size-cells = <1>;
        ranges;
		
        hdmi_fb_reserved_region@1FC00000 {
            compatible = "removed-dma-pool";
            no-map;
            reg = <0x1FC00000 0x400000>;
        }; 
    };
  
    hdmi_fb: framebuffer@0x1FC00000 {           
        compatible = "simple-framebuffer";
        
        reg = <0x1FC00000 (1280 * 720 * 4)>;    
        width = <1280>;                         
        height = <720>;                       
        stride = <(1280 * 4)>;                 
        format = "a8b8g8r8";
        status = "okay";
    };
 
    vcc_3V3: fixedregulator@0 {
        compatible = "regulator-fixed";
        regulator-name = "vccaux-supply";
        regulator-min-microvolt = <3300000>;
        regulator-max-microvolt = <3300000>;
        regulator-always-on;
    };
};

&qspi {
    #address-cells = <1>;
    #size-cells = <0>;
    status = "okay";
    flash0: flash@0 {
        compatible = "jedec,spi-nor";
        reg = <0x0>;
        #address-cells = <1>;
        #size-cells = <1>;
        spi-max-frequency = <50000000>;
        partition@0x00000000 {
            label = "boot";
            reg = <0x00000000 0x00500000>;
        };
        partition@0x00500000 {
            label = "bootenv";
            reg = <0x00500000 0x00020000>;
        };
        partition@0x00520000 {
            label = "kernel";
            reg = <0x00520000 0x00a80000>;
        };
        partition@0x00fa0000 {
            label = "spare";
            reg = <0x00fa0000 0x00000000>;
        };
    };
};

&video_out_axi_vdma_0 {
	status = "disabled";	
};

&video_in_axi_vdma_1 {
	status = "disabled";
};

&gpio0 {
    interrupt-controller;
    #interrupt-cells = <2>;
};
  
/* I2C1 */  
&i2c1 {
	#address-cells = <1>;
	#size-cells = <0>;

	i2cmux: i2cmux@70  {
		compatible = "nxp,pca9540";
		#address-cells = <1>;
		#size-cells = <0>;
		reg = <0x70>;

		ID_I2C@0 {
			#address-cells = <1>;
			#size-cells = <0>;
			reg = <0>;
		};
		CSI_I2C@1 {
			#address-cells = <1>;
			#size-cells = <0>;
			reg = <1>;
		};
	};

};

/* USB */  
/{
    usb_phy0: usb_phy@0 {
        compatible = "ulpi-phy";
        #phy-cells = <0>;
        reg = <0xe0002000 0x1000>;
        view-port = <0x0170>;
        drv-vbus;
    };
};
  
&usb0 {
    usb-phy = <&usb_phy0>;
};

You might have noticed that I didn't use the system-user.dtsi provided in the example design from Trenz...

I mentioned back in the creation of the base hardware design in Vivado, both of the AXI Video DMA IP blocks were named axi_vdma_0 in the block design, which causes Vivado to try to map them to the same address space in the DDR. In the device tree provided in the example design, this same error was carried over where both of the nodes for the Video DMA block for video in and video out are titled axi_vdma_0. This prevents the PetaLinux project from building successfully, so I again changed the video in node to be named axi_vdma_1 (&video_in_axi_vdma_1) so it now matches the hardware node name in the hardware platform and the embedded Linux image isn't trying to use the same video DMA node for both video in and video out.

I also removed a bunch of the extraneous comments and all of the hardware configurations for the 128MB version of the ZynqberryZero since I'm using the 512MB version.

Build PetaLinux Project

With the project configured and customized to the ZynqberryZero, build PetaLinux project:

~/zynqberry_zero_prj/zynqberry_zero_os/$ petalinux-build

Which should succeed at this point with the proper modifications made in the previous steps.

Generate Boot Binary

Once the PetaLinux project as been built successfully, you'll see all of the output files such as the kernel image, root filesystem, Zynq FSBL ELF file, etc. be output into the /zynqberry_zero_prj/zynqberry_zero_os/images/linux directory of the PetaLinux project. The only thing that isn't there is the boot binary, which is generated manually with the following command:

~/zynqberry_zero_prj/zynqberry_zero_os$ petalinux-package --boot --fsbl ./images/linux/zynq_fsbl.elf --fpga ./images/linux/system.bit --u-boot

Since you can package different things into the boot binary depending on how you've configured your Linux image and hardware, that's why it isn't generated automatically in the PetaLinux project build. In this case, I'm including the Zynq FSBL, FPGA bitstream, and U-BOOT in the boot binary.

Load SD Card

Prepare a MircoSD card (at least an 8GB, class 10) by partitioning it with a FAT32 partition 500MB in size with 4MB of free space preceding it. Then make the rest of the SD card an EXT4 partition:

This partitioning scheme is per the PetaLinux user guide (UG1144). The FAT32 partition is where the kernel image and device tree file will live, while the EXT4 partition is where the root filesystem will live.

After partitioning the SD card appropriately mount each partition to some point on your host system:

~$ sudo mount /dev/sdc1 /media/BOOT
~$ sudo mount /dev/sdc2 /media/rootfs

Copy the kernel image and device tree file to the first (FAT32) partition:

~$ sudo cp ./zynqberry_zero_prj/zynqberry_zero_os/images/linux/image.ub /media/BOOT/
~$ sudo cp ./zynqberry_zero_prj/zynqberry_zero_os/images/linux/system.dtb /media/BOOT/

And unzip the root filesystem onto the second (EXT4) partition:

~$ sudo tar xvf ./zynqberry_zero_prj/zynqberry_zero_os/images/linux/rootfs.tar.gz -C /media/rootfs/

Run the sync command to ensure the file transfers have been fully completed (you can't always trust that just because the command line was returned to you that the transfer is actually complete -- I've learned this the hard way):

~$ sync

Finally unmount each of the partitions to eject the SD card from your host PC:

~$ sudo umount /media/BOOT/
~$ sudo umount /media/rootfs/

Create QSPI Boot Application in Vitis

Xilinx Zynq devices in CLG225 package, such as the one used on both the Zynqberry and ZynqberryZero, do not support booting from an SD Card directly. Thus you have to perform the primary boot (Zynq FSBL) from the QSPI. Therefore, the boot binary (BOOT.BIN) needs to be programmed into the QSPI flash of the ZynqberryZero since it contains the Zynq FSBL.

In order to get the boot binary image onto the QSPI of the ZynqberryZero, we need a different FSBL that will boot the ARM-core of the Zynq from JTAG for programming. To do this, we'll create an FSBL embedded application in Vitis. I'm using the same Vitis workspace I created for the ZynqberryZero in my last project post.

Before creating the embedded application for the FSBL, the BSP (board support package) needs the Xilinx FAT file system (FFS) library added to it, xilffs. Open platform.spr from the Explorer window in Vitis. Select Board Support Package under standalone on ps7_cortex9_0 and the button for Modify BSP Settings... will appear in the window to the right.

In the Overview tab of the window that pops up for board support package settings, you'll see a list of libraries that can be enabled in the BSP. Check the box next to xilffs then click OK.

Rebuild the platform with ctrl+B.

Next create a new application project by selecting New > Application Project...

Give the project the desired name, selecting the option to create a new system project for it. Select the ZynqberryZero custom platform project to base it on, and target the ARM-core 0 processor of the Zynq in C. Finally, select the Zynq FSBL application template for it.

Open main.c and make the following modifications starting at line 286:

/*
 * Print the FSBL Banner
 */
fsbl_printf(DEBUG_GENERAL,"\n\rXilinx First Stage Boot Loader \n\r");
fsbl_printf(DEBUG_GENERAL,"Release %d.%d %s-%s\r\n", SDK_RELEASE_YEAR, SDK_RELEASE_QUARTER,__DATE__,__TIME__);

xil_printf("\r\n--------------------------------------------------------------------------------\r\n");
xil_printf("Xilinx Zynq First Stage Boot Loader to write QSPI Flash (TE modified) \n\r");
xil_printf("Release %d.%d %s-%s\r\n", SDK_RELEASE_YEAR, SDK_RELEASE_QUARTER, __DATE__,__TIME__);
#ifdef XPAR_PS7_DDR_0_S_AXI_BASEADDR

/*
 * DDR Read/write test 
 */
//	Status = DDRInitCheck();
//	if (Status == XST_FAILURE) {
//		fsbl_printf(DEBUG_GENERAL,"DDR_INIT_FAIL \r\n");
//		/* Error Handling here */
//		OutputStatus(DDR_INIT_FAIL);
//		/*
//		 * Calling FsblHookFallback instead of Fallback
//		 * since, devcfg driver is not yet initialized
//		 */
//		FsblHookFallback();
//	}

Then add the JTAG boot mode mask starting at line 382:

/*
 * Read bootmode register
 */
BootModeRegister = Xil_In32(BOOT_MODE_REG);
BootModeRegister &= BOOT_MODES_MASK;
BootModeRegister = JTAG_MODE;

These edits are simply disabling the DDR read/write test since it's not necessary and changing the boot mode mask to JTAG boot mode.

Save main.c and build the application project.

Flash QSPI

With the FSBL for programming created, plug in the ZynqberryZero without the SD card in it:

Right-click on the FSBL application project name in the Explorer window and select Program Flash.

In the window that pops up to program the flash memory, browse to the BOOT.BIN in /zynqberry_zero_prj/zynqberry_zero_os/images/linux/ for Image File and the JTAG FSBL ELF file: /zynqberry_zero_prj/vitis_workspace/qspi_flash_app/Debug/qspi_flash_app.elf for FSBL file.

Check the box to verify after flash the click Program.

Wait for the console to read out that the flash operation was successful.

Boot ZynqberryZero

Unplug the ZynqberryZero from your host PC and insert the prepared SD card into the SD card slot. Open your serial terminal application of choice with a baud rate of 115200.

Plug in the ZynqberryZero and connect to its UART with the serial terminal application. The initial boot will fail due to the u-boot still needing to be configured to load the kernel image and device tree from the SD card before continuing with the rest of the boot sequence since the PetaLinux project was configured to look for the device tree on the SD card.

Since boot will fail during the u-boot bootloader, it will automatically launch into the u-boot terminal denoted in this case by the Zynq> in the command line:

Zynq>

Define the commands in u-boot to load in the kernel image and device tree file from the SD card with the following two commands:

Zynq> setenv cp_kernel2ram 'fatload mmc 0 ${netstart} ${kernel_img}'
Zynq> setenv cp_dtb2ram 'fatload mmc 0 ${dtbnetstart} ${dtb_img}'

Set the default boot command to run the commands to load in the kernel image and device tree file from the SD card

Zynq> setenv default_bootcmd 'run cp_kernel2ram && run cp_dtb2ram && bootm ${netstart} - ${dtbnetstart}'

Then set the boot arguments to specify the device node of where the root filesystem is located on the SD card and the root filesystem's type:

Zynq> setenv bootargs 'console=ttyPS0,115200 earlyprintk root=/dev/mmcblk0p2 rootfstype=ext4 ru rootwait'

Finally, save the new u-boot environment. This will write the new commands to the QSPI flash where the boot binary lives:

Zynq> saveenv

And boot the ZynqberryZero:

Zynq> boot

At this point, you'll see the normal boot sequence commence and finally bring you to the login. Which the username and password with both be root unless otherwise modified in the PetaLinux project:

As you can see, getting PetaLinux running on the Zynqberry and ZynqberryZero boards can be a bit tricky given that they can't boot straight from an SD card. But still one of my favorite little FPGA development boards!