Empowering DUNE: Creating Petalinux 2022.2 OS Image for KRIA

Before start, please review this tutorial.

In the DUNE experiment, the photon detection system, DAPHNE, has undergone an evolution leading to the use of SoM (System on Modules). This transition aims to achieve greater compactness in both hardware and software task implementation, enhanced system control, and operational security. Given that the DAPHNE system will be located 1 km underground in the Fermilab caverns, access for subsequent system services will be challenging. Hence, high security and robustness are required in the system implementation. This is where the KRIA KR260 takes center stage as the chosen SoM to manage the photon detection system.

In this tutorial, I will demonstrate the necessary steps to create a Petalinux 2022.2 image. This operating system will be embedded on the PS side of the KRIA KR260 SoM, equipped with essential configurations to enable peripherals, essential Linux packages, Python, and C for running projects designed for high-performance embedded systems. Additionally, it will provide comprehensive support for Vitis SDK to implement custom applications using Xilinx's development stack and all necessary tools to work with this system. The tutorial will cover step-by-step instructions on configuring the kernel, the root file system, creating a project from scratch, necessary manual modifications to enable support for peripherals like SPI and I2C, packaging the image, incorporating an Overlay Platform created in Vivado, and mounting it onto the Petalinux image once it has been sent to the board via SSH.

At times, in the Vitis 2022.2 on Ubuntu 22.04 installation project post, KRS doesn't fully support PetaLinux for the KR260. It solely supports cross-compiling an Ubuntu 22.04 image as the root filesystem. Consequently, there might be instances where switching back to the gcc-multilib GNU C compiler on the host machine becomes necessary to compile PetaLinux root filesystems. As a result, any PetaLinux projects intended for the KR260 may not involve KRS and will primarily consist of accelerated applications and/or regular Linux applications.

Upgrade the Petalinux SDK (Optional)

The BSPs for the Kria KR260 and KV260 require the Update to be applied to PetaLinux; otherwise, the project might encounter build issues or may not build at all. Begin by sourcing the PetaLinux tools in the host machine environment.

source /tools/Xilinx/PetaLinux/2022.2/settings.sh

Next, upgrade the PetaLinux SDK with Update1:

petalinux-upgrade -u http://petalinux.xilinx.com/sswreleases/rel-v2022/sdkupdate/2022.2_update1/ -p "aarch64" --wget-args "--wait 1 -nH --cut-dirs=4"

Source the Petalinux folder

To use the Petalinux Toolkit, it is necessary to source the Petalinux settings file in the main folder. To do this, it is just necessary to open a linux terminal and run the follow command:

source /tools/Xilinx/PetaLinux/2022.2/settings.sh

Create a new Petalinux 2022.2 project for KR260 BSP

First, it's essential to disable the HTTP and HTTPS proxy to prevent any issues during the execution of Python scripts in the petalinux-build process. Use the following commands:

unset http_proxy 
unset https_proxy

Next, navigate to the desired location to create the PetaLinux project. For example, create the project in the top-level folder of the Vivado directory where the hardware design was created. Once in the desired directory location, create a PetaLinux project using the Kria KR260 BSP (download available here). Name the project as linux_os and change to the directory.

In this case, we downloaded the BSP file into the /tools/Xilinx/Petalinux/2022.2/bsp_files/ folder

Run the follow commands:

cd <project folder path>/Kria_KR260/
petalinux-create --type project -s /tools/Xilinx/PetaLinux/2022.2/bsp_files/xilinx-kr260-starterkit-v2022.2-10141622.bsp --name linux_os
cd linux_os

Then import the hardware platform exported from Vivado that has the hooks for an accelerated design, this imports the XSA file and will use it as platform for Petalinux:

petalinux-config --get-hw-description ../

In the system configuration editor, start by enabling FPGA manager. Then set the root filesystem type to INITRD with the INITRAMFS/INITRD Image name set as petalinux-initramfs-image, and disable TFTP boot if you don't plan to use it.

FPGA Manager --> Fpga Manager[*]
Image Packaging Configuration --> Root Filesystem Type --> INITRD[*]
Image Packaging Configuration --> INITRAMFS/INITRD Image name --> petalinux-initramfs-image
Image Packaging Configuration --> Copy final images to tftpboot[]

Changing the image name to petalinux-initramfs-image (the default is petalinux-image-minimal) is extremely important. Failure to do so will result in the ramdisk image not being generated in the early stages of the boot process. This omission will cause the root filesystem to be non-persistent between power cycles on the KR260.

Exit the system configuration editor saving the changes.

Pre-building Petalinux

To save time in subsequent steps, try building the project for the first time before making customizations in the kernel and the root filesystem. Remember to be located in the ~/Kria_KR260/linux_os$ folder path. To perform the initial build, use the following command:

Note: This initial iteration of the project build takes a significant amount of time. In my case, it took nearly two hours to complete.

petalinux-build

The expected output after the completion of the building process is as follows.

Customizing and Configuring the Kernel

If you wish to add any kernel packages, access the kernel configuration editor and make the necessary selections. To enable hardware driver support in the kernel, execute the following command from the ~/Kria_KR260/linux_os$ folder path:

petalinux-config -c kernel

Once the Menu Window opens, navigate to the Device Drivers section and ensure the following drivers are enabled:

• Device Drivers > I2C support > I2C Hardware Bus support, enable:
• Cadence I2C Controller
• Xilinx I2C Controller
• Device Drivers > SPI support, enable:
• Cadence SPI Controller
• Cadence Quad SPI Controller
• Xilinx SPI Controller common module
• Xilinx ZynqMP GQSPI Controller
• User mode SPI device driver support

Device Drivers --> I2C Support --> i2C Hardware Bus Support --> [*] Cadenve I2C Controller
Device Drivers --> I2C Support --> I2C Hardware Bus Support --> [*] Xilinx I2C Controller
Device Drivers --> SPI Support --> [*] Cadence SPI Controller
Device Drivers --> SPI Support --> [*] Cadence Quad SPI Controller
Device Drivers --> SPI Support --> [*] Xilinx SPI Controller common module
Device Drivers --> SPI Support --> [*] Xilinx ZynqMP GQSPI Controller
Device Drivers --> SPI Support --> [*] User mode SPI device driver support

Customizing and Configuring the Root Filesystem

To include XRT support and necessary packages for accelerated applications within the root filesystem, ensure the following packages are enabled if they haven't been enabled by the KR260 BSP. To configure the rootfs, execute the following command from the ~/Kria_KR260/linux_os$ folder path:

petalinux-config -c rootfs

Once you're in the Menu Window, configure the following packages in the filesystem:

• Filesystem Package > base > i2c-tools, enable:
• i2c-tools
• i2c-tools-dev

Filesystem Packages --> console --> utils --> git --> [*] git
Filesystem Packages --> base --> dnf --> [*] dnf
Filesystem Packages --> base --> i2c-tools --> [*] i2c-tools
Filesystem Packages --> base --> i2c-tools --> [*] i2c-tools-dev
Filesystem Packages --> x11 --> base --> libdrm --> [*] libdrm
Filesystem Packages --> x11 --> base --> libdrm --> [*] libdrm-tests
Filesystem Packages --> x11 --> base --> libdrm --> [*] libdrm-kms
Filesystem Packages --> libs --> xrt --> [*] xrt
Filesystem Packages --> libs --> xrt --> [*] xrt-dev
Filesystem Packages --> libs --> zocl --> [*] zocl
Filesystem Packages --> libs --> libgcc --> [*] libgcc
Filesystem Packages --> libs --> libgcc --> [*] libgcc-dbg
Filesystem Packages --> libs --> libgcc --> [*] libgcc-dev
Filesystem Packages --> libs --> opencl-headers --> [*] opencl-headers
Filesystem Packages --> libs --> opencl-clhpp --> [*] opencl-clhpp-dev
Filesystem Packages --> misc --> packagegroup-core-buildessential --> [*] packagegroup-core-buildessential
Filesystem Packages --> misc --> packagegroup-core-buildessential --> [*] packagegroup-core-buildessential-dev
Filesystem Packages --> misc --> python3 --> [*] python3
Filesystem Packages --> misc --> python3 --> [*] <all-others>
Petaliunx Package Groups --> packagegroup-petalinux --> [*] packagegroup-petalinux
Petaliunx Package Groups --> packagegroup-petalinux-gstreamer --> [*] packagegroup-petalinux-gstreamer
Petaliunx Package Groups --> packagegroup-petalinux-opencv --> [*] packagegroup-petalinux-opencv
Petaliunx Package Groups --> packagegroup-petalinux-v4lutils --> [*] packagegroup-petalinux-v4lutils
Petaliunx Package Groups --> packagegroup-petalinux-x11 --> [*] packagegroup-petalinux-x11

Note: Take care of select all the tools that you need, it is just a recommendation, in Python3, select all the packages, I also recommend to search vim editor to work in petalinux, but also in this tutorial we will present a link with instructions to install nano editor.

Once the comfiguration is finished you spect to see the follow in the terminal.

Modifying Config and system_user.dtsi Files to Enable SPI Driver

To enable the SPI driver in Petalinux, you'll need to manually modify certain configuration files generated during the configuration process. Follow these steps:

• Locate the config file in ./linux_os/project-spec/config/ and add the following code block

#
# SPI Config
#
CONFIG_SPI_SUN4I=y
CONFIG_SPI_SUN6I=y
CONFIG_SPI=y
CONFIG_SPI_MASTER=y
CONFIG_EXPERIMENTAL=y
CONFIG_SPI_SPIDEV=y

• Navigate to ./linux_os/project-spec/meta-user/recipes-bsp/device-tree/files/ and modify the system-user.dtsi file as shown below:

/include/ "system-conf.dtsi"
/ {
};
&spi0{
    status = "okay";
    spidev@0x00 {
        status = "okay";
        compatible = "rohm,dh2228fv";
        spi-max-frequency = <25000000>;
        reg = <0>;
    };
};
&spi1{
    status = "okay";
    spi-slave;
    slave {
        compatible = "spi-slave-time";
    };
    spidev@0x01 {
        status = "okay";
        compatible = "rohm,dh2228fv";
        spi-max-frequency = <25000000>;
        reg = <0>;
    };
};

By making these modifications, the SPIDev driver for the SPI interface will be enabled, allowing you to work with SPI devices effectively.

Petalinux build and Image Packaging

Rebuild the project after adding the packages to the root filesystem (and kernel if you choose), remember to be located in ~/Kria_KR260/linux_os$ folder path:

petalinux-build

Build the Sysroot (SDK) for the project

After the project has been built, build the SDK for the project to get a sysroot to use with Vitis to compile any custom accelerated applications later on, remember to be located in ~/Kria_KR260/linux_os$ folder path:

petalinux-build --sdk

Package the image WIC for SD Card

Once the project has been built, package the boot binary for the custom image:

petalinux-package --boot --u-boot --force

Then package the WIC image for the SD card:

petalinux-package --wic --images-dir images/linux/ --bootfiles "ramdisk.cpio.gz.u-boot,boot.scr,Image,system.dtb,system-zynqmp-sck-kr-g-revB.dtb" --disk-name "sda"

Note that since the KR260 has the SD card connected to the Zynq FPGA via the USB controller on the KR260 carrier board, that's why we need to specify the disk name as sda.

Flash SD Card

You can manually load the wic onto the SD card from the command line, but I go the lazy route and use balenaEtcher:

As point you can put the SD Card in the KRIA KR260 board and boot this, the first time it will ask for an user and to change the password, the user was assigned by default as Petalinux and you can to use the password that you want, in the case of this example we use Petalinux as well.

The next steps are related with the use of the hardware platform generated by Vivado to create a Device Tree Overlay.

Generate the Device Tree Overlay from the PL Design

The device tree blob containing the overlay nodes needs to be compiled for the design, which is easily done by using the Xilinx Software Command Line Tools (XSCT), for this, locate in the hardware project folder (./Kria_KR260) and run the follow commands:

source /tools/Xilinx/Vitis/2022.1/settings64.sh
xsct

In the xsct environment (xsct%), open the exported XSA from Vivado and use the createdts command to create the device tree source files for the PL design:

xsct% hsi::open_hw_design kria_base.xsa
xsct% createdts -hw kria_base.xsa -zocl -platform-name kria_kr260 -git-branch xlnx_rel_v2022.2 -overlay -compile -out ./dtg_kr260_v0
xsct% exit

After exiting XSCT, use the standard Linux device tree compiler (dtc) to compile the source files into the needed device tree blob:

dtc -@ -O dtb -o ./dtg_kr260_v0/dtg_kr260_v0/kria_kr260/psu_cortexa53_0/device_tree_domain/bsp/pl.dtbo ./dtg_kr260_v0/dtg_kr260_v0/kria_kr260/psu_cortexa53_0/device_tree_domain/bsp/pl.dtsi

Transfer the files of the PL Design to the KR260

I like to create a folder to copy all of the necessary design files to that need to be uploaded to the KR260 for an accelerated application or PL design like this (note: the Kria_KR260 directory is the top level directory of the Vivado project):

mkdir -p overlay_file_transfer
cd ./overlay_file_transfer/

It’s here I’ll create the description file, `shell.json`, for the design:

~/Kria_KR260/overlay_file_transfer$ nano shell.json

And copy paste the following to shell.json:

{
"shell_type": "XRT_FLAT",
"num_slots": "1"
}

Then copy the generated device tree blob and .bin file into the folder:

cp ../dtg_kr260_v0/dtg_kr260_v0/kria_kr260/psu_cortexa53_0/device_tree_domain/bsp/pl.dtbo ./
cp ../Kria_KR260.runs/impl_1/kria_bd_wrapper.bin ./

Rename the device tree blob and .bin file to the same thing (the only difference should be their respect file extensions (also change the .bin extension to .bit.bin):

mv kria_bd_wrapper.bin kr260_overlay.bit.bin
mv pl.dtbo kr260_overlay.dtbo

Then, with the KR260 booted up and running with the new SD card image generated in the previous steps and connected to the local network, transfer the new PL design files to it:

scp kr260_overlay.dtbo kr260_overlay.bit.bin shell.json petalinux@xilinx-kr260-starterkit-20222:/home/petalinux

Run PL Design on KR260

This part of the tutorial is execute in the KRIA KR260, you need to connect to it through SSH using the follow command and your access password:

ssh petalinux@xilinx-kr260-starterkit-20222

In the KRIA KR260 terminal, verify if the files are presented there, then create a directory in the /lib/firmware/xilinx directory with the same name as was given to the device tree blob and .bin file and copy them into it:

sudo mkdir /lib/firmware/xilinx/kr260_overlay
sudo mv kr260_overlay.dtbo kr260_overlay.bit.bin shell.json /lib/firmware/xilinx/kr260_overlay

At this point, the PL design will show up just like an accelerated application would using the xmutil commands:

sudo xmutil listapps

Unload the default application then load the PL design which flashes the PL design’s bitstream into the PL and loads its device tree overlay:

sudo xmutil unloadapp
sudo xmutil loadapp kr260_overlay

When the device tree overlay loads, the terminal will print out as such indicating the new device tree nodes for each of the AXI GPIO IP blocks are now present in the system.

Utilities installation

To install nano package, follow the next tutorial.