Ultra96v2 Linux-Based Platform in Xilinx Vitis 2020.1

Note that the goal of this project is creating a hardware platform for Ultra96v2 that can be used for accelerating functions under the Xilinx Vitis toolset.

If you are interested to know how to use this platform to accelerate different compute-intensive tasks such as Support Vector Machine (SVM) on an FPGA-based embedded system, please refer to here.

If you would like to have a fully functional platform that supports all the sensors and modules on the board, you should consult the Avent website and resources.

Creating the Ulra96v2 platform in the Xilinx Vitis 2020.1 has five steps:

• XSA design – Generating a Vivado project containing the underlying hardware
• Linux OS – Generating a PetaLinux project to configure Linux
• TestLinux – Preparing the SDCard and Test the generated Linux
• CreatePlatform – Using Xilinx Vitis to generate the Platform
• Test– Create a simple application to test the generated platform

In the sequel, I am trying to briefly explain each step. I have installed Xilinx Vitis and Petalinux 2020.1 under the Ubuntu 18.04 OS in a VirtualBox environment. Note that, the Xilinx Vitis software includes Vivado, so you do not need to install that separately.

To make Vivado detects the board, copy the Avnet ultra96v2 board definition files (at here) to the <Vivado installation folder>/data/boards/board_files.

1- XSA design

1- create a directory called ultra96v2-vitis-pkg

mkdir ultra96v2-vitis-pkg
cd ultra96v2-vitis-pkg/

2- create a director called vivado

mkdir vivado
cd vivado

3- Run Vivado and create a project called ultra96v2-xsa in the ultra96v2-vitis-pkg/vivado folder. Figures 1-6 show the flow of the project creation.

4- In the Vivado project, create a block design with the name of ultra96v2_design.

5- Add a Zynq UltraScale+ MPSoC IP into the Diagram view

6- Click on “Run Block Automation” and make sure “Apply Board Reset” is selected.

7- Double click on the Zynq IP and configure that as Figure 10

8- Add a Clocking Wizard IP to the design.

9- Double click on the added IP and configure that as Figure 12. Add four output clocks with different output frequencies, and make sure that the reset type is “Active Low”.

10- Add four Processor System Reset IPs corresponding to the four output clocks.

11- Add connections as Figure 14.

12- Add an AXI interrupt controller IP

13- Select the AXI HPM0 LPD checkbox, keep the AXI HPM0 LPD Data width settings as default 32.

14- Connect maxihpm0_lpd_aclk to the 100 MHz clock of Clocking Wizard IP.

15- Click Run Connection Automation, Expand output interface Interrupt of axi_intc_0 to show the port irq, connect this irq portto zynq_ultra_ps_e_0.pl_ps_irq0Setup *PFM_IRQ property by typing following command in Vivado console:

set_property PFM.IRQ {intr {id 0 range 32}} [get_bd_cells /axi_intc_0]

16- Now, we should declare the platform (PFM) interface and properties. For this purpose, select “Platform Interfaces” option from the “Window” menu in the Vivado IDE. (Window–>Platform Interfaces).

17- Click on “Enable platform interfaces” if this the first time you select the Platform Interfaces option in the project.

18- Right-click on each port in the following list and select Enable

M_AXI_HPM0_FPD
M_AXI_HPM1_FPD
M_AXI_HPM0_LPD
S_AXI_HPC0_FPD
S_AXI_HPC1_FPD
S_AXI_HP0_FPD
S_AXI_HP1_FPD
S_AXI_HP2_FPD
S_AXI_HP3_FPD

19- Select S_AXI_HPC0_FPD port and write HPC0 in the sptag under Option in Platform Interface Properties panel. For other ports do similar setting

S_AXI_HPC0_FPD ---> HPC0
S_AXI_HPC1_FPD ---> HPC1
S_AXI_HP0_FPD ---> HP0
S_AXI_HP1_FPD ---> HP1
S_AXI_HP2_FPD ---> HP2
S_AXI_HP3_FPD ---> HP3

Note, to change the sptag filed you must press the Enter key on your keyboard. So, don't forget to press Enter each time.

20- Enable the following clock interfaces

clk_out1
clk_out2
clk_out3
clk_out4

21- Select the enabled clk_out1 and in the Options window change the id to 0 and select the is_default. Note, to change the id filed you must press the Enter key on your keyboard. Change ids of clk_out2, clk_out3, and clk_out4 to 1, 2, and 3, respectively.

22-Enable the M01_AXI ~ M08_AXI ports of ps8_0_axi_periph IP, and set these ports with the same sptag name to HPM0_LPD and memport type to M_AXI_GP

23- Go to the “TCL Console” view and run these commands

set_property platform.design_intent.embedded true [current_project]
set_property platform.design_intent.server_managed false [current_project]
set_property platform.design_intent.external_host false [current_project]
set_property platform.design_intent.datacenter false [current_project]
set_property platform.default_output_type "sd_card" [current_project]

24- Press the right-click in Diagram view and in the popup menu select Validate Design.

25- Press the right-click on the “ultra96v2_design (ultra96v2_design.bd)” option in the source view panel. And from the popup menu select “Generate Output Products…”.

26- Select the Generate button and wait for the process to finish.

27- Again, press the right-click on the “ultra96v2_design (ultra96v2_design.bd)” option in the source view panel. And from the popup menu select “Create HDL Wrapper… ” and select the “Let Viviado manage wrapper and auto-update” option in the dialog box.

28- Then under the PROGRAM AND DEBUG option in the left-hand side panel, select “Generate Bitstream” and wait until the end of the process.

29- After generating the bitstream, go to the TCL Console view and make sure you are in the right folder. you can use pwd command to check the folder. Go to “ultra96v2-vitis-pkg/vivado” that you have created. and run the following TCL command to generate the XSA file

write_hw_platform -include_bit ultra96v2.xsa

30- You can validate the generated XSA file by running the following TCL command.

validate_hw_platform ./ultra96v2.xsa

2- Linux OS

1- Go to the “ultra96v2-vitis-pkg” folder generated in the first step (i.e., 1-XSA design) and run the following commands

mkdir linux_files
mkdir  linux_files/boot

2- Run the setting.sh file located at the PetaLinux installation folder by running this command

source /tools/Xilinx/petaliux/2020.1/settings.sh

3- In the “ultra96v2-vitis-pkg” folder create a PetaLinux project

petalinux-create -t project --template zynqMP -n ultra96v2-petalinux

4- Go to the generated project folder and configure the project

cd ultra96v2-petalinux
petalinux-config --get-hw-description=../vivado

5- In the “misc/config System Configuration” window, go to “Subsystem AUTO Hardware Settings–>Serial Setting” and change all the “psu_uart_0” to “psu_uart_1” and back to the main menu.

6- select DTG Settings->MACHINE_NAME, modify it to avnet-ultra96-rev1

7- Under the “Image Packaging Configuration” option, change the “Root filesystem type” to “EXT4 (SD/eMMC/SATA/USB)”

8- Save and exit the configuration window.

9- The next step is kernel configuration using the following command

petalinux-config -c kernel

10- In the “Linux/arm64 4.19.0 Kernel Configuration” window, go to “Library routines->Size in Mega bytes” and change 256 to 1024

11- Turn off “CPU Power Management –> CPU Idle –> CPU idle PM support” and “CPU Power Management –> CPU Frequency scaling –> CPU Frequency scaling”

12- Save and exit the kernel configuration window

13- Open the “./project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi” file and modify that as follows.

/include/ "system-conf.dtsi"
/ {
};
&amba {
zyxclmm_drm {
compatible = "xlnx,zocl";
status = "okay";
interrupt-parent = <&axi_intc_0>;
interrupts = <0  4>, <1  4>, <2  4>, <3  4>,
<4  4>, <5  4>, <6  4>, <7  4>,
<8  4>, <9  4>, <10 4>, <11 4>,
<12 4>, <13 4>, <14 4>, <15 4>,
<16 4>, <17 4>, <18 4>, <19 4>,
<20 4>, <21 4>, <22 4>, <23 4>,
<24 4>, <25 4>, <26 4>, <27 4>,
<28 4>, <29 4>, <30 4>, <31 4>;
};
};
&axi_intc_0 {
xlnx,kind-of-intr = <0x0>;
xlnx,num-intr-inputs = <0x20>;
interrupt-parent = <&gic>;
interrupts = <0 89 4>;
};

14- Apply the patch explained here, if it didn’t work (as in my case), simply open and edit the “./components/yocto/workspace/sources/linux-xlnx/drivers/tty/serial/xilinx_uartps.c” file. Comment out the following code at Line 1488 and then save the file.

cdns_uart_console.index = id; --> //cdns_uart_console.index = id;

15- Edit the file “./project-spec/meta-user/conf/user-rootfsconfig” and add the following lines

CONFIG_xrt
CONFIG_xrt-dev
CONFIG_zocl
CONFIG_opencl-clhpp-dev
CONFIG_opencl-headers-dev
CONFIG_packagegroup-petalinux-opencv

16- Configure the Linux rootfs by running the following command

petalinux-config -c rootfs

17- Go to the “user packages” menu and select the following packages

opencl-hpp-dev
opencl-headers-dev
packagegroup-petalinux-opencv
xrt
xrt-dev
zocl

18- Then build the peralinux project by running this command.

petalinux-build

19- Then go to images/linux folder

cd images/linux

and run this command

petalinux-build --sdk

20- Then run the./sdk.sh command and enter the full path of the linux_files folder that you created in the first phase.

21- Copy the following files from the image/linux folder into the linux_files/boot folder that you created in the first phase.

image.ub
zynqmp_fsbl.elf
pmufw.elf
bl31.elf
u-boot.elf
rootfs.cpio

22- In the linux_files/boot folder create the linux.bif file containing the following lines

/* linux */
the_ROM_image:
{
[bootloader] /home/numvar/workspace/ultra96v2/ultra96v2-vitis-pkg/linux_files/boot/zynqmp_fsbl.elf
[pmufw_image] /home/numvar/workspace/ultra96v2/ultra96v2-vitis-pkg/linux_files/boot/pmufw.elf
[destination_device=pl] <bitstream>
[destination_cpu=a53-0, exception_level=el-3, trustzone] /home/numvar/workspace/ultra96v2/ultra96v2-vitis-pkg/linux_files/boot/bl31.elf
[destination_cpu=a53-0, exception_level=el-2] /home/numvar/workspace/ultra96v2/ultra96v2-vitis-pkg/linux_files/boot/u-boot.elf
}

3- Prepare SDCARD and Test Linux

1- Insert the sdcard in the host computer USB port (or a VirtualBox with Ubuntu).

2- Run GParted program to perform the sdcard partitioning. Select the sdcard (e.g., /dev/sdc)

3- Create a new partition with

size=2048MB
filesystem:fat32
Label: BOOT
Create as: Primary Partition

4- Assign the rest of the space on the sdcard to a new partition with

size = <the rest of spapce>
filesystem:ext4
Label: rootfs
Create as: Primary Partition

5- Apply all operations and then the sdcard with two partitions is ready.

6- cd <petalinux project folder>/images/linux/

7- Run this command to generate to files: BOOT.BIN and boot.scr.

petalinux-package --boot --format BIN --fsbl zynqmp_fsbl.elf --u-boot u-boot.elf --pmufw pmufw.elf --fpga system.bit --force

3- copy three files (BOOT.BIN, boot.scr, and image.ub) into the sdcard BOOT parttion

cp BOOT.BIN boot.scr image.ub /media/numvar/BOOT/

4- write the rootfs files into the sdcard rootfs partition

sudo tar -zxvf rootfs.tar.gz -C /media/numvar/rootfs/

5- Run the command sync and wait to finish.

6- Use a program that provides a serial terminal such as putty or “Tera Term” and connect to the board serial USB connection.

7- Then, unmount the SDCard and insert that to the Ultra96v2 board and turn on the board. You should see the login prompt in the serial terminal.

4 – Create Platform

1- Create a folder named pfm in the ultra96v2-vitis-pkg folder that you created in the first step.

cd pfm

To create the ultra96v2 platform for Vitis. First, run the Vitis IDE and choose the pfm folder (that you created in the first step) as the workspace.

2- Create a new platform project by selecting “File–>New–>Platform Project…”. Choose “ultra96v2” as the project name and accept the default folder.

3- In the next window select the XSA option and press Next

4- Click on Browse… button and locate the ultra96v2.xsa file generated in the first step by Vivado, and then select the Linux as Operating System and click Finish button.

5- Clock the Welcome window if it is open. Select “Linux on psu_cortexa53” and configure the Linux domain as the following figure.

6- Press the right-click on the project name on the left panel and select Build Project.

5- Test the Platform

1- In the Vitis IDE, select “File–>New–>Application Project…” to create a new application.

2- Select vector_add as the Project name

4- Press the Next button in the next window.

5- Select the Vector Addition as the template and press Finish button.

6- Select Hardware as the active build configuration from the menu on the right.

7- Build the project by pressing the right-click on the project name and selecting the Build project option.

8- Copy the content of the sd_card folder generated by the application project under the into an SD-CARD and boot the Ultra96v2 board. Notice that you need to copy at least five files: binary_container_1.xclbin, BOOT.BIN, image.ub, vector_add and boot.scr. If you cannot find the boot.csr here, use the one that has been generated earlier during testing our Linux OS.

9- Using putty open a serial terminal with the following configuration

10- Boot the Ultra96v2 board with the SDCard and login to the board by using root as both username and password. Then go to /run/mnt/sd-mmcblk0p1 folder

11- Run the following commands to define the Xilinx XRT library

export XILINX_XRT=/usr

12- Run the application by running this command

./vector_add.exe binary_container_1.xclbin