MicroZed Chronicles: PetaLinux Image Processing

Over the course of this series we have looked at image processing several times, mostly using bare-metal or PYNQ. However, for many applications and indeed for Vitis acceleration applications, we need to create a PetaLinux-based image processing chain.

This is exactly what we are going to do over the next couple of blogs. Of course, we will be starting in Vivado and targeting the Ultra96-V2 board. We will use the Ultra96-V2 board, MIPI interface board, the JTAG/UART board, and the Digilent Pcam 5C camera.

Creating the hardware design is remarkably similar to what we have previously implemented; however, there are some differences that we need to take into account.

When we are developing a PetaLinux image processing pipeline, we will be making use of the Xilinx V4L2 driver. This enables us to configure and use the image processing pipeline from within PetaLinux. We will also do a considerable amount of device tree work as we examined last week.

We will be starting with a standard MIPI CSI-2 receiver subsystem. I am doing this development in 2019.2 but from 2020.1 on, these MIPI cores are bundled free with Vivado.

I am going to make the image processing chain quite simple for this application. As such, I will only include the Sensor Demosaic IP core to convert the RAW 10-bit output to RGB color space. Once this has been completed, we will move the image to the PS DDR, unlike in previous designs when I have used VDMA. In this example, we will use the frame_write and frame_read buffers to perform this transfer to and from the PS DDR.

We also need to connect our resets differently than we have previously in other image processing examples. In previous examples, we have used the PL_RESET from the PS. In this case, we will be using GPIO to provide resets for each of the processing blocks in the image processing chain. This provides the ability of V4L2 to be able to reset elements of the image processing chain if required.

To do this, I enabled all the EMIO GPIO within the Zynq processing block and selected appropriate bits from the 95-bit EMIO BUS for the reset of the processing blocks. To do this selection, I used the slice IP core within the block diagram. I allocated the following EMIO GPIO to the processing blocks:

• EMIO GPIO 7 – Connected to the Write Frame Buffer IP
• EMIO GPIO 8 – Connected to the Demosaic IP
• EMIO GPIO 9 – Connected to the Read Frame Buffer IP

The MIPI CSI-2 IP core is reset as per usual from a processor reset block to ensure the image processing pipeline does not get corrupted should a reset be asserted. I connected any AXI subset streaming IP cores to the appropriate reset as well.

Once these blocks are connected up in the design, we can build the project and export the XSA to create our PetaLinux project. Obviously, this is where the real fun starts!

See My FPGA / SoC Projects: Adam Taylor on Hackster.io

Get the Code: ATaylorCEngFIET (Adam Taylor)