One of the coolest FPGA Technologies: AMDs DFX

Introduction

In typical embedded systems the hardware is fixed and function changes during runtime are implemented in software. This means that the hardware must support all features at all times. With FPGAs, only the needed hardware (logic) is implemented, unlike standard SoCs.

Dynamic Function eXchange (DFX) takes this flexibility even further with hardware time slicing. This means that part of the FPGA logic can, for example, function as an Ethernet controller and then, microseconds later, switch to a MIPI core.

This technology has several advantages:

• Smaller FPGA requirements
• Simplified configuration management
• Reduced power consumption
• Easier design updates, as only partial bitstreams need to be updated

In the following sections, we will guide you through the DFX flow. The new Avnet Artix UltraScale+ Board (AU15P) was chosen as the hardware platform for this demonstration.

The Artix UltraScale+ device from AMD is a classic FPGA based on UltraScale+ technology. More details can be found here.

Reference Design Explanation

The reference design we provide focuses on simplicity to make the DFX flow easier to understand.In this demo, a soft processor boots a Linux system from SPI NOR flash. We chose Linux because it includes a modern flash filesystem (UBI + UBIFS) and a TCP/IP network stack at no additional cost.The Linux application loads four partial bitstreams from the filesystem into memory (two for each DFX partition). After that, the DFX controller is set up and triggered every 5 seconds. Each partial bitstream contains a different LED controller, resulting in different LED blinking sequences. The time required for reconfiguration is measured and displayed on the UART console.

If you choose to rebuild the design, the build and boot log files are attached for your reference.

The Vivado Design

The following images show the block design that resides in the static part. It includes a MicroBlaze soft processor configured with an MMU and a cache controller to run a Linux system. Additionally, standard peripherals like Ethernet, the DDR4 memory controller, and I2C are part of the static design.

An important part of the static part is the DFX controller (part of the peripherals block):

The DFX controller can be programmed via its AXI-Lite register interface. The application writes the memory address to the corresponding DFX controller register and then starts the controller. The bitstream data is fetched using the DFX controller's DMA logic and pushed through the ICAP (Integrated Configuration Access Port) to the configuration memory.

In this reference, we define two partitions: u_count and u_shift. In the VHDL code, these partitions are implemented as standard components. When the DFX flow is enabled, a tab called "Partition Definition" is displayed:

Here, the different HDL modules are associated with the partitions. In our example, we have two modules per partition, but you can define as many as needed.

After defining the HDL modules, you need to create the area constraints for the partitions:

After synthesizing the design, you will receive feedback on whether the areas are large enough to accommodate the modules.

We have an Integrated Logic Analyzer (ILA) within our partition, which explains the high LUT utilization in the u_count partition. This is why it is larger than the u_shift partition.

The next step is to implement the design.

After implementation, the full bitstream with the default settings for the partitions is generated, along with the four partial bitstreams.

Rebuilding the Vivado Design with the provided project sources

First, set the TRD_HOME variable from a console in the project root:

export TRD_HOME=$PWD

Next, source the AMD Tools:

source /opt/Xilinx/Vivado/2023.2/settings64.sh
source /opt/Xilinx/Vitis/2023.2/settings64.sh
source /opt/Xilinx/petalinux/2023.2/settings.sh

Now, build the entire design and the partial bitstreams:

cd $TRD_HOME/vivado
make

This step builds all the necessary IP cores, the Vivado project, and the bitstreams.

After the build is complete, you will find the generated Vivado project in:

$TRD_HOME/vivado/build/prj

You can open the xpr file using Vivado in GUI mode to investigate the design.

All generated bitstreams are located in:$TRD_HOME/vivado/build/bitstreams

At this stage, you can test the DFX bitstreams using the Vivado Hardware Manager. Download the top.bit file and try down.bit, up.bit, left.bit and right.bit.

You should see different LED patterns on LED D35 and D36.

Please note that at this stage, the top.bit bitstream does not contain a bootloader for the MicroBlaze.

Building the Custom Bootloader "bobbyb" Using Vitis

The default bootloader for embedded systems, U-Boot - is too large for the local memory of the MicroBlaze. Therefore, a custom first-stage bootloader is used. We achieve this by using xsct, a tool that allows us to create a Vitis workspace from the command line and compile the FSBL sources.

cd $TRD_HOME/vitis
xsct

In the Xilinx Software Command Tool shell, type the following commands:

xsct% file mkdir workspace;cd workspace
xsct% source ../tcl/build.tcl
xsct% gen_app release
xsct% exit

The freshly built FSBL is located in:

$TRD_HOME/vitis/workspace/bobbyb/Release/bobbyb.elf

Building the Linux System using AMD's PetaLinux

We use PetaLinux together with a custom Avnet-Silica Yocto Layer to build the Linux images.

First, we create the PetaLinux project with the AMD defaults for the MicroBlaze:

mkdir -p $TRD_HOME/plnx/work
cd $TRD_HOME/plnx/work
petalinux-create --type project --template microblaze --name aup15

Next, we add our custom Yocto layer and overwrite the defaults:

cd $TRD_HOME/plnx/work/aup15
ln -s $TRD_HOME/plnx/configs/meta-avs project-spec/
ln -sf $TRD_HOME/plnx/configs/config project-spec/configs/config
ln -sf $TRD_HOME/plnx/configs/rootfs_config project-spec/configs/rootfs_config

We then import the Xilinx Support Archive into our project:

petalinux-config --get-hw-description $TRD_HOME/vivado/build/prj/  --silentconfig

Next, we copy the partial bitstreams generated by Vivado intoto our custom root filesystem recipe. The purpose is to include these bit files in the final root filesystem so they can be loaded by the Linux application:

cp $TRD_HOME/vivado/build/bitstreams/*.bin project-spec/meta-avs/recipes-apps/dfx-demo/files/bitstreams/

This recipe places the.bin files in /usr/share/avs/bitstreams of the final Linux root filesystem.

Finally, we start the PetaLinux build:

petalinux-build

After the PetaLinux build is complete, all images are located here:

$TRD_HOME/plnx/work/aup15/images/linux

Vivado Part 2: Updating the Bitstream with the MicroBlaze FSBL and Generating MCS Flash Images from the PetaLinux Images

cd $TRD_HOME/vivado/build/prj
vivado -mode tcl

The Vivado console is now active.

Vivado% source ../../tcl/procedures.tcl
Vivado% gen_flash_image_fpga ../../../vitis/workspace/bobbyb/Release/bobbyb.elf

We have now updated the main bitfile with the fsbl and created a flash image (MCS file).

Next, we create all the images for the other flash partitions:

Vivado% source ../../tcl/procedures.tcl
Vivado% gen_flash_image_uboot
Vivado% gen_flash_image_kernel
Vivado% gen_flash_image_ubi

All images are located in this build folder:

$TRD_HOME/vivado/build/flash_images

Vivado Part 3: Downloading Flash Images via JTAG

I personally like the command line over the GUI. Therefore, I developed some Tcl procedures that allow me to program the target SPI NOR flash using the Vivado console. For this step, connect the JTAG UART USB cable to the board and power it on.

Vivado% source ../../tcl/procedures.tcl
Vivado% jtag::flash ../flash_images/fpga.mcs
Vivado% jtag::flash ../flash_images/u-boot.bin.mcs
Vivado% jtag::flash ../flash_images/image.ub.mcs
Vivado% jtag::flash ../flash_images/rootfs.ubi.mcs

Demo Time

After flashing the images to the SPI NOR flash, connect a UART terminal to the second ttyUSB port:

tio /dev/ttyUSB1 -b 115200

Press the PROG B button and observer the complete boot flow.

Start the dfx demo in the UART terminal and observe the outputs:

root@avs-au15p-trd:~# dfx-demo

Demo log:

copied /usr/share/avs/bitstreams/left.bin to buffer @ 0xa7d00000 (290552 bytes)
copied /usr/share/avs/bitstreams/right.bin to buffer @ 0xa7d46ef8 (290552 bytes)
copied /usr/share/avs/bitstreams/up.bin to buffer @ 0xa7d8ddf0 (386864 bytes)
copied /usr/share/avs/bitstreams/down.bin to buffer @ 0xa7dec520 (386864 bytes)
Baseaddr: 0x48602000
Sending shutdown commands to dfx controller...
writing addresses and sizes of partial bitstreams to dfx controller...

pushing partial bitstream #0: /usr/share/avs/bitstreams/left.bin
Configuration time 600 us

pushing partial bitstream #1: /usr/share/avs/bitstreams/right.bin
Configuration time 606 us

pushing partial bitstream #2: /usr/share/avs/bitstreams/up.bin
Configuration time 719 us

pushing partial bitstream #3: /usr/share/avs/bitstreams/down.bin
Configuration time 727 us

pushing partial bitstream #0: /usr/share/avs/bitstreams/left.bin
Configuration time 606 us

Conclusion

I hope you enjoyed learning about the exciting DFX technology. If you have any questions, please don’t hesitate to contact me. Depending on the feedback and how useful the project is to others, I will continue maintaining and improving it.

Check the improvement list and let me know what features you would like to see next.

Improvements

• Move the design to the latest AMD Tools
• Replace the MicroBlaze with the RISC-V version
• Add something more eye catching than LEDs, e.g. use the HDMI RX and HDMI TX port to manipulate a Video stream. Different optical filers (e.g. Sobel, Canny etc.) are applied during runtime.
• Use block design containers in addition to VHDL components for the partitions

References

• AMD DFX User Guide UG947

Acknowledgements

• AMD: Thanks for this great Technology!
• Avnet Embedded: Thanks for sponsoring the Early Access Artix UltraScale+ Board

Version History

• 2024/08/27 - Initial Version