RF Modulation Recognition with GNU Radio

What is this Project about?

This is a fresh take on an existing idea from the AMD-Xilinx Vitis AI tutorial:

RF Modulation Recognition

But, we will develop an Application on the top of the GNU Radio, a platform for developing a Software Defined Radio (SDR).

The application will listen for some random RF radio signals on a tuned frequency (on a 2m radio amateur band, for example) and predict a modulation of the received signal in real time by using a Convolutional Neural Network (CNN), implemented in the Zynq UltraScale+ FPGA.

We use an RTL-SDR v3 USB dongle as an SDR receiver hardware and an FPGA development board ZCU104 (or similar), made by AMD-Xilinx company.

We develop an Out-Of-Tree module (OOT) for GNU Radio that can access a Deep learning Neural Network Unit (DPU) implemented in the programming logic (FPGA). The GNU Radio software application will run on the CPU of the Zynq device under Python and PetaLinux.

The OOT module (gr-fpga-ai) will use a CNN model that is developed, trained and compiled for DPU on the FPGA by using Vitis-AI tool from AMD-Xilinx according to previously mentioned tutorial.

A freely available dataset 2018-RF will be used for training the CNN model. But, due to very large size of the dataset (> 20 GB), we reduce the dataset to be smaller and we train the model on the freely available Colab service provided by Google. So, we don’t need so powerful and very expensive graphic card just for this project for model training.

Vitis-AI tool will be used for quantization, calibration and compiling the model only.The result of this project is shown in this short demo:

The prepared SD card images for ZCU104 and AXU2CGB boards can also be downloaded from our GitHub repository.

Quick Test

Un-tar the selected SD card image from GitHub repository and write it on an SD card (at least 8 GB) with an Etcher or similar program for writing SD cards.

To run the AI inference test on DPU without any additional USB dongles needed (receiver, sound blaster, ect...), type the following command to the Serial Terminal:

cd /media/sd-mmcblk0p1/test-dpu
./run_test_performance.sh
./run_test_accuracy.sh

To test the RTL-SDR USB receiver (should be plugged in to USB), run the FM-Radio application by typing the following command to the SSH -X terminal:

cd /media/sd-mmcblk0p1/test-dpu
./run_gnuradio.sh

To run the AI Inference test on Real Signals from an antenna (RTL-SDR USB receiver with an external antenna should be plugged in to USB), type the following command to the SSH -X terminal:

cd /media/sd-mmcblk0p1/test-dpu
./run_rf_classification	.sh

Why this Tutorial?

The radio spectrum is very large and even more signals with very different kind of modulations appears in it, recently. Even on the radio amateur bands there are a lot of new digital modulations developed, especially for radio amateur satellites. It is difficult to guess for a signal in real time, what kind of modulation is used.

So, we developed a small platform on the top of GNU Radio that can also be used for future development of AI Inference models for RF field.

How to Build this Project?

This is a Step-by-Step tutorial explained in four main parts:

• Part-1 - Create the Vivado Hardware design
• Part-2 - Build the PetaLinux and GNU Radio
• Part-3 - Create the Vitis Platform with DPU
• Part-4 - Build the AI model in Colab and Vitis-AI

At the end we run some test script to check the performance and accuracy of the model on DPU on the real hardware and finally run the application.

IMPORTANTNOTE

These tutorials are based on the AXU2CGB development board, made by Alinx, because this was the only board we managed to get in time, when writing this project.

Our original Four Tutorials for AXU2CGB boards are written in such a way, that everything can be reproduced for ZCU104 or similar Zynq UltraScale+ board with slightly modifications.

In the following sections the whole process from Part-1 to Part-4 is repeated with some modifications required for the ZCU104 board to build the final SD card image, which is available in the Attachment section.

But the real life demo was tested only on our available AXU2CGB board.

Let’s Start…

The final result of this project is the SD Card image that can be run on the ZCU104 board. It can also be downloaded from the Attachment section.

The SD card must be at least 8 GB. The image file can be written to SD card with the Linux dd command or any other program for writing SD card images.

Prerequisite

Before starting, you should already have installed the following software tools from AMD-Xilinx:

• Vitis 2021.2
• Vivado 2021.2
• Petalinux 2021.2

PART-1

Create the Vivado Hardware design

We start to prepare the folder hierarchy. For this purpose, perform the following commands:

$ mkdir zcu104
$ cd zcu104
$ mkdir hardware
$ mkdir software
$ mkdir platform

$ cd hardware

1- Run the Vivado and create a project in the hardware folder with zcu104_2021_2-vivado name.

$ source <Vitis_Install_Directory>/settings64.sh
$ vivado &

2- On the Project Type page, choose the RTL project and select the following two options:

• Do not specify sources at this time
• Project is an extensible Vitis platform

3- In the Boards tab choose the board named:

• Zynq UltraScale+ ZCU104 Evaluation Board (or similar board you have)

Click Next and Finish.

4- Create Block Design and a Zynq UltraScale+ MPSoC IP to the design:

• Click the Run Block Automation link
• Enable Zynq_ultra_ps_e_0 and Apply Board Preset, then OK.

5- Add the Clocking Wizard to the block design and double-click the clk_wiz_0 IP block to open the Re-Customize IP dialog box

Click the Output Clocks tab. Enable clk_out1 through clk_out3 in the Output Clock column. Set the Requested Output Freq as follows:

• clk_out1 to 100 MHz
• clk_out2 to 200 MHz
• clk_out3 to 400 MHz

At the bottom of the dialog box set the Reset Type to Active Low.

# Equivalent TCL command for adding the Clocking Wizard

create_bd_cell -type ip -vlnv xilinx.com:ip:clk_wiz:6.0 clk_wiz_0

set_property -dict [list CONFIG.CLKOUT2_USED {true} CONFIG.CLKOUT3_USED {true} CONFIG.CLKOUT2_REQUESTED_OUT_FREQ {200.000} CONFIG.CLKOUT3_REQUESTED_OUT_FREQ {400.000} CONFIG.RESET_TYPE {ACTIVE_LOW} CONFIG.MMCM_CLKOUT1_DIVIDE {6} CONFIG.MMCM_CLKOUT2_DIVIDE {3} CONFIG.NUM_OUT_CLKS {3} CONFIG.RESET_PORT {resetn} CONFIG.CLKOUT2_JITTER {102.086} CONFIG.CLKOUT2_PHASE_ERROR {87.180} CONFIG.CLKOUT3_JITTER {90.074} CONFIG.CLKOUT3_PHASE_ERROR {87.180}] [get_bd_cells clk_wiz_0]

6- Add three “Processor System Reset” IPs to the design and connect the IPs together, as shown in figure below

# Adding 3 Processor System Reset IPs
create_bd_cell -type ip -vlnv xilinx.com:ip:proc_sys_reset:5.0 proc_sys_reset_0
create_bd_cell -type ip -vlnv xilinx.com:ip:proc_sys_reset:5.0 proc_sys_reset_1
create_bd_cell -type ip -vlnv xilinx.com:ip:proc_sys_reset:5.0 proc_sys_reset_2

connect_bd_net [get_bd_pins zynq_ultra_ps_e_0/pl_clk0] [get_bd_pins clk_wiz_0/clk_in1]

connect_bd_net [get_bd_pins zynq_ultra_ps_e_0/pl_resetn0] [get_bd_pins clk_wiz_0/resetn]
connect_bd_net [get_bd_pins zynq_ultra_ps_e_0/pl_resetn0] [get_bd_pins proc_sys_reset_1/ext_reset_in]
connect_bd_net [get_bd_pins zynq_ultra_ps_e_0/pl_resetn0] [get_bd_pins proc_sys_reset_0/ext_reset_in]
connect_bd_net [get_bd_pins zynq_ultra_ps_e_0/pl_resetn0] [get_bd_pins proc_sys_reset_2/ext_reset_in]

connect_bd_net [get_bd_pins clk_wiz_0/clk_out1] [get_bd_pins proc_sys_reset_0/slowest_sync_clk]
connect_bd_net [get_bd_pins clk_wiz_0/clk_out2] [get_bd_pins proc_sys_reset_1/slowest_sync_clk]
connect_bd_net [get_bd_pins clk_wiz_0/clk_out3] [get_bd_pins proc_sys_reset_2/slowest_sync_clk]

connect_bd_net [get_bd_pins clk_wiz_0/locked] [get_bd_pins proc_sys_reset_1/dcm_locked]
connect_bd_net [get_bd_pins clk_wiz_0/locked] [get_bd_pins proc_sys_reset_0/dcm_locked]
connect_bd_net [get_bd_pins clk_wiz_0/locked] [get_bd_pins proc_sys_reset_2/dcm_locked]

regenerate_bd_layout

7- Enable all three clocks for the platform. Use menu Window -> Platform Setup. Set clock-2 as default.

8- Add Interrupt support.

• double-click the Zynq UltraScale+ MPSoC block
• Select PS-PL Configuration > PS-PL interfaces > Master interface
• Enable the AXI HPM0 LPD option
• Disable AXI HPM0 FPD and AXI HPM1 FPD
• Enable General > Interrupts > PL to PS > IRQ0[0-7] Interrupts

9- Add the AXI Interrupt ControllerIP and configure it as a Single Interrupt Output Connection. Click OK

Then click on the “Run Connection Automation” link.

Select clk_out2 (200 MHz) as a source for master and slave interface and press OK.

10- Connect the irq output pin to the Zynq interrupt input.

11- Enable AXI Interfaces for the Platform

• Enable first eight interfaces under zynq_ultra_ps_e_0 and set SP Tag

• Enable M01_AXI to M07_AXI interfaces under ps8_0_axi_periph

Equivalent TCL commands:

# Enable AXI Interfaces in Platform Setup
set_property PFM.AXI_PORT {M_AXI_HPM0_FPD {memport "M_AXI_GP" sptag "" memory "" is_range "false"} M_AXI_HPM1_FPD {memport "M_AXI_GP" sptag "" memory "" is_range "false"} S_AXI_HPC0_FPD {memport "S_AXI_HPC" sptag "HPC0" memory "" is_range "false"} S_AXI_HPC1_FPD {memport "S_AXI_HPC" sptag "HPC1" memory "" is_range "false"} S_AXI_HP0_FPD {memport "S_AXI_HP" sptag "HP0" memory "" is_range "false"} S_AXI_HP1_FPD {memport "S_AXI_HP" sptag "HP1" memory "" is_range "false"} S_AXI_HP2_FPD {memport "S_AXI_HP" sptag "HP2" memory "" is_range "false"} S_AXI_HP3_FPD {memport "S_AXI_HP" sptag "HP3" memory "" is_range "false"}} [get_bd_cells /zynq_ultra_ps_e_0]

set_property PFM.AXI_PORT {M01_AXI {memport "M_AXI_GP" sptag "" memory "" is_range "false"} M02_AXI {memport "M_AXI_GP" sptag "" memory "" is_range "false"} M03_AXI {memport "M_AXI_GP" sptag "" memory "" is_range "false"} M04_AXI {memport "M_AXI_GP" sptag "" memory "" is_range "false"} M05_AXI {memport "M_AXI_GP" sptag "" memory "" is_range "false"} M06_AXI {memport "M_AXI_GP" sptag "" memory "" is_range "false"} M07_AXI {memport "M_AXI_GP" sptag "" memory "" is_range "false"}} [get_bd_cells /ps8_0_axi_periph]

12- In the interrupt tab, enable the intr

13- Update platform name

14- Fast Track

If you are stuck sumwhere, there is a Vivado TCL script create_zcu104_hw_acc_platform.tcl available in the Attachment section. It create a Vivado project and everything in it to this point.

Run the script as soon as Vivado started in the Tcl Console window with a source command.

source create_zcu104_hw_acc_platform.tcl

15- Export hardware - XSA

• Validate the block design - press F6 (ignore axi_intc_0/intr Warning)
• Create HDL Wrapper (In Source tab, right click design_1.bd file in Design Sources group)
• Generate Block Design and select Synthesis Options to Global. Then click Generate.

• Generate Bitstream - Click the button in the Flow Navigator window.
• Export the Platform - click the button in Flow Navigator window. Select Hardware, then Pre-synthesis and enable Include bitstream.

This is the end of the first Vivado part of creating Platform. You can now close the Vivado program.

PART-2

Build the PetaLinux and GNU Radio

We will create the Software Components under PetaLinux 2021.2 and add a new Yocto meta layer META-SDR for building GNU Radio SW with additional modules.

Creating the PetaLinux project

Go to the software folder we created earlier in Part-1.

$ cd ../software

1- - Run the settings.h script where the PetaLinux tools is installed

$ source /tools/petalinux/2021.2/settings.sh

2- Create a PetaLinux project named zcu104_2021_2-petalinux and configure the hardware with the XSA file created before in Part-1:

$ petalinux-create -t project --template zynqMP -n zcu104_2021_2-petalinux
$ cd zcu104_2021_2-petalinux
$ petalinux-config --get-hw-description=../../hardware/zcu104_2021_2-vivado/

The “misc/config System Configuration” window will appear.

3- Set to use ZCU104 device tree in this configuration window.

• Go to the DTG Settings->MACHINE_NAME and modify it to zcu104-revc.

4- Set the bootarguments:

• Go to DTG settings -> Kernel Bootargs -> generate boot args automatically to NO
• Update User Set Kernel Bootargs to

earlycon console=ttyPS0,115200 clk_ignore_unused root=/dev/mmcblk0p2 rw rootwait cma=512M

5- Add EXT-4 support

• Go to the Image Packaging Configuration > Root filesystem type and choose EXT4 (SD/eMMC/QSPI/SATA/USB).

Save and exit. Wait for the configuration to complete.

Modifying the Device Tree

We need to add some driver configurations for SD card.

Edit the project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi file.

$ vi project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi

And add the following content to the file:

/include/ "system-conf.dtsi"
/ {
};

/* SD */
&sdhci1 {
        disable-wp;
        no-1-8-v;
};

Save file and exit.

Config the Kernel

Just run the following command and wait about a half an hour for the menu to open (only the first time).

$ petalinux-config -c kernel

Disable CPU IDLE in kernel config:

• Go to CPU Power Management > CPU Idle > CPU idle PM support - set 'N' n the [ ] menu selection.

• Go to CPU Power Management > CPU Frequency scaling > CPU Frequency scaling - set 'N'.

Accept all other defaults, save and exit. Wait for a few minutes for the process to finish.

Adding GNU Radio software to RootFS file system

We need to add a new Yocto layer for the following SW to build in the system:

• GNU Radio v3.8 - main program
• gr-osmosdr v3.8 - module needed for SDR receivers
• gr-fpga_ai v3.8 - module needed for accessing DPU on FPGA

All the components above are located in the GitHub repositories and will be added to build system by adding additional Yocto recipes, stored in META-SDR GitHub repository.

1- Change directory to project-spec and clone the META-SDR repository from GitHub. Change branch to DPU-FPGA, because only this branch in S59MZ GitHub repository is appropriate for our PetaLinux building.

$ cd project-spec
$ git clone https://github.com/s59mz/meta-sdr
$ cd meta-sdr
$ git checkout dpu-fpga

2- Now, we need to tell the build system the location of these new added META-SDR Yocto layer.

• Change directory back to zcu104_2021_2-petalinux directory, from where we started.

$ cd ../..
$ pwd
 
/tools/workspace/axu2cgb/software/zcu104_2021_2-petalinux

3- Run petaLinux-config

$ petalinux-config

4- Go to Yocto Settings > User Layers and add a new user layer, named:

${PROOT}/project-spec/meta-sdr

5- Optionally, you can also set the MAC Address in this menu program.

• Go to Subsystem AUTO Hardware Settings > Ethernet Settings and untick Randomise MAC
• Enter a new MAC Address.

Save and exit.

6- Verify the new added layer by viewing the build/conf/bblayers.conf file. The file should have a line at the end that refers to the meta-sdr directory.

$ tail build/conf/bblayers.conf

# /tools/workspace/zcu104/software/zcu104_2021_1-petalinux/project-spec/meta-sdr \

Config the RootFS file system

Before we start the configuration program, we need to edit the user-rootfsconfig file, and add some names of user packages into. These packages will be built and added to the RootFS file system.

1- Edit the project-spec/meta-user/conf/user-rootfsconfig file.

$ vi project-spec/meta-user/conf/user-rootfsconfig

And add the following content into:

Note: Mention Each package in individual line
#These packages will get added into rootfs menu entry

CONFIG_gpio-demo
CONFIG_peekpoke

# Packages for base XRT support
CONFIG_xrt

# ackages for easy system management
CONFIG_dnf
CONFIG_e2fsprogs-resize2fs
CONFIG_parted
CONFIG_resize-part

# Packages for Vitis-AI dependencies support
CONFIG_packagegroup-petalinux-vitisai

# Optional Packages for natively building 
# Vitis AI applications on target board
CONFIG_packagegroup-petalinux-self-hosted
CONFIG_cmake
CONFIG_packagegroup-petalinux-vitisai-dev
CONFIG_xrt-dev
CONFIG_opencl-clhpp-dev
CONFIG_opencl-headers-dev
CONFIG_packagegroup-petalinux-opencv
CONFIG_packagegroup-petalinux-opencv-dev

# Optional Packages for running i
# Vitis-AI demo applications with GUI
CONFIG_mesa-megadriver
CONFIG_packagegroup-petalinux-x11
CONFIG_packagegroup-petalinux-v4lutils
CONFIG_packagegroup-petalinux-matchbox

# Packages for date and time settings
CONFIG_ntp
CONFIG_ntpdate
CONFIG_ntp-utils

# Some utils
CONFIG_git
CONFIG_zip
CONFIG_unzip

# Gnuradio and its modules
CONFIG_gnuradio
CONFIG_gr-osmosdr
CONFIG_gr-fpga-ai

Save file and exit.

2- Config the rootfs by running this command:

$ petalinux-config -c rootfs

3- Enable all the previously added user packages:

• Go to User Packages. Enable all, don't forget for gnuradio, gr-osmosdr and gr-fpga-ai.

4- Optionaly, enable OpenSSH and disable dropbear:

• Go to Image Features
• Disable ssh-server-dropbear and enable ssh-server-openssh.

• Go one level up and to Filesystem Packages > misc >packagegroup-core-ssh-dropbear and disable packagegroup-core-ssh-dropbear

• Go three level up again and to Filesystem Packages > console > network > openssh and enable openssh, openssh-sftp-server, openssh-sshd, openssh-scp

5- Enable Package Management:

• Go to four level up and to Image Features and enable package-management and debug_tweaks

6- Save and exit from the menu.

Extra Image Feature Settings

Because we want to build our own Out of Tree (OOT) modules on this platform, we need to put the dev-pkgs EXTRA_IMAGE_FEATURE flag somewhere in the Yocto recipes.

We put this flag into the top of petalinux-image-minimal.bb recipe, otherwise, feel free to put it in the more convenient place, if you want to.

Edit the petalinux-image-minimal.bb file:

vi components/yocto/layers/meta-petalinux/recipes-core/images/petalinux-image-minimal.bb

and add the following content into the file:

DESCRIPTION = "OSL image definition for Xilinx boards"
LICENSE = "MIT"

require petalinux-image-common.inc

EXTRA_IMAGE_FEATURES += " \
    tools-debug \
    tools-profile \
    tools-sdk \
    dev-pkgs \
    package-management \
    debug-tweaks \
    "

Save file and exit.

Build PetaLinux Images

Run the command and wait for several hours to finish:

$ petalinux-build

At the end, check if all images (rootfs, Image, u-boot, etc) are built successfully in the image/linux directory. Check the time of files creation.

$ ls -lrt images/linux

Build SDK

Run the build --sdk command and wait for about an hour for the process to finish.

$ petalinux-build --sdk

This is the end of the Part-2 - Build the PetaLinux and GNU Radio sowtware components.

PART-3

Create the Vitis Platform with DPU for AI Inference

Prepare Files for Platform Packaging

1- Go to the previously created "platform" directory under your project main directory, or create it if doesn’t exist.

$ pwd
/tools/workspace/zcu104

$ mkdir platform
$ ls
hardware  platform  software

$ cd platform

2- Create some new directories for storing some previously created files.

$ mkdir -p pfm/boot
$ mkdir -p pfm/sd_dir

3- Copy the generated Linux software boot components from Part-2 to pfm/boot directory.

$ cp ../software/zcu104_2021_2-petalinux/images/linux/zynqmp_fsbl.elf pfm/boot/
$ cp ../software/zcu104_2021_2-petalinux/images/linux/pmufw.elf pfm/boot/
$ cp ../software/zcu104_2021_2-petalinux/images/linux/bl31.elf pfm/boot/
$ cp ../software/zcu104_2021_2-petalinux/images/linux/system.dtb pfm/boot/
$ cp ../software/zcu104_2021_2-petalinux/images/linux/u-boot-dtb.elf pfm/boot/u-boot.elf

$ ls pfm/boot/
bl31.elf  pmufw.elf  system.dtb  u-boot.elf  zynqmp_fsbl.elf

NOTE: The last file u-boot-dtb.elf is renamed to u-boot.elf. Please, don't use the original u-boot.elf, because, it doesn't contain the device tree block.

4- Copy the boot.scr and system.dtb to pfm/sd_dir folder.

$ cp ../software/zcu104_2021_2-petalinux/images/linux/boot.scr pfm/sd_dir/
$ cp ../software/zcu104_2021_2-petalinux/images/linux/system.dtb pfm/sd_dir/

$ ls pfm/sd_dir/
boot.scr  system.dtb

5- Install sysroot into pfm folder.

Before installing sysroot, you must unset the system variable LD_LIBRARY_PATH

export LD_LIBRARY_PATH=

Now, you can install the sysroot from the previously generated sdk.sh script.

$ ../software/zcu104_2021_2-petalinux/images/linux/sdk.sh -d pfm

NOTE: The -d option tells the sdk.sh script where to install the sysroot.

PetaLinux SDK installer version 2021.2
======================================
You are about to install the SDK to "/tools/workspace/zcu104/platform/pfm". Proceed [Y/n]? 
Extracting SDK.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................done
Setting it up...done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
 $ . /tools/workspace/zcu104/platform/pfm/environment-setup-cortexa72-cortexa53-xilinx-linux

Create a Vitis Platform

Before starting the Vitis IDE program make sure you have set all the following paths and settings:

source /tools/Xilinx/Vitis/2021.2/settings64.sh

export OPENCV_INCLUDE=/usr/include/opencv4/
export OPENCV_LIB=/usr/lib/x86_64-linux-gnu/
export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/

NOTE: Set the LD_LIBRARY_PATH variable back, because it was removed during sysroot installation.

1- Launch Vitis IDE in the current platform folder in background mode and set the same folder as workspace directory.

$ pwd
/tools/workspace/zcu104/platform

$ vitis

2- Create a New Platform:

• File > New > Platform Project and name it zcu104_2021_2-platform for example. Click Next.

3- In Hardware Specification, click Browse button and select the XSA file generated by the Vivado at the end of Part-1.

/tools/workspace/zcu104/hardware/zcu104_2021_2-vivado/design_1_wrapper.xsa

• Select Operating system as linux and psu_cortexa53 and 64-bit.
• Uncheck Generate boot components. Click Finish.

4- Click the linux on psu_cortexa53 domain.

Browse to the locations and select the directory or file needed to complete the dialog box for the following:

• Bit File - click Generate BIF (at the end of Browse button
• Boot Components - browse to pfm/boot
• FAT32 Partition Directory - browse to pfm/sd_dir

Leave all other settings as they are.

NOTE: Leave Linux RootFS and Sysroot Directory blank for now.

Build the Platform

In the Explorer tab window right click on the created platform:

• zcu104_2021_2-platform and select Build Project. Wait for few seconds to finish.

Export the Platform

In the Explorer tab window right click on the created platform zcu104_2021_2-platform:

• Select Exportas Archive.
• Set platform name zcu104_2021_2-platform.zip
• Tick your Platform Project
• And tick Include build folders.

Wait for a few minutes to finish.

You can use this exported zip file for creating new projects.

Install Vitis AI

We create a new application that integrate a Deep Learning Processor unit DPU as a kernel. There is a script already provided from AMD-Xilinx, that create everything we needed.

For more details about the DPU, please read DPU-TRD GitHub repository and PG338 DPU IP Product Guide.

At first, we need to add Vitis-AI repository into Vitis IDE.

1- Launch Vitis IDE if you have not. We can reuse the workspace of the previously created platform.

2- Open menu Window > Preferences > Library Repository tab > Add and set the following:

• Input ID: vitis-ai
• Name: Vitis AI
• Git URL:https://github.com/Xilinx/Vitis-AI.git
• Branch: v2.0 or master

Apply and Close.

3- Download the Vitis-AI library:

• Open menu Xilinx -> Libraries.
• Find the Vitis-AI entry we just added.
• Click the Download button on it.
• Wait until the download of Vitis-AI repository completes (several minutes).
• Click OK

NOTE:

• git commit ID for v2.0 is: d02dcb6041663dbc7ecbc0c6af9fafa087a789de
• git commit ID for master is currently: ba8905493876674716ab08ea4266962193df970b

Create DPU Application Project

1- Create a New Application Project based on our previously created zcu104_2021_2-platform:

• Go to menu File > New > Application Project > Next
• Select platform zcu104_2021_2-platform. Click Next

2- Name the project dpu-test, click next.

3- Select a domain: linux on psu_cortexa53 and update in the Domain details:

• Set Sys_root path to sysroot installation path in previous step:

tools/workspace/zcu104/platform/pfm/sysroots/cortexa72-cortexa53-xilinx-linux

• Set the Root FS to rootfs.ext4

/tools/workspace/zcu104/software/zcu104_2021_2-petalinux/images/linux/rootfs.ext4

• Set Kernel Image to Image

/tools/workspace/zcu104/software/zcu104_2021_2-petalinux/images/linux/Image

NOTE: The last two files are located in zcu104_2021_2-petalinux/images/linux folder created in Part-2.

Click next.

4- Select a Template:

• dsa > DPU Kernel (RTL Kernel)

And click Finish to generate the application.

5- Set the Active Build Configuration:

• Double click the system project file dpu-test_system.sprj and
• Change Active Build Configuration to Hardware

DPU settings for ZCU104

Because our real hardware was actually AXU2CGB board, which doesn't have enough resources to build the 2xDPU kernels with default configuration, we changed the DPU configuration, so it uses less resources.

To keep the same settings as we tested on real hardware, we do the same for ZCU104 board. Feel free to configure a more powerful options, but in this case you need to re-build all AI models with the correct fingerprint parameter value set in the arch.json file. More about that later in the Part-4.

We need to integrate:

• only one kernel (not 2 by default)
• change the DPU processor selection
• disable some functionality.

Open dpu_conf.vh from dpu-test_kernels/src/prj/Vitis directory.

• change B4096 to B1152 (line #26)
• change DRAM_DISABLE to DRAM_ENABLE (line #59)
• change CHANNEL_AUGMENTATION_ENABLE to CHANNEL_AUGMENTATION_DISABLE (line #92)
• change DWCV_ENABLE to DWCV_DISABLE (line #103)
• change RELU_LEAKYRELU_RELU6 RELU_RELU6 (line #136)

Fill free to experiment with some more options enabled in case you have more powerful FPGA, but:

• make sure POOL_AVG is enabled, because our model needs it. (line 114),
• you should re-compile the AI model in case of different configuration as above.

Save the file and check everything twice, because the kernel build process lasts very long.

//Setting the arch of DPU, For more details, Please read the PG338 


/*====== Architecture Options ======*/
// |------------------------------------------------------|
// | Support 8 DPU size
// | It relates to model. if change, must update model
// +------------------------------------------------------+
// | `define B512               
// +------------------------------------------------------+
// | `define B800                 
// +------------------------------------------------------+
// | `define B1024                 
// +------------------------------------------------------+
// | `define B1152                 
// +------------------------------------------------------+
// | `define B1600                 
// +------------------------------------------------------+
// | `define B2304                 
// +------------------------------------------------------+
// | `define B3136                 
// +------------------------------------------------------+
// | `define B4096                 
// |------------------------------------------------------|

`define B1152 

// |------------------------------------------------------|
// | If the FPGA has Uram. You can define URAM_EN parameter               
// | if change, Don't need update model
// +------------------------------------------------------+
// | for zcu104 : `define URAM_ENABLE               
// +------------------------------------------------------+
// | for zcu102 : `define URAM_DISABLE                 
// |------------------------------------------------------|

`define URAM_DISABLE 

//config URAM
`ifdef URAM_ENABLE
    `define def_UBANK_IMG_N          5
    `define def_UBANK_WGT_N          17
    `define def_UBANK_BIAS           1
`elsif URAM_DISABLE
    `define def_UBANK_IMG_N          0
    `define def_UBANK_WGT_N          0
    `define def_UBANK_BIAS           0
`endif

// |------------------------------------------------------|
// | You can use DRAM if FPGA has extra LUTs               
// | if change, Don't need update model
// +------------------------------------------------------+
// | Enable DRAM  : `define DRAM_ENABLE               
// +------------------------------------------------------+
// | Disable DRAM : `define DRAM_DISABLE                 
// |------------------------------------------------------|

`define DRAM_ENABLE 

//config DRAM
`ifdef DRAM_ENABLE
    `define def_DBANK_IMG_N          1 
    `define def_DBANK_WGT_N          1
    `define def_DBANK_BIAS           1
`elsif DRAM_DISABLE
    `define def_DBANK_IMG_N          0
    `define def_DBANK_WGT_N          0
    `define def_DBANK_BIAS           0
`endif

// |------------------------------------------------------|
// | RAM Usage Configuration              
// | It relates to model. if change, must update model
// +------------------------------------------------------+
// | RAM Usage High : `define RAM_USAGE_HIGH               
// +------------------------------------------------------+
// | RAM Usage Low  : `define RAM_USAGE_LOW                 
// |------------------------------------------------------|

`define RAM_USAGE_LOW

// |------------------------------------------------------|
// | Channel Augmentation Configuration
// | It relates to model. if change, must update model
// +------------------------------------------------------+
// | Enable  : `define CHANNEL_AUGMENTATION_ENABLE              
// +------------------------------------------------------+
// | Disable : `define CHANNEL_AUGMENTATION_DISABLE                
// |------------------------------------------------------|

`define CHANNEL_AUGMENTATION_DISABLE

// |------------------------------------------------------|
// | DepthWiseConv Configuration
// | It relates to model. if change, must update model
// +------------------------------------------------------+
// | Enable  : `define DWCV_ENABLE              
// +------------------------------------------------------+
// | Disable : `define DWCV_DISABLE               
// |------------------------------------------------------|

`define DWCV_DISABLE

// |------------------------------------------------------|
// | Pool Average Configuration
// | It relates to model. if change, must update model
// +------------------------------------------------------+
// | Enable  : `define POOL_AVG_ENABLE              
// +------------------------------------------------------+
// | Disable : `define POOL_AVG_DISABLE                
// |------------------------------------------------------|

`define POOL_AVG_ENABLE

// |------------------------------------------------------|
// | support multiplication of two feature maps
// | It relates to model. if change, must update model
// +------------------------------------------------------+
// | Enable  : `define ELEW_MULT_ENABLE           
// +------------------------------------------------------+
// | Disable : `define ELEW_MULT_DISABLE               
// |------------------------------------------------------|

`define ELEW_MULT_DISABLE

// +------------------------------------------------------+
// | RELU Type Configuration
// | It relates to model. if change, must update model
// +------------------------------------------------------+
// | `define RELU_RELU6
// +------------------------------------------------------+
// | `define RELU_LEAKYRELU_RELU6
// |------------------------------------------------------|

`define RELU_RELU6

// |------------------------------------------------------|
// | DSP48 Usage Configuration  
// | Use dsp replace of lut in conv operate 
// | if change, Don't need update model
// +------------------------------------------------------+
// | `define DSP48_USAGE_HIGH              
// +------------------------------------------------------+
// | `define DSP48_USAGE_LOW                
// |------------------------------------------------------|

`define DSP48_USAGE_HIGH 

// |------------------------------------------------------|
// | Power Configuration
// | if change, Don't need update model
// +------------------------------------------------------+
// | `define LOWPOWER_ENABLE              
// +------------------------------------------------------+
// | `define LOWPOWER_DISABLE               
// |------------------------------------------------------|

`define LOWPOWER_DISABLE

// |------------------------------------------------------|
// | DEVICE Configuration
// | if change, Don't need update model
// +------------------------------------------------------+
// | `define MPSOC              
// +------------------------------------------------------+
// | `define ZYNQ7000               
// |------------------------------------------------------|

`define MPSOC

Remove SoftMax function and one kernel instance (optional)

We use software implementaion of SoftMax function that run on the main CPU, so we remove it from kernel to save some resources.

Again, we can integrate only one kernel instance of DPU, if we are using AXU2CGB board.

Double click dpu-test_system_hw_link.prj.

• Remove sfm_xrt_top function by right clicking and select Remove
• Change the Compute Unit of DPUCZDX8G to to 1

V++ Configuration settings

In case we removed one kernel instance of DPUCZDX8G, we also need to remove all clocks and connectivity dedicated to previously removed second instance.

• Go to Assistant View
• Double click dpu-test_system [System]
• Expand the left tree panel and find: dpu-test_system > dpu-test_system_hw_link > Hardware > dpu

Click ... button on the line of V++ Configuration Settings. Change the V++ linker settings window content to the following:

[clock]
freqHz=300000000:DPUCZDX8G_1.aclk
freqHz=600000000:DPUCZDX8G_1.ap_clk_2

[connectivity]
sp=DPUCZDX8G_1.M_AXI_GP0:HPC0
sp=DPUCZDX8G_1.M_AXI_HP0:HP0
sp=DPUCZDX8G_1.M_AXI_HP2:HP1

Click OK, Apply, Apply and then Apply and Close.

Adding a custom Python Application to project

Now, we add our new python test scripts from GitHub repository from command line interface.

• Go to Ubuntu Terminal program and change directory to dpu-test/src.
• Clone repository named test-dpu from my (S59MZ) GitHub account.
• Switch to branch mmcblk0, because we use the ZCU104 board.

$ cd dpu-test/src

$ git clone https://github.com/s59mz/test-dpu
$ cd test-dpu
$ git checkout mmcblk0
$ rm -rf test-dpu/.git

Now we have a new folder test-dpu with some test python scripts.

NOTE: In case you have a different hardware, that uses SD1 interface for SD card (axu2cgb board for example), you should checkout the main branch.

Replace old resnet50 model (Optionally)

You can replace the original resnet50.xmodel in the app/model folder with a new re-compiled one, because we changed the DPU settings in the dpu_conf.vh file.

But only, if you need to run the DPU demo test with default included resnet50 model. This is not needed for our RF Modulation Recognition application on GNU Radio.

There is the re-compiled resnet50.xmodel, with the right fingerprint already in the previously cloned GitHub repository in the dpu-test/models directory. Replace the old model with the re-compiled one.

$ cd ../../src/app/model
$ rm resnet50.xmodel
$ mv ../../test-dpu/models/resnet50.xmodel .

NOTE: There is also the dpu_conf.vh file that should go to dpu-test_kernels/src/prj/Vitis directory, in case you didn’t set the DPU configuration in the previous section already. You can compare both files, should be the same.

Application Packaging

Go back to the Vitis IDE.

Update package options to add app and this new test-dpu folder also into the SD Card.

• Double click dpu-test_system.sprj
• Click... button on Package options
• Input:

--package.sd_dir=../../dpu-test/src/test-dpu --package.sd_dir=../../dpu-test/src/app

Click OK

NOTE: there is a space between /test-dpu and --package.sd_dir

NOTE: A folder to be added in the src directory should not start with the word dpu-something. That causes a Makefile error (don't know why).

Build the hardware design

NOTE: Make sure you have set the LD_LIBRARY_PATH variable correctly. Optional, restart the Vitis IDE too, just in case.

Right click on the dpu-test_system[zcu104_2021_2-platform] system project and select BuildProject. Wait for about an hour to finish.

The generated SD card image is located at dpu-test_system/Hardware/package/sd_card.img. We need that one for our Target board.

The most important file, generated in this build process, is dpu.xclbin. It is the FPGA binary image that implements the DPU in the FPGA. It is located in Hardware directory, added to the boot partition of the SD card and included in the sd_card.img image automatically.

Test the GNU Radio Platform on Target

Prepare an empty SD card (at least 8GB).

The prepared SD card image for ZCU104 board can be also downloaded from the GitHub repository.

Write the sd_card.img created in the previous step on an empty SD card with a Linux DD command.

NOTE: Double check the output device of=/dev/sd... It must point to the SD card, unless you can loose all of your data on your Host machine.

$ cd dpu-test_system/Hardware/package
$ sudo dd if=sd_card.img of=/dev/sdXX bs=512; sync

Insert the SD Card into the micro-SD card slot of your ZCU104 board, connect the Serial Console cable to your Host machine via USB port and run the Serial Terminal program, minicom for example.

$ minicom -D /dev/ttyUSB0

Connect the Ethernet cable to the board. The board and the Host Machine should be in the same local area network.

Power up the ZCU104 board. The U-Boot should start, then the Linux Kernel and the rootfs file system. Wait for a prompt.

Check the Linux version and the IP address of the board by running the "uname -a" and ifconfig Linux commands, respectively.

$ uname -a
$ ifconfig

Note the IP address, because we need it for SSH access.

Prepare the Host Machine

Now we use a Host computer as a Display, so we prepare the Host computer as X11Server and ssh to the board with "-X" option.

On the Host computer:

• Check the $DISPLAY variable. Should be ":1".
• Add the local connection to the access control list. Type: "xhost +local:"
• Open ssh -X session to the board from the Host machine.
• Login as root with the password root.

$ echo $DISPLAY
$ xhost +local:
$ ssh -X root@192.168.1.73

Prepare the Target

The first thing to do is resize the EXT4 partition of the SD Card. This command should be run only once.

resize-part /dev/mmcblk0p2

Also, update the Packet Manager.

$ dnf update

If you want to build your own OOT modules for GNU Radio, the following python libraries should be installed manually. Run these commands only once.

$ pip3 install Click
$ pip3 install click-plugins

Tesing the DPU unit on real Target

Now, we can run some tests on the board.

Before running any tests, you should set some system variables.

For using the DPU unit, this variable points to the FPGA binary image:

$ export XLNX_VART_FIRMWARE=/media/sd-mmcblk0p1/dpu.xclbin

For executing any C/C++ programs that uses DPU, this variable points to the needed libraries:

$ export LD_LIBRARY_PATH=/media/sd-mmcblk0p1/app/samples/lib

For using any GNU Radio modules, this variable points to location of Python packages:

$ export PYTHONPATH=/usr/lib/python3/dist-packages/

To get the DPU fingerprint, needed for building our own AI modules, the "xdputil query" command can be useful:

$ export XLNX_VART_FIRMWARE=/media/sd-mmcblk0p1/dpu.xclbin
$ xdputil query

Find the "fingerprint":"0x100002062010103" parameter at the near end of the console print and store this parameter inside curled brackets {} to a new created arch.json file.

Save this file on the Host machine somewhere. This file is needed for compiling modules with Vitis-AI.

$ echo '{"fingerprint":"0x100002062010103"}' > arch.json

NOTE: Make sure that brackets and double quotes are included in the file.

$ cat arch.json
{"fingerprint":"0x100002062010103"}

NOTE: The original arch.json generated by Vitis during creating DPU-Test application has wrong value, due to unknown reason. Fingerprint ended with -203 instead of -103. Please, use fingerprint read from FPGA directly for compiling your own models.

Change directory to the test-dpu on BOOT partition of the SD card, where all needed testing scripts are located:

$ cd /media/sd-mmcblk0p1/test-dpu

There are five bash test scripts lotated in this directory, all started with "run_" prefix.

Feel free to explore the scripts and check if all system variables in the scripts points to the right location, especially check mmcblk0p1 (for zcu104) or mmcblk1p1 (for axu2cgb) SD card partitions, depending on your hardware platform you chose.

$ ls -l run_*

run_recognize_image.sh
run_test_performance.sh
run_test_accuracy.sh

run_gnuradio.sh
run_rf_classification.sh

Run each script and check if everything works correctly.

The last two tests, that uses GNU Radio, should be run via ssh -X session only, because they use graphic display through X11 Server on your host machine.

NOTE: In case of repeating errors, try to reboot the board first.

$ reboot -h

Image Classification Test

Run the run_recognize_image.sh script:

$ pwd
/media/sd-mmcblk0p1/test-dpu

$ ./run_recognize_image.sh

Recognising Unknown Image...
score[688]  =  0.999914     text: oscilloscope, scope, cathode-ray oscilloscope, CRO,
score[662]  =  1.30062e-05  text: modem,
score[844]  =  1.30062e-05  text: switch, electric switch, electrical switch,
score[592]  =  1.01292e-05  text: hard disc, hard disk, fixed disk,
score[811]  =  7.88864e-06  text: space heater,
/media/sd-mmcblk0p1/test-dpu

The test is passed if the picture is recognized correctly. Feel free to change the picture inside the script with a different one.

RF Modulation Classification Performance Test

Run the run_test_performance.sh script.

Make sure the measured FPS is about 500 frame per seconds (measured on axu2cgb board).

$ pwd
/media/sd-mmcblk0p1/test-dpu

$ ./run_test_performance.sh

Number of RF Samples is  2000
FPS=532.79, total RF frames = 2000.00 , time=3.753795 seconds

RF Modulation Classification Accuracy Test

Run the run_test_accuracy.sh script.

Make sure the top-1 accuracy is about 0.53. We used a baseline model that was trained with only 6% of available samples in the dataset.

$ pwd
/media/sd-mmcblk0p1/test-dpu

$ ./run_test_accuracy.sh

Number of RF Samples Tested is  998
Batch Size is  1
Top1 accuracy =  0.53

FM Radio Test

This test uses graphic display and should be run through ssh -X session.

It is a simple FM radio implemented in GNU Radio platform. The board should have inserted an RTL-SDR USB dongle into the available USB port.

It is a software defined radio receiver as a USB dongle. An appropriate small indor antenna should be connected to the RTL-SDR.

Optionally but not needed, connect a USB Sound Card on the second USB port (in case your hardware have it). PC audio speakers or phones connect to the audio jack input of the sound card. It is also possible to route audio to X11 server, without using USB sound card (more about that later).

When everything is connected, run the run_gnuradio.sh script.

• The GNU Radio will start.
• Click on the Play button on the Toolbox above or press F6 on the keyboard.
• A new GUI would open. Ignore the Warning message (it shows only once)
• Resize the new GUI to be more visible.
• Set the Frequency with the GUI slider to the local FM broadcast radio station nearby.
• You should see the received radio spectrum in the GUI widget and optionally, hear the selected local radio station in the speakers (if they are connected).
• To route audio into X11 Server, close the radio GUI and double click on the Audio Sink block.
• Delete the content of the Device Name field that has a value of "hw:0, 0".
• Click OK and then Play again.
• You should hear the audio on your Host computer's speakers now.
• Feel free to change the frequency to any other FM broadcast station nearby.

Running RF Modulation Recognition app in GNU Radio

This is a simple GUI application developed in GNU Radio platform that runs under Python.

The application receives signals on the 2m radio amateur band and predicts modulations of the receiving digital signals in real time by using AI inference in the FPGA. It uses the Deep learning Processor Unit (DPU) implemented in the FPGA on the board.

The test uses graphic display and should be run through ssh -X session.

The board should have inserted an RTL-SDR USB dongle into available USB port on the target board.

It is a software defined radio receiver as a USB dongle. An external VHF/UHF antenna is recommended to be connected to the RTL-SDR.

Optionally, connect a USB Sound Card with speakers or phones connected to the audio jack input of the sound card. It is also possible to route audio to X11 server, without using USB sound card.

NOTE: The tablet computer in the demo works as a remote desktop of X11 Server only, and shows the GUI from GNU Radio, running on the FPGA board.

When everything is connected, run the run_rf_classification.sh script.

• The application starts automatically. Resize the GUI appropriately.
• The SDR receiver is tuned to 144.8 MHz on radio amateur band and listens for some random digital signals on this band.
• When a strong signal appears, it can be shown on a waterfall.
• At this moment the program start predicting the modulation on the received signal by running AI inference. The predicted modulation is printed on the SSH console and the histogram in the GUI widget counts all of the predicted class indexes.
• Samples of analyzed signal are also shown in the time domain too (oscilloscope widget).

NOTE: To get the block design view in the background, run gnuradio-companion first and then open the RF_Classification.grc file:

$ export XLNX_VART_FIRMWARE=/media/sd-mmcblk0p1/dpu.xclbin
$ export PYTHONPATH=/usr/lib/python3/dist-packages/
$ gnuradio-companion RF_Classification.grc

The AI model is already included in the application in the test-dpu/models/rfClassification.xmodel file. It is a baseline model trained with 10 epochs only and with only 6% of the available samples from the dataset.

To change the AI model, double click on the "RF Classification" block and change the path to the .xmodel file.

It is also possible to test the AI model with I/Q samples from a file. Open the other example in the same directory, named IQ_Samples.grc. An I/Q sample can be chosen in the Sample Modulation Source block by double click on it and change the Index parameter. The selected modulation will be print on the console at start.

Feel free to build your own model and train it with more available samples in the next (last) part of these series:

PART-4

Build the AI Model in Colab and Vitis-AI

In this part we build and train our own model based on TensorFlow framework. Then the model will be quantize, calibrated and compiled, so can be used on the DPU of the FPGA.

First, we need a dataset for training a model.

Get the RF Datasets

The RF dataset can be freely downloaded from RF Datasets for Machine Learning website. Select the RADIOML 2018.01A dataset and download the 2018.01.OSC.0001_1024x2M.h5.tar.gz file.

You should provide your contact information for download, but the dataset is free of charge. You need about 20 GB of free space on disk just for download.

There is another option to reduce the size of the dataset to about 6% only. That is enough for start. This tutorial uses a reduced dataset (about 1.2 GB only) and can be downloaded from provided links inside the attached Jupyter Notebook scripts.

Here is the script that was used for reducing the original large dataset:

reduce_dataset.ipynb

You can skip this downloading step and just jump to the next step, in case, you don't want to download and use the whole dataset.

Otherwise, if you want to use the original dataset as a whole, jump to the Xilinx Tutorial - RF Modulation Recognition GitHub page and use the their original script to build the model. In this case, you can skip the remaining part of this tutorial.

Get the Jupyter Notebook Scrips

There are two Jupyter notebook scripts provided in this tutorial. The first one will be run on Google Colab, and the second one on the Xilinx Vitis-AI Docker image.

Clone the s59mz/test-dpu GitHub repository and note both Jupyter scripts in the Jupyter subdirectory. We will need it later.

• rf_classification-Colab.ipynb
• rf_classification-Vitis-AI.ipynb

$ git clone http://github.com/s59mz/test-dpu
$ cd test-dpu/
$ ls jupyter/
rf_classification-Vitis-AI.ipynb
rf_classification-Colab.ipynb
reduce_dataset.ipynb

NOTE: The reduce_dataset.ipynb can be used for manually reducing the original dataset, mentioned in the previously section.

Get the arch.json file

This is very important file for compiling the model for the DPU. The file is located in the models directory:

$ ls models/*.json
models/arch_b1152.json
$ cat models/arch_b1152.json
{"fingerprint":"0x100002062010103"}

NOTE: The fingerprint can be read from Target board directly by running the command (on the FPGA board):

$ export XLNX_VART_FIRMWARE=/media/sd-mmcblk0p1/dpu.xclbin
$ xdputil query

NOTE: Edit the arch.json file and update the fingerprint parameter manually in case you change the DPU build configuration (dpu_conf.vh file)

Building and Training the AI model

The AI model that we build is developed in TensorFlow. We use the Google Colab service for training the model, because we can use their very powerful graphic cards for computing for free, but only for a limited time (for a few hours).

Be sure to download and save your trained model to your computer before Colab disconnect you from their service. You can repeat the whole process on the next day, as many times you want, but each time you must start from the beginning.

• Open your Web Browser (Google Chrome or Firefox is recommended) and open the Google Colab web page.
• Upload the first script rf_classification-Colab.ipynb to the Colab (File > Upload notebook). You should Sign-In with your Google account if you didn't already.
• Expand the Filesicon on the left hand side of the window, so you can see new created files there (sample_data).

• Set the Runtime to GPU: Runtime > Change Runtime type > GPU and Save, if it is available, otherwise use CPU, but it will take much longer time for training.
• Start the script: Runtime > Run all and wait for about a half an hour.

What justhappened?

• The script first download the reduced dataset from my Google Drive. You can change the path in Cell #2 (!wget...) to use your own dataset. You should see a new file reduced_rf_dataset_XYZ.hdf5 appeared on the left side.
• After some data manipulation and preparation, the model will be trained and you can see some saved checkpoint states in the checkpoint folder.

When the script finishes, you noticed another new file: the saved best trained model, named rf-model-best.h5.

• Download the trained model rf-model-best.h5 to your computer, before the Colab disconnects you from the service due to inactivity.
• The trained model is also available on the GitHub repository (under different name): rfClassification_fp_model.h5
• Explore the results. Here's a Confusion Matrix we got with only 6% of the available samples from the dataset and we trained the model for only 10 epochs.

• Not perfect, but good enough as a starting point to test the model on the real FPGA.
• Feel free to repeat the whole process again with larger dataset or longer training time (epochs) or even improve the model, if you want, to get more accurate results.
• But, remember: You have a limited time per a day for using Google Colab (about 2 hours, maybe). So close the session each time after you download the trained model (Connect > Manage sessions > Terminate).

Compiling the AI Model on Vitis-AI

Now, as we have a floating point model named rf-model-best.h5, we need to quantize, calibrate and compile it for using it on the DPU unit of the real FPGA.

The details of this process is out of scope of this tutorial. Feel free to:

• Explore the provided script rf_classification-Vitis-AI.ipynb
• Read the official GitHub page for Vitis-AI.

Here are the main steps:

• Install the Docker on your Host machine, if you don't have it already.
• Download the latest Vitis AI Docker image with the following command:

docker pull xilinx/vitis-ai-cpu:latest

Clone the Vitis-AI GitHub repository to your Host Machine.

git clone --recurse-submodules https://github.com/Xilinx/Vitis-AI

• Copy the previously cloned rf_classification-Vitis-AI.ipynb script to the Vitis-AI repository and also the trained model rf-model-best.h5 from the previous section. You can made a new my_model directory and a sub-directory fp_model where you copy your trained model.
• Copy also the arch_b1152.json file to the same directory and don't change the file name unless you update the Jupyter notebook script too.

$ cd Vitis-AI
$ mkdir -p my_model/fp_model
$ cp ~/test-dpu/jupyter/rf_classification-Vitis-AI.ipynb my_model
$ cp ~/Downloads/rf-model-best.h5 my_model/fp_model
$ cp ~/test-dpu/models/arch_b1152.json my_model
$ cd my_model
$ ls
arch_b1152.json  fp_model  rf_classification-Vitis-AI.ipynb

Start the Docker image from the my_model directory with a bash script.

$ ../docker_run.sh xilinx/vitis-ai

• When Docker started, switch to TensorFlow2 by executing conda activatevitis-ai-tensorflow2 command.

• Start Jupyter Notebook by running the command (inside the Docker):

$ jupyter notebook --no-browser --ip=0.0.0.0 --NotebookApp.token='' --NotebookApp.password=''

• Now, open the Web Browser on your Host Machine and open the web page named http://localhost:8888/

• Click on the previously copied script rf_classification-Vitis-AI.ipynb

• Run the script: Kernel > Restart & Run All and wait for a few minutes to get the compiled model.
• NOTE: The reduced dataset (1.2 GB), needed for calibration, will be downloaded to your computer. You can modify the script and use your own dataset for calibration.
• Next, a new quantize model will be created, which is optimized by the calibration dataset, downloaded previously. It is located in quantize_results/quantized_model.h5 directory.
• At the end, a new directory vai_c_output will be created.
• Inside is our rfClassification.xmodel file, ready to be uploaded to our Target board.

Test on the Target Board

Now, we are ready to test our model on the real Hardware.

• Copy or replace the original rfClassification.xmodel file on the SD card with a new one. It is located on the first partition in the test-dpu/models directory.

$ pwd
/media/sd-mmcblk0p1/test-dpu/models

• If you change the name of the model, you should update the test scrips also and also the RF Claasification GNU Radio module. Otherwise, just replace the original file with a new one.
• Run the Accuracy Test from the Part-3:

$ pwd
/media/sd-mmcblk0p1/test-dpu
$ ./run_test_accuracy.sh
Number of RF Samples Tested is  998
Batch Size is  1
Top1 accuracy =  0.53

And at last...

Start the RF Modulation Recognition application with updated module on GNU Radio again.

$ pwd
/media/sd-mmcblk0p1/test-dpu
$ ./run_rf_classification.sh

That's all !

If you come here to the end through all four parts successfully, Congratulations!

Anyway, feel free to comment on this blog.