Wi-Fi Connectivity on the Ultra96-V2 in Vivado+PetaLinux 2019.2

The Ultra96 is a unicorn of sorts in the FPGA development board market, as it is by far the most affordable board hosting an MPSoC UltraScale+ Zynq chip. One of the most pivotal peripherals on the Ultra96 board is the Microchip ATWILC3000 Wi-Fi+Bluetooth radio module. While it is technically possible to utilize the radio with bare metal applications, it is more ideal to configure an embedded Linux image configured to support the ATWILC3000.

Host Platform: Ubuntu 18.04.3 LTS running in a Parallels VM on Mac OSX
Target Development Software: Vivado/Vitis 2019.2 + PetaLinux 2019.2
Target Development Board: Ultra96-V2 with U96 JTAG/UART to USB adapter

Even though I've covered generating the hardware design for the Ultra96-V2 in a previous post, it was only a base design meant for getting started that only laid out connections for the low speed mezzanine connector and the GPIO to control the cooling fan for the MPSoC chip. A more complete design that gives the MPSoC on the Ultra96 access to the ATWILC3000, high speed mezzanine connector, and LEDs is necessary. I have exported my block design in the form of TCL script and placed it in the 'documentation' directory in the GitHub repository for this project to allow for straightforward project recreation.

For the pulse-width modulation (PWM) control of the CPU fan on the Ultra96, Avnet has some custom IP that needs to be added to the Vivado project in the form of an IP repository. I've also copied this into my GitHub repo for this project in the 'documentation' directory along with the TCL script to regenerate the block design. Since the TCL script for the block design calls out the Avnet IP for the PWM, it's important to add the PWM repo to the Vivado project before running the TCL script to generate the block design.

To add the PWM repo, go to Settings > IP > Repository and point to the directory for PWM_w_Int. Click 'Apply' before exiting the Settings menu.

Now from the TCL console, run the script with the following command:

source <path>/u96_design.tcl

Once the block design is complete, run validation as a sanity check then save it. Create an HDL wrapper auto managed by Vivado and import the constraints file for this project from my project repo on GitHub (ultra96v2_1/ultra96v2_1.srcs/constrs_1/new/u96_pinout.xdc). At this point, run synthesis, implementation, and generate a bitstream.

As a final step in Vivado, export the hardware to the default directory (checking the box to 'include bitstream').

The XSA packages the hardware design in the format needed to create the PetaLinux project.

To create the PetaLinux project, first run the PetaLinux environment settings script to source the needed command set. Then run the following:

petalinux-create --type project --template zynqMP --name <PROJECT NAME>

Change directories into the newly created project folder and then import the XSA hardware definition with the 'petalinux-config' command.

petalinux-config --get-hw-description /<location of XSA file>

This command will bring up the hardware system configuration editor. Only a few tweaks are needed, the first of which is to change the serial port from PS UART0 to PS UART1 under Subsystem AUTO Hardware Settings.

The root filesystem will be living on an SD card and the Ultra96 will also be booting from the SD card so the hardware settings for the boot image need to be change to 'primary SD'.

This includes changing the boot image, u-boot env partition, kernel image, and dtb image to 'primary sd' under Subsystem AUTO Hardware Settings > Advanced bootable images storage settings.

The root filesystem type needs to be changed to EXT (SD/eMMC/QSPI/SATA/USB) and the option for 'Copy final images to tftpboot' unchecked under the Image Packaging Configuration tab. After this change, exit the configuration editor and save.

While PetaLinux will generate a bare minimum device tree for the Zynq MPSoC system, any peripherals external to the chip need to be added manually to the device tree package of BSP recipe in the meta-user layer(<PetaLinux project directory>/project-spec/meta-user/recipes-bsp/device-tree). Most of the device tree source files in this directory are auto-generated based on other project settings, but the two files that are meant for user edits are system-user.dtsi and pl-custom.dtsi. For the Ultra96, only the system-user.dtsi needs to have nodes added to it to define the peripherals on the board for the kernel. There are several repositories that have device tree examples for the Ultra96 board, but I ended up needing to create my own using nodes from several of these examples (linked here). While all of the examples for the Ultra96-V2 had the SDHC interface node for the ATWILC3000 chip:

&sdhci1 {
    bus-width= <0x4>;
    disable-wp;
    max-frequency = <50000000>;
    /delete-property/cap-power-off-card;
    mmc-pwrseq = <&sdio_pwrseq>;
    status = "okay";
    #address-cells = <1>;
    #size-cells = <0>;
    wilc_sdio@1 {
        compatible = "microchip,wilc3000", "microchip,wilc3000";
        reg = <0>;
        bus-width= <0x4>;
        status = "okay";
    };
// Remove TI child node from U96 V1 DT
/delete-node/wifi@2;
};

None of the Ultra96-V2 device tree examples had the node for the SD card slot:

/* SD0 only supports 3.3V, no level shifter */
&sdhci0 {
no-1-8-v;
disable-wp;
};

Unfortunately I spent more time than I'd like to admit troubleshooting why the boot sequence would freeze trying to load the root filesystem before I finally noticed that this node was missing so the kernel wasn't able to see the SD card to be able to load the root filesystem from it.

The u-boot package also lives in the BSP recipe with the device tree package (<PetaLinux project directory>/project-spec/meta-user/recipes-bsp/u-boot), which also needs some modifications for the Ultra96's SD boot mode. The platform.h file in the u-boot recipe is the source file from which the u-boot environment is generated. This is where the proper command set is passed to tell u-boot where to load the kernel and device tree from after the first stage boot loader passes control off to u-boot. A bsp.cfg file enables a few more options needed by the Ultra96 such as DHCP and ping mode. After updating the platform.h file to match the linked one, copy this BSP file into the same directory with the platform.h file and update the source URI of the bitbake file to include it in the build by adding the following line:

SRC_URI += "file://bsp.cfg"

The BSP recipe of the Ultra96 PetaLinux project needs a couple of extra packages added to it to specify the ATWILC3000 firmware and the scripts to start the Wi-Fi and Bluetooth services as well as the wpa supplicant configuration file.

In the <PetaLinux project directory>/project-spec/meta-user/recipes-bsp/ directory of the project, copy in the wilc3000-fw and ultra96-misc directories from my project repository.

With the hardware properly configured and the BSP recipe updated, the kernel needs to have some of the PetaLinux modules enabled, as well as some custom modules added for the WILC3000 and user LED brightness control.

To enable the appropriate PetaLinux kernel modules, run the 'petalinux-config' command again with the following option:

petalinux-config -c kernel

Enable the following modules (to find where these options are in the menu, type '/' and enter the name of the module in the search box that appears):

CONFIG_CMA_SIZE_MBYTES=1024
CONFIG_EDAC_CORTEX_ARM64=y
CONFIG_XILINX_APF=y
CONFIG_XILINX_DMA_APF=y

When you exit out of the kernel configuration editor, build the project ('petalinux-build') and the 'recipes-kernel' directory will appear in the meta-user layer (full directory path: <PetaLinux project directory>/project-spec/meta-user/recipes-kernel). Once this directory is present, a patch file and Ultra96 BSP configuration file need to be added to it.

Under <PetaLinux project directory>/project-spec/meta-user/recipes-kernel/linux/linux-xlnx copy over the bsp.cfg and fix_u96v2_pwrseq_simple.patch from my project repository (respectively linked). Then add the following lines to the linux-xlnx_%.bbappend file one directory level up.

SRC_URI += "file://bsp.cfg"
SRC_URI_append = " file://fix_u96v2_pwrseq_simple.patch"

The patch file is necessary for the proper power up sequence of the ATWILC3000 and the BSP configuration adds the rest of the PetaLinux kernel modules needed (yes, all could have been enabled by hand in the previous step, but this little hack saves some time).

The kernel now needs two custom modules created and enabled in the overall kernel image. To create a custom kernel module in PetaLinux, run the following command:

petalinux-create -t modules --name <MODULE NAME> --enable

Use this command twice to create two modules: one named 'led-brightness' and another named 'wilc'. After running these commands, the source file directories for them will appear under the <PetaLinux project directory>/project-spec/meta-user/recipes-modules directory.

For the led-brightness module, only the main source file (<PetaLinux project directory>/project-spec/meta-user/recipes-modules/led-brightness/files/led-brightness.c) needs to be edited (code here).

The wilc module doesn't need any source code modification, as the bitbake file for it (<PetaLinux project directory>/project-spec/meta-user/recipes-modules/wilc/wilc.bb) is simply being modified to point to the Avnet repository for the WILC3000 so that it will pull those source files and build the kernel module. The bitbake file in my project (linked here) uses the direct source URI to Avnet's repository for the WILC3000 driver to pull from. This means when I build the PetaLinux project, it will need an internet connection or the build will fail. If you want to work offline, clone Avnet's repository to your local drive then edit the source URI in wilc.bb to point to that.

After modifying the files for each custom kernel module, build each module using the 'petalinux-build' command using the following options:

petalinux-build -c <MODULE NAME>

Like the kernel for the Ultra96, the root filesystem also needs both PetaLinux packages enabled in it and some custom applications added.

To enable the PetaLinux packages for the root filesystem, run the petalinux-create command with the following options:

petalinux-config -c rootfs

For the list of which filesystem packages to enable, refer to the list in my project repository here. Again you can type '/' in the root filesystem configuration editor to search for the appropriate menu pathways.

Only one custom application is needed for this Ultra96 project, which is actually a custom library to add the SDS library (Simply Dynamic Strings, read more here if interested) to the project. In PetaLinux, custom user applications and custom user libraries for the root filesystem are created in the same way using the following command:

petalinux-create -t apps --template c --name <NAME> --enable

Name the library 'sds-lib'. Add the library files (libsds_lib.so and libsds_lib_dbg.so linked here) to the <PetaLinux project directory>/project-spec/meta-user/recipes-apps/sds-lib/files directory and edit the bitbake file (sds-lib.bb) one directory above to add these library files to the source URI (see my sds-lib.bb here).

I believe the WILC driver from Avnet uses it for the memory allocation of the messages being passed back and forth to/from the ATWILC3000.

Same as with each custom kernel module, build each custom application/library after modifying the source files, using the 'petalinux-build' command using the following options:

petalinux-build -c <APP/LIBRARY NAME>

Three more recipes (recipes-core, recipes-graphics, and recipes-utils) for the access point webpage application and example projects within it are the final directories that need to be copied into the PetaLinux project directory at <PetaLinux project directory>/project-spec/meta-user/.

To make the PetaLinux project aware of all of the custom applications/packages for the root filesystem added by these three recipes, the user-rootfsconfig file (located in <PetaLinux project directory>/project-spec/meta-user/conf) needs the following lines added to it:

#Note: Mention Each package in individual line
#These packages will get added into rootfs menu entry
CONFIG_gpio-demo
CONFIG_peekpoke
CONFIG_sds-lib
CONFIG_led-brightness
CONFIG_wilc
CONFIG_packagegroup-base-extended
CONFIG_cmake
CONFIG_lmsensors-sensorsdetect
CONFIG_python-pyserial
CONFIG_libftdi
CONFIG_python3-pip
CONFIG_iperf3
CONFIG_packagegroup-petalinux-ultra96-webapp
CONFIG_packagegroup-petalinux-v4lutils
CONFIG_packagegroup-petalinux-96boards-sensors
CONFIG_packagegroup-petalinux-x11
CONFIG_packagegroup-petalinux-matchbox
CONFIG_ultra96-ap-setup
CONFIG_nano
CONFIG_iperf3
CONFIG_bonnie++
CONFIG_ultra96-misc

# Bluetooth stack
CONFIG_bluez5

# Wi-Fi driver firmware and utilities
CONFIG_iw
CONFIG_wpa-supplicant
CONFIG_wilc3000-fw
CONFIG_wilc
CONFIG_ultra96-wpa
CONFIG_ultra96-radio-leds
CONFIG_sds-lib
CONFIG_led-brightness
CONFIG_wilc

The the petalinuxbsp.conf file in the same directoy needs the following lines added to it:

MACHINE_FEATURES_remove_ultra96-zynqmp = "mipi"
DISTRO_FEATURES_append = " bluez5 dbus"
PREFERRED_VERSION_wilc-firmware = "15.2"
IMAGE_INSTALL_remove = "ultra96-ap-setup"

Finally, build the project as a whole:

petalinux-build

This will take quite a while, and in the mean time the SD card itself needs to be prepped.

The partition size requirements have changed just slightly with PetaLinux version 2019.2. Two overall partitions are still required, the first being FAT32 with 4MB of free space ahead of it and the second being EXT4 taking up the rest of the space on the SD card. The FAT32 partition originally only had a minimum size requirement of 60MB before, but now UG1144 now states this partition should be 500MB minimum. I still like to use a GUI application to partition my SD cards, gparted is my preference.

Once the build has completed, a boot image binary (BOOT.BIN) containing the first stage bootloader, FPGA bitstream, PMU firmware, and u-boot environment needs to be generated. Using the 'petalinux-package' command, specify the paths to the bitstream and ELF files for everything located in <PetaLinux project directory>/images/linux. The boot image binary file will be generated in this same directory.

petalinux-package --boot --fsbl ./images/linux/zynqmp_fsbl.elf --fpga ./images/linux/system.bit --pmufw ./images/linux/pmufw.elf --u-boot

Mount the SD card to the host system (your PC) and copy the boot image binary (BOOT.BIN), kernel image (image.ub), and device tree blob (system.dtb) onto the FAT32 partition from <PetaLinux project directory>/images/linux. Then use the tar command to extract the root file system (rootfs.tar.gz from the same <PetaLinux project directory>/images/linux directory) onto the EXT4 partition of the SD card. I've covered these steps in great detail several times in past posts if you need any further reference.

The Ultra96 boot mode is controlled by the DIP switch configuration of SW3. To put the Ultra96 into SD boot mode, SW3 switch one needs to be in the off position while SW3 switch two is in the on position.

Install the SD card in the slot on the Ultra96 configure SW3 for SD boot mode, and plug in the 12V wall adapter power supply. Connect to the UART of the Ultra96 with a serial console application and press the power button (SW4).

Like in my past Ultra96 blog I still highly recommend the U96 JTAG/UART to USB adapter, otherwise individual jumper wires are necessary to connect to the UART and JTAG pins on the Ultra96 board.

The boot sequence takes a little bit of time, and you might see some Matchbox warnings which can safely be ignored at this point since there is no display connected.

Running the ifconfig command from the Ultra96's console is a good sanity check that the wireless interfaces for the Wi-Fi (wlan0) has indeed successfully initialized. At this point, the Ultra96 should appear as an available wireless network to connect to.

Use a browser to connect to the Ultra96 by typing its default local address (192.168.2.1) into the browser's address field. This will bring up the Ultra96's Access Point webpage.

Under the example projects, the user LEDs on the the board can be controlled over the wireless connection.

To connect the Ultra96 to an external Wi-Fi network, use the Wi-Fi Setup option under the Board Configuration tab. Note, that once you connect the Ultra96 to an external Wi-Fi network this Access Point webpage will stop working at the default local IP address 192.168.2.1. Run ifconfig again from the Ultra96's console to see what new IP address has been assigned to it from the wireless router it's now connected to. Connect your PC to the same network that you connected the Ultra96 and you can get back to the Access Point webpage by typing in the Ultra96's new IP address into your browser's address bar.

As I've linked several times throughout this post, all of the source files from this Vivado+PetaLinux build can be found in this project's GitHub repository.