Python & PYNQ on the Snickerdoodle!

Introduction

The KRTKL Snickerdoodle is System on Module (SoM) which not only breaks out the IO enabling integration onto a carrier card, but also provides WiFi and Bluetooth connectivity.

To get started developing with the Snickerdoodle we can use the piSmasher breakout board which breaks out the IO from the Snickerdoodle to provide:-

• HDMI In and Out
• Line In, Out and Headphone
• 4xUSB 2.0
• Dual Ethernet - With support for TSN

We can therefore use the piSmahser to implement the application, or get a head start on its development while the application specific carrier card is developed.

To get the application developed as fast as possible we can also leverage the PYNQ framework which enables python application acceleration using the programmable logic within the Zynq device itself.

In this project we are going to create a PYNQ image for the Snickerdoodle such that we can deploy image processing and machine learning overlays.

This is different to the previous projects where we have created PYNQ images as this time we have the WiFi to contend with and a significantly different board architecture which is very different to the Pynq Z1 / Pynq Z2 development boards. The main differences are the Snickerdoodle does not provide Pmod, Arduino or Raspberry Pi headers and has a different networking architecture. This is beside the processor system configuration such as timing, DDR configuration etc.

Of course, these differences result from different use cases the Snickerdoodle is a SoM intended for both prototyping and deployment in the end applications and not just for prototyping and teaching as development boards are.

Vivado Build

When we have previous created PYNQ builds we have used the PYNQ -Z1 base platform as a starting point. As the Snickerdoodle is significantly different to the PYNQ -Z1 this is not a sensible starting point instead we need to create a new base overlay.

To do this we will first need to create the a Vivado application, we can pull down a basic design from the Snickerdoodle github.

However, to ensure it is compatible with the image processing and machine learning overlays we need he project version created in 2018.2. This equates to a PYNQ version of 2.3

To do this using a Git bash enter the following commands

git clone https://github.com/krtkl/snickerdoodle-base.git
cd snickerdoodle-base
git checkout  f6d0e5406f0ea1ec7f94bcda223e53469f470c78

This will clone the directory and then check out the version of the design created in Vivado 2018.2.

We can then open the project and see the base design for the Snickerdoodle in Vivado. This will have the processor system all configured correctly for the Snickerdoodle design.

One of the key things for the piSmasher is to ensure the USB is connected to the USB Hub.

If we examine the piSmasher schematics we see the USB is connected to J3 on the Snickerdoodle which is connected to pins 28 to 39 on the PS MIO.

To be able to use the USB Hub we need to ensure we have enable USB support in the Processing System (PS)

All of the other IO on the piSmasher is connected to the Programmable Logic (PL) as such we can include them or not in overlays to our hearts content.

Once this is completed we need to build the Vivado hardware and export the HDF file with the bitstream

PetaLinux Build

To create a PYNQ image the best way to start is from a PetaLinux BSP for the target board. This means we need to create a Petalinux BSP for the Sncikerdoodle base design, this will be quite involved as we need to ensure the WiFi device will be supported.

Using a virtual machine configured for building a PYNQ image (if you are unsure how to do this see here) we can first create a Petalinux image and then a BSP.

The first thing we need to do is create the project and define the target processor type Zynq, ZynqMP or MicroBlaze.

Once the project has been created, we need to change directory in to the new project and then apply the hardware description we created in Vivado.

This will customize the Petalinux project inline with the Vivado project.

petalinux-create — type project — template zynq — name SD_PYNQ
cd SD_PYNQ 
petalinux-config — get-hw-description 

On completion of the hardware application, a dialog will open which allows us to customize the Linux configuration. We do not want to make any changes to this configuration, so we can exit the dialog (save if requested).

We do however need to be able to configure the kernel to support the WiFi interface.

To customize the kernel enter the following command

petalinux-config -c kernel 

This will open a kernel configuration dialog, where we can enable the WiFi drivers. To support the WiFi we need the following enabled in the kernel

CONFIG_CFG80211=y
CONFIG_CFG80211_DEFAULT_PS=y
CONFIG_CFG80211_CRDA_SUPPORT=y
CONFIG_WAN=y
CONFIG WL18XX=y
CONFIG WLCORE_SDIO=y
CONFIG_WLCORE_SDIO=y
CONFIG_WILINK_PLATFORM_DATA=y
CONFIG_WEXT_CORE=y
CONFIG_WEXT_PROC=y
CONFIG_NL80211_TESTMODE=y
CONFIG_CFG80211_DEVELOPER_WARNINGS=y
CONFIG_CFG80211_CERTIFICATION_ONUS=y
CONFIG_CFG80211_REG_CELLULAR_HINTS=y
CONFIG_CFG80211_REG_RELAX_NO_IR=y
CONFIG_CFG80211_INTERNAL_REGDB=y
CONFIG_CFG80211_WEXT=y
CONFIG_MAC80211=y
CONFIG_MAC80211_HAS_RC=y
CONFIG_MAC80211_RC_MINSTREL=y
CONFIG_MAC80211_RC_MINSTREL_HT=y
CONFIG_MAC80211_RC_DEFAULT_MINSTREL=y
CONFIG_MAC80211_RC_DEFAULT="minstrel_ht"
CONFIG_CRYPTO_AEAD=y
CONFIG_CRYPTO_BLKCIPHER=y
CONFIG_CRYPTO_RNG=y
CONFIG_CRYPTO_RNG_DEFAULT=y
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_GF128MUL=y
CONFIG_CRYPTO_NULL=y
CONFIG_CRYPTO_CCM=y
CONFIG_CRYPTO_GCM=y
CONFIG_CRYPTO_SEQIV=y
CONFIG_CRYPTO_CTR=y
CONFIG_CRYPTO_CMAC=y
CONFIG_CRYPTO_HMAC=y
CONFIG_CRYPTO_GHASH=y
CONFIG_CRYPTO_SHA256=y
CONFIG_CRYPTO_ARC4=y
CONFIG_CRYPTO_BLOWFISH=m
CONFIG_CRYPTO_BLOWFISH_COMMON=m
CONFIG_CRYPTO_DRBG_MENU=y
CONFIG_CRYPTO_DRBG=y
CONFIG_CRYPTO_JITTERENTROPY=y
  

If you are unsure how to find these settings in the kernel configuration dialog you can search using the / function.

Once you have entered the search criteria, if the value is present you will see the Symbol and if it is currently enabled or not. This is indicated by the [=n] or [=y] next too the symbol name, if it is not enabled you can see the dependencies below and if they are enabled or not. Knowing if the dependencies are enabled or not allows us to ensure we know what needs enabling to enable the desired symbol.

Once we have all the symbols enabled, save the kernel configuration and exit the dialog.

We are now nearly in a position to generate the Petalinux BSP, however before we do that I will explain a little about the Petalinux build flow.

Petalinux uses a Yocto build flow, as such any customizations we make are contained within the Project-Spec / Meta-User directory.

The changes we made to the kernel using the dialog, will be stored within the recipes-kernel/linux folder. This is a text file and can be opened with a simple editing program if required.

We also need to make some changes to the device tree to enable the TI WiFi device. We can find an editable device tree under /recipes-bsp/device-tree/files it will be names system-user.dsti

In the system-user.dsti add the following

/include/ "system-conf.dtsi"
/ {
	aliases {
		mmc1 = &sdhci1;
	};
	wlan_en_reg: fixed-regulator-wlen {
		compatible = "regulator-fixed";
		regulator-name = "wlan-en-regulator";
		regulator-min-microvolt = <1800000>;
		regulator-max-microvolt = <1800000>;
		gpio = <&gpio0 9 4>;
		startup-delay-us = <70000>;
		enable-active-high;
	};
};
&sdhci1 {
	status = "okay";
	vmmc-supply = <&wlan_en_reg>;
	bus-width = <4>;
	cap-power-off-card;
	keep-power-in-suspend;
	#address-cells = <1>;
	#size-cells = <0>;	
	wlcore: wlcore@0 {
		reg = <2>;
		interrupt-parent = <&gpio0>;
		interrupts = <0 4>;
	};
};
&wlcore {
	compatible = "wlcore", "ti,wl1837";
};

Once this is complete we are ready to build the Petalinux image using the following commands

petalinux-build 

If desired we can test the Petalinux image using the FPGA configuration and the Petalinux Kernel.

If we are connected we can download the images using the commands

petalinux-boot --jtag --hw_server-url TCP:<ip addr>:3121 --fpga
petalinux-boot --jtag --hw_server-url TCP:<ip addr>:3121 --kernel

This will take some time however, once it is built we can then package the project and create the BSP using the following commands. We can run this command from within the <project>/images/linux/ directory

petalinux-package --boot --fsbl zynq_fsbl.elf --u-boot u-booot.elf --fpga system.bit

This will create a bootable image called boot.bin

To create the BSP we need to exit the project directory and from the petalinux directory run the following command

petalinux-package --bsp -p zybo -o sd_pynq.bsp

PYNQ Build

With the Petalinux BSP created we are now ready to create a PYNQ image for the Snickerdoodle.

The first step in this is to clone from the Xilinx PYNQ GitHub repository the PYNQ files, this will provide all we need to build new PYNQ images.

git clone https://github.com/Xilinx/PYNQ.git

Once we have the PYNQ files downloaded we need to create a new board definition for the snicker doodle.

This board definition will contain the petalinux BSP, board specification file and the base hardware definition.

It will reside under the <clone directory>/pynq/boards

For this application I called the directory sd inside it will be the Petalinux BSP and the specification file. I have also added in base definition and the getting started notebook.

The final thing we need to define is the base.bit file

Within the sd.spec file I entered the following

ARCH_sd := arm
BSP_sd := sd_pynq.bsp
BITSTREAM_sd := base/base.bit
STAGE4_PACKAGES_sd := pynq ethernet wpa_ap wifi_connect

This spec include a number of stage 4 packages which will support the WiFi interface.

However before can build the PYNQ image we need to make an update the PYNQ repository we cloned earlier. IF we fail to do this the WiFi will not work in the directory

<pynq clone>/sdbuild/boot/meta-pynq/recipes-kernel/linux

Within the file linux-xlnx_%.bbappend ensure the wifi.cfg file is removed, the resultant file should be

SRC_URI += " file://pynq.cfg"
SRC_URI += " file://greengrass.cfg"
SRC_URI += " file://zynqmp_fix.patch"
FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"

With that we can change directory into the <clone directory>/pynq/sdbuild/scripts here we can ensure the system is ready to generate the PYNQ image by running the script

source setup_host.sh

Once this is complete CD to the sdbulds directory and run the command

make BOARDS=sd

This will take some time, it is at this point I normally get a cup of tea.

Once the build completes we can write the sd-2.3.img to a SD Card and boot the Sncikerdoodle.

Running PYNQ

Connecting to a terminal window and watch the PYNQ image boot, once booted you should see the PYNQ command prompt.

We still have a few things to do before we can connect to the WiFi, in this instance I set it up to connect to my home network.

We need to create two files in the Snickerdoodle file system

/etc/network/interfaces.d/wlan0

/etc/wpa_supplicant.conf

The first file wlan0 is where we define the wireless LAN interface.

auto wlan0
iface wlan0 inet dhcp
  wireless_mode managed
   wireless_essid BTHub6-FM2P
   wpa-driver nl80211
   wpa-conf /etc/wpa_supplicant.conf

In the second file we need to define the access point parameters

ctrl_interface=/run/wpa_supplicant
update_config=1
network={
       key_mgmt=WPA-PSK
       ssid="BTHub6-FM2P"
       psk="<KEY>"
}

Once this is completed we can start the wpa_supplicant using the command

sudo wpa_supplicant -B -i wlan0 -c /etc/wpa_supplicant.conf
  

To check that the WiFi chip is working correctly, we can use the wpa command line interface.

sudo wpa_cli
scan
scan_results

We are now ready to try and connect to the PYNQ Jupyter server, reboot the Snickerdoodle.

When it completes the power cycle we should be connected to the network, run the command

ifconfig

Once we see this, we can open a browser window and navigate to

pynq:9090 

This will open up the jupyter notebooks allowing us to get going with PYNQ and developing for the Snickerdoodle.

To confirm we have internet access, the best way is to ping a website using the terminal

Jupyter Labs

We can also take advantage of the Jupyter lab environment as well with the PYNQ flow by navigating to

pynq:9090/lab

This will open up the jupyter lab environment in the browser, this is more customizable than the standard notebook interface and we can spin up notebooks, terminals and python shells.

Wrap Up

We are now in a position to start working with PYNQ overlays such as the BNN and image processing. Alternatively, we can create custom overlays for our own applications it is dead simple and easy to get going and creating them.

What are you going to do with your Snickerdoodle and piSmahser?

See previous projects here.

Additional Information on Xilinx FPGA / SoC Development can be found weekly on MicroZed Chronicles.