MiniZed Linux Tutorial Part I

Getting Started

This should be obvious but the first page you should go is Avnet MiniZed. There you should grab the BSP. Notice the BSP has different version depending on Vivado release.

This tutorial part I will cover the usage of Xilinx tools to build a Linux system and also the integration of changes on the PL such as a new peripheral UART to use in Linux userspace.

Please refer to Vivado and petalinux installation guide to setup the environment. I am using 2017.2 editions since I have been experiencing a few issues with 2017.4

This guide assume you have a working MiniZed with Linux and WiF connectivity, which will be used to transfer files using scp utility.

Create Linux Project from BSP

Create the petalinux project by open a terminal and typing

petalinux-create -t project -n minized_hackster -s ~/Documents/MiniZed/BSP/2017.2/minized.bsp

This will create the project, you can then build using petalinux-build, but since we are going to customize PL we need to do a few steps before. Open vivado project located in hardware/MINIZED directory local to our minized_hackster project.

For this example I have added a couple of AXI_UART16550 IP's and one additional AXI_GPIO such as

For simplicity I have added the constraints to already existing file minized_LED.xdc (see attachment). The UART uses the PMOD connectors and the I/O is on arduino type connector.

Generate bitstream and export hardware.

Once you have exported hardware description file, by default the local project will use minized_petalinux.sdk folder, so in the root of petalinux project apply the new configuration by doing.

petalinux-config --get-hw-description=/<PATH

This will include any modification made to PL in our project, for example you can confirm our UART lite IP is included by navigating in displayed menu as shown below.

Go inside

Subsystem AUTO Hardware Settings -> Serial Settings -> Primary stdin/stdout

Under Primary stdin/stdout menu you should see our newly added uartlite axi.

I found the hard way that after a change in PL you need to replace image.ub and boot.bin from emmc.

The first step is to build using petalinux-build. After that you should have a directory under project root directory with similar content as below.

To create boot.bin I have taken the approach to use a boot.bif file and bootgen utility, which should be available after you source vivado settings64.sh.

But before we need to create the First stage bootloader in the form of a executable cpu code image or ELF file, using the Hardware Software Interface or HSI we can achieve it. First create a folder to hold the elf project, for example fsbl_project under project root folder.

Enter the hsi prompt by typing in bash hsi, you will enter hsi command prompt from there you proceed with the following commands

hsi% set hwdsgn [open_hw_design /<PATH
hsi% generate_app -hw $hwdsgn -os standalone -proc ps7_cortexa9_0 -app zynq_fsbl -compile -sw fsbl -dir ./fsbl_project/

The first command need the complete path, a shortcut will give you a not found file error.

With the second command you will see some debug output, look for a final line like this

generate_app: Time (s): cpu = 00:00:15 ; elapsed = 00:00:31 . Memory (MB): peak = 446.352 ; gain = 128.000 ; free physical = 120 ; free virtual = 6226

You should have a file called executable.elf inside fsbl_project.

Create a new file boot.bif in project root directory (you might use other of course) with the following contents.

image : {
	[bootloader] fsbl_project/executable.elf
	images/linux/minized_petalinux_wrapper.bit
	images/linux/u-boot.elf
}

With the file in place execute the following command to generate boot image.

bootgen -image boot.bif -o i BOOT.BIN -w on

You should have in project root directory the BOOT.BIN file.

Copy the files to emmc. For this I use scp command but you can use sctp.

scp /<PATH
scp /<PATH

Last but not least you need to copy smallboot.bin to qspi by doing.

flashcp /mnt/emmc/smallboot.bin /dev/mtd0

After this you should have new uartlite or uart16550 available depending on which one you add to PL.

In this example I have connected the uart to arduino pins 1 and 2, as the standard for uart. You can loopback easily with a jumper to test using command line under MiniZed bash console.

A couple of things to take into account for the test.

First disable the echo and onclr features of serial port. If you want to know the reason here is a good one, if you don't do that, terminal will become madness printing forever.

Open to terminals on minized, one will be use for getting messages using

cat /dev/ttyUL1

And send anything from other terminal such

echo 123 > /dev/ttyUL1

One thing with AXI Uart Lite is that you cannot change baudrate from software, for this you need the AXI UART 16550.

Changing the baudrate is done using

stty -F /dev/ttyS0 115200

With the provided bsp, AXI runs at 100MHz clock, AXI_UART_16550 is configured for external clock (similar to BT UART in PL) of 48MHz. This makes possible baudrates up to 230400, going faster is not possible unless AXI clock is increase since there is a limitation of of UART clock that can be up to half the AXI frequency.

Petalinux initramfs limitations.

Very nice to have linux but if you are new to initramfs, sooner you realize the rootfs is not persistent across reboot, which somehow defeat the linux filesystem advantage or make it painful to develop things.

Another problem I found while working with Petalinux was that after you start adding packages to your rootfs, the image size increase. As MiniZed factory /dev/mmcblk1p1 partition is about 128MB in size, I first thought the limitation can be solve by modifying the partition size. It turns out the problem was initramfs in nature (or at least what I believe).

After erasing and recreating partition 1 with ~512MB (later we will see how to set up partitions), I start adding packages to MiniZed image rootfs using

petalinux-config -c rootfs

Selecting them using the GUI interface. After the petalinux-build finish the new image.ub will be available, no problem so far. After copying the image using scp command I found this problem during boot. At the end the way I understand it is that the image.ub don't fit into RAM, the conclusion is based on:

• The image.ub is a compressed image, I notice that after more than ~75M, the image stop booting with aforementioned error.

• After studying a little bit about initramfs, I notice petalinux has 16 block of 16384 kbytes for initramfs, that makes 256MB of space for rootfs image (uncompressed).

Unfortunately I notice this after finding a workaround and so I am not positive that changing the block size solve the issue, but actually there is only 512M of RAM so why bother us with this.

Overcoming the Limitations

I found that working around those limitations was really painful, perhaps my lack of experience on this matter. The solution is to use a second partition on the EMMC for the ROOTFS and remove initramfs, somehow going backward to use initrd.

The solution involves a few steps that I would like to improve but it works.

Partition the EMMC Memory

Remember we need to extend the first partition? We also need to create a ext4 partition for ROOTFS. The first one is straightforward.

Stop MiniZed at u-boot and type

Zynq> run boot_qspi

to run Linux from QSPI.

The reason to boot from QSPI is because we need to unmount the EMMC partitions in order to modify them.

In case you haven't done, it's a good idea to program the Flashback image into QSPI, which will try to run from EMMC but can be use to run from QSPI if something wrong happen. Actually leave your MiniZed there and go to your development machine to build the QSPI Flashback image.

Download the "Minized_QSPI.bsp 2017.2 (QSPI flash fallback for booting from flash to eMMC)" from here, follow the steps in guide "Restore MInized to Factory Status", section "Programming a Fall-back image into QSPI Flash" but with the following modifications.

Before building, do

petalinux-config -c rootfs

Under Filesystem Packages --> base, go into e2fsprogs menu and select all.

This will make mkfs.ext4 available in order to format the rootfs partition after petalinux-build finish.

I tried to following the user guide QSPI programming but for some reason I found no mke2fs binary under /sbin folder when booting again from QSPI. I haven't tried again, but since this should be done only once I found a workaround.

Under your Minized QSPI project root directory, make a temporal directory to mount the already build image.

cd /<YOUR QSPI PROJECT DIR>/images/linux/
mkdir tmp_mnt
gunzip -c rootfs.cpio.gz | sh -c 'cd tmp_mnt/ && cpio -i'

Confirm you have mke2fs binary under /sbin folder in tmp_mnt dir.

After that, connect your Minized running from QSPI to your network and copy the following to it.

scp -p ./images/linux/tmp_mnt/sbin/mke2fs  root@192.168.1.104:/sbin
scp -p ./images/linux/tmp_mnt/sbin/mkfs.ext4  root@192.168.1.104:/sbin
scp -p ./images/linux/tmp_mnt/lib/libext2fs.so.2  root@192.168.1.104:/lib
scp -p ./images/linux/tmp_mnt/lib/libcom_err.so.2  root@192.168.1.104:/lib
scp -p ./images/linux/tmp_mnt/lib/libe2p.so.2  root@192.168.1.104:/lib

Now you can go and partition EMMC and format by doing

umount /dev/mmcblk1p1
fdisk /dev/mmcblkp1
....
Command (m for help): d
Selected partition 1
Command (m for help): n
Command action
  e   extended
  p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-232448, default 1): +4MB
Last cylinder or +size or +sizeM or +sizeK (122-232448, default 232448): +512M
Command (m for help): a
Partition number (1-4): 1
Command (m for help): p
Disk /dev/mmcblk1: 7616 MB, 7616856064 bytes
4 heads, 16 sectors/track, 232448 cylinders
Units = cylinders of 64 * 512 = 32768 bytes
       Device Boot      Start         End      Blocks  Id System
/dev/mmcblk1p1   *         122       15747      500032  83 Linux
Command (m for help): n
Command action
  e   extended
  p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (1-232448, default 1): 15748
Last cylinder or +size or +sizeM or +sizeK (15748-232448, default 232448): Using default value 232448
Command (m for help): p
Disk /dev/mmcblk1: 7616 MB, 7616856064 bytes
4 heads, 16 sectors/track, 232448 cylinders
Units = cylinders of 64 * 512 = 32768 bytes
       Device Boot      Start         End      Blocks  Id System
/dev/mmcblk1p1   *         122       15747      500032  83 Linux
/dev/mmcblk1p2           15748      232448     6934432  83 Linux

End with 'w' to write your changes.

Now format the first partition using FAT and second using EXT4 using

mkfs.vfat -F 32 -n BOOT /dev/mmcblk1p1
mkfs.ext4 -L root /dev/mmcblk1p2

Now you are ready to copy image files. But first we need to configure the project.

You can use your already working project and do the following changes.

petalinux-config

Go into Subsystem AUTO Hardware Settings --> Advance bootable images storage Settings and make sure the following images are set to primary sd

• boot image settings

• kernel image settings

• dtb image settings

Go back to Main Menu and select Image Packaging Configuration.

Make sure Root File System type is set up for SD card and Device Node of SD device is set to /dev/mmcblk1p2

Save the config file and exit. Build using petalinux-build.

After build finish you can generate BOOT.bin using the following command

petalinux-package --boot --fsbl ./images/linux/zynq_fsbl.elf --fpga ./images/linux/minized_petalinux_wrapper.bit --u-boot --force

The output image directory should contain

With MiniZed running from QSPI, connect to your wireless network and copy the files as follows.

scp ./images/linux/BOOT.BIN  root@192.168.1.104:/mnt/emmc
scp ./images/linux/image.ub   root@192.168.1.104:/mnt/emmc
scp ./images/linux/system.dtb    root@192.168.1.104:/mnt/emmc
scp ./images/linux/rootfs.ext4.gz root@192.168.1.104:/mnt/emmc

Your BOOT partition of EMMC should be similar to

Now we need to copy the rootfs to ext4 partition. Do the following to mount the second partition

mkdir /mnt/emmc_rootfs
mount /dev/mmcblk1p2 /mnt/emmc_rootfs

And to mount the rootfs

root@MiniZed:/mnt/emmc# gunzip -c rootfs.ext4.gz > /mnt/emmc_rootfs/rootfs.ext4
root@MiniZed:/mnt/emmc# mkdir /mnt/ext4
root@MiniZed:/mnt/emmc# mount /mnt/emmc_rootfs/rootfs.ext4 /mnt/ext4/ -o loop

We extract first to second partition since image might be bigger in size than the available space in BOOT partition. From there we temporarily mount it and copy the contents using

cp -rf /mnt/ext4/* /mnt/emmc_rootfs/

After that umount the ext4 image and remove files.

root@MiniZed:/mnt/emmc# umount /mnt/ext4/        
root@MiniZed:/mnt/emmc# rm /mnt/emmc_rootfs/rootfs.ext4
root@MiniZed:/mnt/emmc# rm rootfs.ext4.gz

A last step is to replace BOOT.bin in QSPI, which I am not 100% sure if needed but if you modify the PL it might be to get rid of the problem seen before.

root@MiniZed:/mnt/emmc# flashcp BOOT.BIN /dev/mtd0

Reboot and boot into new rootfs. Partitions should look

Now your changes will persist and bigger images can be used.

One step I have thought to allow faster rootfs replacement is to enable NFS server in Kernel and install NFS-utils in rootfs for QSPI image. But I am still figuring out how to do it since the image.ub generated seems to big for /dev/mtd2.

Petalinux 2017.2 Wireless Issue

If you have follow above steps using 2017.2 and you have boot your image from emmc you might have notice that the broadcom kernel driver complain at boot - aka no wireless. The problem is somehow reported here. I believe the issue is because the driver load is taken place before rootfilesystem is mounted. Here is an extract of the error I got during boot

Waiting for root device /dev/mmcblk1p2...
mmc1: new high speed MMC card at address 0001
random: fast init done
mmcblk1: mmc1:0001 Q2J55L 7.09 GiB 
mmcblk1boot0: mmc1:0001 Q2J55L partition 1 16.0 MiB
mmcblk1boot1: mmc1:0001 Q2J55L partition 2 16.0 MiB
mmcblk1rpmb: mmc1:0001 Q2J55L partition 3 4.00 MiB
mmcblk1: p1 p2
mmc0: new high speed SDIO card at address 0001
brcmfmac mmc0:0001:1: Direct firmware load for brcm/brcmfmac43430-sdio.bin failed with error -2
EXT4-fs (mmcblk1p2): couldn't mount as ext3 due to feature incompatibilities
EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Opts: (null)
VFS: Mounted root (ext4 filesystem) on device 179:2.

brcm/brcmfmac43430-sdio.bin is actually in rootfs, and you can see the last line is where actually root is ready. I tried without success to make the wireless driver wait for rootfs, the file minized-firmware_2017.2.bb under./meta-user/recipes-bsp/minized-firmware folder is the layer that load the driver, so I added

DEPENDS += "virtual/kernel"
inherit module-base
addtask make_scripts after do_patch before do_configure
do_make_scripts[lockfiles] = "${TMPDIR}/kernel-scripts.lock"
do_make_scripts[deptask] = "do_populate_sysroot" 

this was not working, perhaps Avnet staff can submit a patch.

I decide to build an image using 2017.4 which makes some changes to wireless firmware to see if the issue is fixed, and it actually works. Now the wireless kernel module loads correctly and if you have leave (you should have) the wpa_supplicant.conf file in first partition of emmc, you can activate wireless using wifi.sh script as normally.

Petalinux 2017.4 Setup

You can follow in general same steps above for EMMC Root Filesystem but for 2017.4 minized.bsp from Avnet. Make sure to select nano for the following steps unless you prefer vim. But continue reading before building.

Adding a Package Manager

This is a must if you want to speed up your development, unless you like the painful petalinux build approach.

I was trying to add opkg apt-get style package manager without sucess until recently came across this, the EXTRA_IMAGE_FEATURES is well known but for some reason adding it to local.conf don't work. Adding as suggested to plnxtool.conf under./build/conf folder make the trick, add/replace the following lines to plnxtool.conf

PACKAGE_CLASSES = "package_rpm package_ipk"
EXTRA_IMAGE_FEATURES += "debug-tweaks package-management"

Make sure you run before petalinux-config -c rootfs and enable opkg package. Enable any other package of your preference and build.

You should have ipk folder under./build/tmp/deply/ folder with the following content. I have notice that if you do a rootfs or kernel configure the above additions to plnxtool.conf are lost, but don't worry keep reading.

Now setup the Host by using nginx as suggested here.

When you are ready, setup minized following same guidelines, make sure the url are setup correctly, i.e. this is my opkg.conf file

src/gz all http://192.168.7.149/all
src/gz cortexa9hf-neon http://192.168.7.149/cortexa9hf-neon
src/gz plnx_arm http://192.168.7.149/plnx_arm
dest root /
option lists_dir /var/lib/opkg/lists

Before we are ready we need to refresh the package index in our Host, actually we better setup yocto to do any further build (yes I am avoiding petalinux).

Set Up Yocto

The procedure is explained better here on page 79, but basically you do

source /<YOUR PETALINUX INSTALL PATH>/components/yocto/source/aarch64/environment-setup-aarch64-xilinx-linux 
source /<YOUR PETALINUX INSTALL PATH>/components/yocto/source/aarch64/layers/core/oe-init-build-env
export PATH=/<YOUR PETALINUX INSTALL PATH>/tools/hsm/bin/:$PATH

Then you execute bitbake to build a package and refresh the index on Host, i.e. if you forgot nano you can

bitbake nano 
bitbake package-index

After that go to MiniZed and do

opkg update
opkg install nano

Changes to Permanent Rootfs from new Petalinux Image.

I found very useful to have NFS during development phase, in my initial configuration I make sure the kernel nfs server was enable and nfs utilities are in rootfs. If you do this, you should have nfsserver under /etc/init.d, i.e.

root@MiniZed:~# ls /etc/init.d/
alignment.sh  dbus-1		     halt	    mountall.sh		  rc			    run-postinsts  syslog	   umountnfs.sh
banner.sh     devpts.sh		     hostapd	    mountnfs.sh		  rcS			    save-rtc.sh    syslog.busybox  urandom
bluetooth     dmesg.sh		     hostname.sh    networking		  read-only-rootfs-hook.sh  sendsigs	   tcf-agent
bootlogd      dropbear		     hwclock.sh     nfscommon		  reboot		    single	   udev
bootmisc.sh   functions		     inetd.busybox  nfsserver		  rmnologin.sh		    stop-bootlogd  udev-cache
checkroot.sh  functions.initscripts  modutils.sh    populate-volatile.sh  rp
cbind		    sysfs.sh	   umountfs

Make a file called exports under /etc such

root@MiniZed:~# cat /etc/exports 
/export       192.168.7.0/24(rw,no_root_squash,fsid=0,insecure,no_subtree_check,async)
/export/users 192.168.7.0/24(rw,no_root_squash,nohide,insecure,no_subtree_check,async)

Change your IP address accordingly. Then execute exportfs.

exportfs -ra

In this case I create a directory called export under rootfs, the only problem is that I need to create a user, one way I found to avoid user permissions when mounting the network share in my development client machine was to match the uid and gid. For example I create a user under minized called "mhanuel", then I check home directory is create for user, that is

adduser mhanuel

Because I cannot login using ssh with my new user, see this issue here, the workaround was to mount the home directory of new user to export/user directory but in order to avoid problems you have to change the ownership, that is

chown -R mhanuel:mhanuel /export/
chown -R mhanuel:mhanuel /export/users

Replace "mhanuel" with your new user username, after that, mount the home directory to the share point

mount --bind /home/mhanuel /export/users/

If you want to mount it upon reboot just add it to fstab

root@MiniZed:~# cat /etc/fstab
# stock fstab - you probably want to override this with a machine specific one
/dev/root            /                    auto       defaults              1  1
proc                 /proc                proc       defaults              0  0
devpts               /dev/pts             devpts     mode=0620,gid=5       0  0
tmpfs                /run                 tmpfs      mode=0755,nodev,nosuid,strictatime 0  0
tmpfs                /var/volatile        tmpfs      defaults              0  0
# uncomment this if your device has a SD/MMC/Transflash slot
#/dev/mmcblk0p1       /media/card          auto       defaults,sync,noauto  0  0
/home/mhanuel    /export/users   none    bind  0  0

Now restart nfsserver service by doing

root@MiniZed:~# /etc/init.d/nfsserver restart

Now you can do on the client machine

sudo mount -v -o rw,nfsvers=2 192.168.7.134:/export/users /<YOUR MINIZED HOME USER FOLDER MOUNT POINT>

As I enable NFS under kernel but not version 3 or 4 as I can remember, the above command works for old NFS server, you might want to play enabling NFS 3 or 4 if needed. You should see a similar output on Host computer.

mount.nfs: timeout set for Fri May  4 10:07:04 2018
mount.nfs: trying text-based options 'nfsvers=2,addr=192.168.7.134'
mount.nfs: prog 100003, trying vers=2, prot=6
mount.nfs: trying 192.168.7.134 prog 100003 vers 2 prot TCP port 2049
mount.nfs: prog 100005, trying vers=1, prot=17
mount.nfs: trying 192.168.7.134 prog 100005 vers 1 prot UDP port 38420 

This save you time, for example I forgot to add the whole python2 modules on my original image, what I did was to add it using petalinux-config -c rootfs, then I mount y image using

sudo mount rootfs.ext4 /<your mount point> -o loop

Thereafter I compress the python module folder at /usr/lib/python2.7 and copy to minized NFS share, from there I did a copy (it's safe in case something wrong) of the original python2.7 folder and decompress the new one such as

root@MiniZed:~# mv /usr/lib/python2.7 /usr/lib/python2.7_bkp
root@MiniZed:~# cd /home/mhanuel
root@MiniZed:/home/mhanuel# ls
python2.7.tar.gz
root@MiniZed:/home/mhanuel# tar xvzf python2.7.tar.gz -C /usr/lib/

MiniZed IoT Ready

Now you have your MiniZed tuned for an IoT project, you have setup an embedded Linux which is friendly and easy to work with but MiniZed is quite new and you cannot find an enclosure for it. If you have access to a 3D printer I have solved that for you too.

The inside of the enclosure can be seen here

The board is somehow difficult to fix into an enclosure by only using the screws, this is because the mounting hole closer to arduino uno header is really close to it, using a 4-40 pan head screw, the screw enter with some inclination, so I took that into account and add a little sleeve next to PMOD connector to support that side of the board as can be seen below.

Because the USB Host connector is not exposed outer in the board, it's difficult to make that corner tied to the Base enclosure, as you see the sleeve and screw at right provide two points of fixation, but the opposite corner tend to pull up a little bit. The first version I didn't notice, and as the lid cover close really hard I think I can add some support material from the cover itself that push the USB Host connector down when inserted.

Here is how it looks from the bottom.

The booton has 4-40 nut holes for easy assembly with screws and some apertures for ventilation.

I will work on a improved version for the lid and giving more clearence to holes, specially the PMOD and arduino apertures, but until them you can use the attached STL files to print yours.

Next Steps

Hope this project help in your next MiniZed adventure. If you like it remember to support me pressing the thumb up button.