Getting Started with the Kria KV260 AI Vision Kit

Xilinx recently released the first development board for their all-new SoM (system on module) chipset family, the Kria KV260 Vision AI Starter kit. It is specifically designed and built for development of advanced vision artificial intelligence applications that require hardware acceleration but does not require a developer/engineer to posses the complex hardware design knowledge to implement hardware acceleration from scratch.

I normally headed straight to Vivado with any new Xilinx development board I get. However, the Kria KV260 development board came out of its box and hit the ground running such that I didn't even get to open Vivado before I had a USB webcam working and streaming video back to my host PC with a face detection AI application successfully identifying any faces that came into frame. (To fully appreciate this, see how much work it took me to get a web cam working with the MicroZed in a past project here).

The Kria KV260 development starter kit ships with the Kria SoM mounted on a baseboard with peripherals including USB 3.0, Ethernet, HDMI, PMOD, etc. It requires a 12V barrel jack power supply. If you don't already have one, it can be bought alongside the purchase of the starter kit here. A USB or MIPI camera is needed as well, along with a micro-USB cable, Ethernet cable, and microSD card.

An HDMI cable is recommended but not necessary for the basic Smart Camera application as streaming the video feed over wired Ethernet via real-time streaming protocol is an option. Xilinx also has all of these items bundled together in an optional accessory pack to purchase with the Kria if you don't already have everything.

Load SD card with KV260 Start Kit Image

On the Getting Started portal for the Kria KV260, there is a link to download the embedded Linux image for the KV260 specifically. I say specifically to the KV260 board because the Kria SoM itself can live on other baseboards, but the KV260 is the combination of the Kria SoM with this particular baseboard. Since the large majority of the peripherals that the device tree in Linux is expecting to see live on this baseboard, the Linux image would not work properly on the Kria with a different baseboard.

Download the Linux image and use an imaging tool such as balenaEtcher to program it onto a class 10 SD card at least 16GB in size. The SD card I received in the basic accessory pack for the Kria was blank, for those to note that also purchase the basic accessory pack.

Connect Everything

After imaging the SD card, install it in the SD card slot located on the topside of the baseboard. Connect the micro-USB cable, Ethernet cable, and camera. I'm using a Logitech Brio webcam, so I connected it to one of the four USB 3.0 ports on the KV260.

There are two options to view the video stream from the camera connected to the KV260: on an external monitor via an HDMI cable, or on a host PC using a media player capable of handling an RTSP feed such as VLC or ffplay over the Ethernet cable. I personally chose to go the RTSP feed via Ethernet route because my monitor is USB C and I didn't want to deal with potential conversion issues with an adapter since this is still an embedded Linux image driving it.

Connect the micro-USB to the host PC and the Ethernet to a router/modem, but leave the barrel-jack power cable disconnected for the moment.

Open COM Port & Power Up Board

After connecting the micro-USB cable to the host PC, at least two enumerated serial ports will appear. The lower numbered COM port is the one associated with the UART of the KV260 to connect a serial terminal of choice to (such as TeraTerm, Putty, Serial, etc.).

I personally had four COM ports appears on my computer (I was running on the Mac OS side of my machine) and the second lowest numbered one ended up being the KV260's UART.

Open COM port with serial terminal of choice with a baud rate of 115200 and power on the KV260 by connecting the power supply to the barrel jack.

The first time the KV260's Linux image is booted, it will require you to set your own password. The username is already set to petalinux.

Connect to the Internet

One of the other exciting things that came along with the launch of the Kria SoM, is the launch of the first Embedded Application Store by Xilinx. The Smart Camera accelerated application to drive the USB webcam with a facial recognition algorithm is available in Xilinx's Embedded App Store for free, and the KV260 needs to be connected to the internet in order to download it.

The easiest way to connect the KV260 to the internet is plug the ethernet cable into a router/modem and the Linux image will use DHCP to assign itself an IP address.

Verify internet connection with a simple ping to google.com

root@xilinx-k26-starterkit-2020_2:~# ping google.com

Download Smart Camera App from Xilinx Embedded App Store

Xilinx has also developed their own platform management utility, xmutil, to handle the accelerated application packages/files. So most anything from downloading new apps from the Embedded App Store to switching them out in the Linux image is done so with the xmutil command.

Check the list of available accelerated apps in the Xilinx embedded App Store (only the free ones show up, paid ones have to be downloaded directly).

root@xilinx-k26-starterkit-2020_2:~# sudo xmutil getpkgs

Currently, there are three free accelerated AI vision apps available in the Embedded App Store. Download and install the Smart Camera accelerated application using the dnf command. The KV260's Linux image is configured such that its mirrors are already pointed to the Embedded App Store.

root@xilinx-k26-starterkit-2020_2:~# sudo dnf install packagegroup-kv260-smartcam.noarch

This part might take several minutes depending on your internet speed.

Load Smart Camera Accelerated Application

While multiple apps can be installed, only one accelerated application can run on the KV260 at a time. This is where the xmutil comes in to handle things such as checking which app is currently loaded to be ran and switching them out.

root@xilinx-k26-starterkit-2020_2:~# sudo xmutil listapps

The listapps option displays all of the apps currently installed on the Kria and shows which one is currently active (ie which one is currently loaded).

Unload the default kv260-dp app using the unloadapp option:

root@xilinx-k26-starterkit-2020_2:~# sudo xmutil unloadapp

And load the Smart Camera app using the loadapp option:

root@xilinx-k26-starterkit-2020_2:~# sudo xmutil loadapp kv260-smartcam

Once the line saying the Smart Camera application has loaded successfully appears, hit enter to return the command line back.

Run Smart Camera Accelerated Application

Each accelerated app will have its own set of commands to control it that will become available one the app is installed and loaded. The Smart Camera app is controlled with the smartcam command.

Run the Smart Camera App targeting the wired Ethernet connection and specifying the USB device number of the webcam (use the --mipi flag instead of the --usb flag if using a camera via the MIPI connector).

root@xilinx-k26-starterkit-2020_2:~# sudo smartcam --usb 0 -W 1920 -H 1080 -r 30 --target rtsp

The rtsp option passed after the --target flag tells the Smart Camera app to send the video feed over the Ethernet cable, if you're using an external monitor with an HDMI cable, pass dp and the option to target instead.

root@xilinx-k26-starterkit-2020_2:~# sudo smartcam --usb 0 -W 1920 -H 1080 -r 30 --target dp

The first thing the Smart Camera app will echo back over the UART when using Ethernet to stream the video feed is the IP address, port number, and channel name of the RTSP feed.

Setup VLC to View Video Stream

To view the RTSP feed, I personally chose to use VLC player. Set up VLC player to view stream by going to File > Open Network... Check the box for Stream output: and click the Settings... button to the right of it.

Configure the feed to stream and display the stream locally (which will make the video feed window automatically pop up when playing the stream). Use the IP address, port number, and channel name output to the UART from the Smart Camera app to configure the stream in VLC as show below:

View video stream by selecting the stream from the playlist and clicking the play button. Sometimes I did have to click play a few times before the video feed window popped up, but once it did, it was immediately recognizing any faces in frame.

Next Steps

I'm super excited for how easy it was to get the KV260 up and running with a camera and AI vision application running. I'm currently planning to revisit my coffee shop digital menu with live webcam project using the KV260. I want to see how I can improve that project by using object detection to recognize when certain pastries have run out and remove them from the digital menu linked to the QR code.

See me walk through this live in my unboxing video here: