Hackster's FPGAdventures: Split Personalities with AMP on the Microchip PolarFire SoC Icicle Kit

Microchip's PolarFire SoC family of chips has something of a split-personality: there's the field-programmable gate array (FPGA) side of the chips, and there's the RISC-V processing side, which turns it into a fully-fledged system-on-chip.

While both halves work in tandem, giving you to ability to run an operating system or bare-metal program on the same chip you've programmed for a task-specific workload, there's another way to split the resources: asymmetric multiprocessing (AMP), which carves out chunks of the chip to turn it into two distinct but interconnected devices — for some use cases effectively cutting the bill of materials in half.

It's this peculiar feature of the PolarFire SoC range we're investigating this month, as we continue our deep dive into the PolarFire SoC Icicle Kit development board and its multifarious capabilities.

Split personalities

In normal use, a PolarFire SoC chip is made up of two primary blocks: the FPGA block and the CPU block, the latter of which is further subdivided into four SiFive U54 RISC-V application-class central processing cores and a fifth E51 real-time core — physically present but actively used for the Hart Software Services (HSS), which keep the device ticking over and handle tying all the resources together, and thus not available for end user workloads.

When the PolarFire SoC boots a workload — whether that's a bare-metal application or an operating system like the Yocto Linux-based distribution, which comes pre-loaded onto the Icicle Kit's on-board storage — it does so in symmetric multiprocessing (SMP) mode. In SMP mode, all four RISC-V CPU cores are exposed to the operating system: looking at the output of lscpu, the contents of /dev/cpuinfo, or a system monitor like htop or top will show you four cores all of which are available to the scheduler, unless instructed otherwise.

For the majority of workloads, SMP is exactly what you want: it provides the highest compute performance, ensuring that you can run an operating system and feed data to whatever gateware you're running on the FPGA at the fastest rate possible. It's certainly the mode you need if you're focusing primarily on the RISC-V side of the Icicle Kit, as it ensures you have the maximum possible resources available to you.

SMP is the default mode, but not the only mode. Its opposite is asymmetric multiprocessing (AMP): where SMP mode sees all four user-accessible RISC-V cores configured the same, sharing resources, and working on the same general tasks, AMP creates a divide. It's this divide that effectively enables a single PolarFire SoC chip to replace not only separate FPGA and CPU parts but also multiple FPGA and CPU components in a single design — a key part of the family's flexibility.

Building AMP

A PolarFire SoC running in AMP mode — whether on the Icicle Kit development board or in a custom design — is configured as two partially-independent devices. At the time the firmware image is built, you can choose exactly how: Microchip's own samples for the Icicle Kit see the PolarFire SoC split unevenly, with three of the SiFive U54 CPU cores dedicated to running a minimalist Linux build and the fourth core split off to run either a bare-metal application or a real-time operating system (RTOS).

Seeing the two halves of the system communicate, using the Hart Software Services and its shared memory as an intermediary, is one thing — but here, as with the rest of the software on the Icicle Kit, Microchip has your back: the demos are provided as open source software, providing a route to pick their operation apart and integrate the same functionality into your own workloads., allowing for an easy and high-performance interlink for split-to-split communication. Peripherals, too, can be assigned to either of the contexts.

Getting started with AMP mode is relatively straightforward, providing you've already set up a machine with a Yocto build environment with Microchip's PolarFire SoC board support package (BSP). If not, you'll need to spend a little time following Microchip's guidance on that subject — a thankfully reasonably straightforward task, though one which requires a little tweaking if you've moved on from the Ubuntu 18.04 host system for which Microchip's instructions were written.

Ordinarily, the build environment and BSP are used to create an up-to-date Linux image for the Icicle Kit running in SMP mode — either the feature-packed mpfs-dev-cli variant or the slimmed-down core-image-minimal-dev version, or even the tiny mpfs-initramfs-image build, which is small enough to operate entirely from the Icicle Kit's RAM.

It's the icicle-kit-es-amp build target that we're interested in this month, though. It's the only one that doesn't have to build Linux, but instead offers a range of choices for what to run on the split-personality PolarFire SoC when running in AMP mode: Linux and a real-time operating system; Linux and a bare-metal application; a real-time operating system with a bare-metal application; two real-time operating systems working independently; and two bare-metal applications working independently. The only thing it can't do: run two separate Linux instances.

Regardless of what you choose, building the image can take a while — up to several hours, depending on the resources available on your build machine and whether you've previously built other images from the same BSP release. Once built, the resulting image needs to be flashed onto the Icicle Kit's on-board eMMC — a relatively quick process, especially if you follow Microchip's advice and use the bmaptool rather than dd or a graphical flashing utility like Balena Etcher. If it's been a while since you last updated your Icicle Kit, though, you'll also need to update the Hart Support Services using FlashPro Express, as demonstrated in FPGAdventures #4.

True flexibility

Assuming you opted for the freertos build, the default if no override is selected, booting the Icicle Kit with the AMP image installed will at first seem immediately familiar: Connect to the first of the four USB UART buses, which appear when you plug a micro-USB cable into the correct port on the Icicle Kit and you'll be greeted with Hart Software Services' boot sequence; switch to the second UART bus and you'll see a Linux login prompt.

Log in, though, and you'll immediately see the difference: Type lscpu or load your favorite system monitorand you'll see the PolarFire SoC has only three cores available, instead of the usual four. It has little actual impact on your ability to use Linux on the Icicle Kit: Everything still works as normal, though multi-threaded or multi-worker workloads will run slower than when all four CPU cores were available.

To see what the fourth CPU core is doing, simply switch to the third UART bus. This bus is linked to the second context you created when you built the image — running a build of the FreeRTOS real-time operating system by default, or Microchip's sample bare-metal application if you chose the bm build target at compile time.

Demo time

Regardless of which image you decided to build, you'll be granted access to a selection of sample applications built to showcase the Remote Processor Messaging (RPMsg) framework — a means for the two halves of the AMP-mode PolarFire SoC to communicate with each other, despite being effectively two distinct and separate logical devices.

Speaking technically, the demos built for the real-time or bare-metal applications running on the fourth CPU core are based on the lighter-weight RPMsg-lite; the Linux context, meanwhile, uses full-fat RPMsg.

RPMsg works on a three-layer communications stack: the transport layer, which is RPMsg itself; the media access control (MAC) layer, which is the VirtIO input/output virtualization framework; and the physical layer, a combination of shared memory and inter-core interrupts, which allow one context to write data to memory then signal to the other that there's a message waiting to be read.

The demos are made available using a simple text-based menu — identical to the ones used in the Linux sample applications we looked at in FPGAdventures #5. Simply pick a number to run the demo — then switch back to the Linux context, or the bare-metal or RTOS context if you built one of the non-Linux image targets, and run the same demo.

Seeing the two halves of the system communicate, using the Hart Software Services and its shared memory as an intermediary, is one thing — but here, as with the rest of the software on the Icicle Kit, Microchip has your back: the demos are provided as open source software, providing a route to pick their operation apart and integrate the same functionality into your own workloads.

Like its Linux-centric demos, though, the comments aren't always as clear or as numerous as you might like — but at least there's supporting documentation on the relevant GitHub repositories to help you on your way.

Join us next time when we'll be looking deeper at the PolarFire SoC's FPGA resources.

More information on the PolarFire SoC Icicle Kit is available on the Microchip website and can be found on Avnet's product page as well.

Read the whole series:

• FPGAdventures Series #1: Unboxing the PolarFire SoC Icicle
• FPGAdventures Series #2: Installing Microchip's Libero SoC
• FPGAdventures Series #3: First Steps with Libero SoC
• FPGAdventures Series #4: Linux on RISC-V
• FPGAdventures Series #5: Linux Code Samples