AC701, HDMI & Test Pattern Generation

Introduction

Over the years we have done a lot of image processing projects using AMD-Xilinx devices and development boards for example the Zynq, Zynq MPSOC, along with 7 Series FPGA.

Typically we have created these applications using development board such as the Zynq Z1/Z2, Ultra 96V2, Arty Z7 / A7 / S7.

While there is commonality between SoC and FPGA based image processing systems. It is good to have reference platforms / designs for both approaches.

For FPGA based imaging we recently looked at the SP701 for more complex image processing as the S7-100 offers increased logic resources. However, the AC701 offers increased logic resources again, making a better reference platform.

As such I want to create a FPGA Imaging demonstration platform for image processing based on the AC701. The AC701 provides an HDMI output and has a FMC interface onto which we can add in the cameras and sensor interfaces. The large logic resources and DDR will make for a good platform for pure FPGA image based solutions.

To get started creating this platform we need to be able to drive the ADC7511 HDMI device on the AC701 board. This is a highly configurable HDMI transmitter which takes a parallel video stream and syncs and encodes a HDMI stream.

Of course, it is highly configurable device and needs the FPGA working with it to configure it correctly over an I2C link. It also requires the video signals are correctly aligned for the mode of desired operation.

Both the SP701 and the AC701 use the same ADV7511 in the same configuration.

Hardware Connections

The AC701 has the ADV7511 connected for YUV 4:4:4 it is connected for a 24 bit pixel with external syncs and a clock

This means there are unused pixels in the 36 bit data bus.

The ADV7511 is configured by a I2C, on the AC701 board the I2C busses are connected to a bus switch therefore before we can communicate with the HDMI device or indeed later the camera we must set the switch appropriately.

Vivado Design

To test the HDMI output is working correctly we are going to create a simple design which can be expanded. In this initial design we will have the following elements

• MicroBlaze
• UART - For communication
• Test Pattern Generator
• Video Timing Controller
• AXI Stream to Video Out
• Video Processing Sub System
• Memory Interface Generator

The complete design can be seen below

The configuration of each block is as follows

Video Test Pattern Generator

VPSS

AXI Stream to Video out

Video Timing Controller

Clocking wise we have two clocks a MicroBlaze / AXI lite clock at 100 MHz, this is generated by the MIG

150 MHz stream / pixel clock

The MIG itself is clocked at 200 MHz and uses a common clock for both Sys and Ref Clock

To prevent implementation warnings about the Clock Wizard location being un optimal for the location of the clock input. I use a differential input buffer and global clock buffer outside of the clock wizard.

The MicroBlaze has been configured to provide AXI Data and Instruction interfaces such that the application can run from DDR.

The constraints for the project can be seen below

set_property PACKAGE_PIN V22 [get_ports HDMI_R_D[7]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_R_D[7]]
set_property PACKAGE_PIN Y23 [get_ports HDMI_R_D[6]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_R_D[6]]
set_property PACKAGE_PIN Y22 [get_ports HDMI_R_D[5]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_R_D[5]]
set_property PACKAGE_PIN AB24 [get_ports HDMI_R_D[4]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_R_D[4]]
set_property PACKAGE_PIN AC24 [get_ports HDMI_R_D[3]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_R_D[3]]
set_property PACKAGE_PIN AB25 [get_ports HDMI_R_D[2]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_R_D[2]]
set_property PACKAGE_PIN AA25 [get_ports HDMI_R_D[1]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_R_D[1]]
set_property PACKAGE_PIN AA23 [get_ports HDMI_R_D[0]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_R_D[0]]


set_property PACKAGE_PIN V23 [get_ports HDMI_G_D[7]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_G_D[7]]
set_property PACKAGE_PIN Y20 [get_ports HDMI_G_D[6]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_G_D[6]]
set_property PACKAGE_PIN U24 [get_ports HDMI_G_D[5]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_G_D[5]]
set_property PACKAGE_PIN W20 [get_ports HDMI_G_D[4]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_G_D[4]]
set_property PACKAGE_PIN W23 [get_ports HDMI_G_D[3]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_G_D[3]]
set_property PACKAGE_PIN U20 [get_ports HDMI_G_D[2]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_G_D[2]]
set_property PACKAGE_PIN V24 [get_ports HDMI_G_D[1]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_G_D[1]]
set_property PACKAGE_PIN U25 [get_ports HDMI_G_D[0]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_G_D[0]]



set_property PACKAGE_PIN U26 [get_ports HDMI_B_D[7]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_B_D[7]]
set_property PACKAGE_PIN W24 [get_ports HDMI_B_D[6]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_B_D[6]]
set_property PACKAGE_PIN W25 [get_ports HDMI_B_D[5]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_B_D[5]]
set_property PACKAGE_PIN W26 [get_ports HDMI_B_D[4]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_B_D[4]]
set_property PACKAGE_PIN V26 [get_ports HDMI_B_D[3]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_B_D[3]]
set_property PACKAGE_PIN Y26 [get_ports HDMI_B_D[2]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_B_D[2]]
set_property PACKAGE_PIN Y25 [get_ports HDMI_B_D[1]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_B_D[1]]
set_property PACKAGE_PIN AA24 [get_ports HDMI_B_D[0]]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_B_D[0]]

set_property PACKAGE_PIN V21 [get_ports HDMI_R_CLK]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_R_CLK]

set_property PACKAGE_PIN AB26 [get_ports HDMI_R_DE]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_R_DE]

set_property PACKAGE_PIN AC26 [get_ports HDMI_R_VSYNC]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_R_VSYNC]

set_property PACKAGE_PIN V21 [get_ports HDMI_R_CLK]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_R_CLK]

set_property PACKAGE_PIN AA22 [get_ports HDMI_R_HSYNC]
set_property IOSTANDARD LVCMOS18 [get_ports HDMI_R_HSYNC]

set_property PACKAGE_PIN N18 [get_ports IIC_MAIN_scl_io]
set_property IOSTANDARD LVCMOS33 [get_ports IIC_MAIN_scl_io]
set_property PACKAGE_PIN K25 [get_ports IIC_MAIN_sda_io]
set_property IOSTANDARD LVCMOS33 [get_ports IIC_MAIN_sda_io]

The project can be built and exported for the software design from here.

Software Design

The software configuration of the ADV7511 is not so complex, we just need to set the I2C switch and the configure the correct registers for that task. But we also need to configure the TPG, VTC, VPSS also to ensure there is video to be passed through.

As such the software algorithm provides the following

• Initialise the peripherals
• Configure the switch for the HDMI channel
• Configure the HDMI device
• Initialize the VPSS
• Configure the Video Timing Generator for 1080p 60 FPS
• Configure the Test Pattern Generator for 1080p
• Start the IP cores

The HDMI write process uses the example given from the MIPI CSI2 SP701 example. To do this a low level IIC driver is used to write in 16 registers into the ADV7511.

Each register needs to have a address written over the IIC Interface followed by the desired data word.

Once we have written the data it is read back to ensure we wrote to the correct data to the device.

The entire application can be seen below

#include <stdio.h>
#include "platform.h"
#include "xil_printf.h"
#include "xstatus.h"
#include "sleep.h"
#include "xiic_l.h"
#include "xil_io.h"
#include "xil_types.h"
#include "xvtc.h"

#include "xv_tpg.h"
#include "xvidc.h"
#include "vga.h"
#include "xvprocss.h"
#define PAGE_SIZE   16


#define IIC_BASE_ADDRESS	XPAR_IIC_0_BASEADDR

#define EEPROM_TEST_START_ADDRESS	0x80

#define IIC_SWITCH_ADDRESS 0x74
#define IIC_ADV7511_ADDRESS 0x39


XVtc	VtcInst;
XVtc_Config *vtc_config ;
XV_tpg tpg;
XV_tpg_Config *tpg_config;
XVprocSs scaler_new_inst;
XVprocSs csc_new_inst;
VideoMode video;


typedef u8 AddressType;

typedef struct {
	u8 addr;
	u8 data;
	u8 init;
} HDMI_REG;

#define NUMBER_OF_HDMI_REGS  16
HDMI_REG hdmi_iic[NUMBER_OF_HDMI_REGS] = {
	{0x41, 0x00, 0x10},
	{0x98, 0x00, 0x03},
	{0x9A, 0x00, 0xE0},
	{0x9C, 0x00, 0x30},
	{0x9D, 0x00, 0x61},
	{0xA2, 0x00, 0xA4},
	{0xA3, 0x00, 0xA4},
	{0xE0, 0x00, 0xD0},
	{0xF9, 0x00, 0x00},
	{0x18, 0x00, 0xE7},
        {0x55, 0x00, 0x00},
        {0x56, 0x00, 0x28},
        {0xD6, 0x00, 0xC0},
        {0xAF, 0x00, 0x4},
	{0xF9, 0x00, 0x00}
};


u8 EepromIicAddr;		/* Variable for storing Eeprom IIC address */

int IicLowLevelDynEeprom();

u8 EepromReadByte(AddressType Address, u8 *BufferPtr, u8 ByteCount);
u8 EepromWriteByte(AddressType Address, u8 *BufferPtr, u8 ByteCount);


//HDMI IIC
int IicLowLevelDynEeprom()
{
  u8 BytesRead;
  u32 StatusReg;
  u8 Index;
  int Status;
  u32 i;
  u8 channel[1] = {0x20};

  Status = XIic_DynInit(IIC_BASE_ADDRESS);
  if (Status != XST_SUCCESS) {
	return XST_FAILURE;
  }
  xil_printf("\r\nAfter XIic_DynInit\r\n");
  while (((StatusReg = XIic_ReadReg(IIC_BASE_ADDRESS,
				XIIC_SR_REG_OFFSET)) &
				(XIIC_SR_RX_FIFO_EMPTY_MASK |
				XIIC_SR_TX_FIFO_EMPTY_MASK |
				XIIC_SR_BUS_BUSY_MASK)) !=
				(XIIC_SR_RX_FIFO_EMPTY_MASK |
				XIIC_SR_TX_FIFO_EMPTY_MASK)) {

  }

  EepromIicAddr = IIC_SWITCH_ADDRESS;
  XIic_DynSend(IIC_BASE_ADDRESS, EepromIicAddr,
		  	  	  channel, sizeof(channel), XIIC_STOP);


  EepromIicAddr = IIC_ADV7511_ADDRESS;
  for ( Index = 0; Index < NUMBER_OF_HDMI_REGS; Index++)
  {
    EepromWriteByte(hdmi_iic[Index].addr, &hdmi_iic[Index].init, 1);
  }

  for ( Index = 0; Index < NUMBER_OF_HDMI_REGS; Index++)
  {
    BytesRead = EepromReadByte(hdmi_iic[Index].addr, &hdmi_iic[Index].data, 1);
    for(i=0;i<1000;i++) {};	// IIC delay
	if (BytesRead != 1) {
      return XST_FAILURE;
	}
  }



  return XST_SUCCESS;

}

u8 EepromReadByte(AddressType Address, u8 *BufferPtr, u8 ByteCount)
{
  u8 ReceivedByteCount;
  u8 SentByteCount;
  u16 StatusReg;

  /*
   * Position the Read pointer to specific location in the EEPROM.
   */
  do {
	StatusReg = XIic_ReadReg(IIC_BASE_ADDRESS, XIIC_SR_REG_OFFSET);
    if (!(StatusReg & XIIC_SR_BUS_BUSY_MASK)) {
	  SentByteCount = XIic_DynSend(IIC_BASE_ADDRESS, EepromIicAddr,
					  (u8 *) &Address, sizeof(Address), XIIC_REPEATED_START);
    }

  } while (SentByteCount != sizeof(Address));
  /*
   * Receive the data.
   */
  ReceivedByteCount = XIic_DynRecv(IIC_BASE_ADDRESS, EepromIicAddr,
		                                          BufferPtr, ByteCount);

  /*
   * Return the number of bytes received from the EEPROM.
   */

  return ReceivedByteCount;

}


u8 EepromWriteByte(AddressType Address, u8 *BufferPtr, u8 ByteCount)
{
  u8 SentByteCount;
  u8 WriteBuffer[sizeof(Address) + PAGE_SIZE];
  u8 Index;

  /*
   * A temporary write buffer must be used which contains both the address
   * and the data to be written, put the address in first based upon the
   * size of the address for the EEPROM
   */
  if (sizeof(AddressType) == 2) {
	WriteBuffer[0] = (u8) (Address >> 8);
	WriteBuffer[1] = (u8) (Address);
  } else if (sizeof(AddressType) == 1) {
	WriteBuffer[0] = (u8) (Address);
	EepromIicAddr |= (EEPROM_TEST_START_ADDRESS >> 8) & 0x7;
  }

  /*
   * Put the data in the write buffer following the address.
   */
  for (Index = 0; Index < ByteCount; Index++) {
	WriteBuffer[sizeof(Address) + Index] = BufferPtr[Index];
  }

  /*
   * Write a page of data at the specified address to the EEPROM.
   */
  SentByteCount = XIic_DynSend(IIC_BASE_ADDRESS, EepromIicAddr,
				WriteBuffer, sizeof(Address) + ByteCount,
				XIIC_STOP);

  /*
   * Return the number of bytes written to the EEPROM.
   */
  return SentByteCount - sizeof(Address);

}


void InitVprocSs_CSC() {
  XVprocSs_Config* p_vpss_cfg1;
  int status;
  int widthIn, heightIn, widthOut, heightOut;

  widthOut = 1920;
  heightOut = 1080;

  // Local variables
  XVidC_VideoMode resIdIn, resIdOut;
  XVidC_VideoStream StreamIn, StreamOut;

  widthIn = 1920;
  heightIn = 1080;
  StreamIn.FrameRate = 60; //rao


    p_vpss_cfg1 = XVprocSs_LookupConfig(XPAR_XVPROCSS_0_DEVICE_ID);
	if (p_vpss_cfg1 == NULL) {
	  xil_printf("ERROR! Failed to find VPSS-based scaler.\n\r");
      return;
	}

	status = XVprocSs_CfgInitialize(&csc_new_inst, p_vpss_cfg1,
				p_vpss_cfg1->BaseAddress);
	if (status != XST_SUCCESS) {
	  xil_printf("ERROR! Failed to initialize VPSS-based scaler.\n\r");
	  return;
	}

  XVprocSs_Stop(&csc_new_inst);

  // Get resolution ID from frame size
  resIdIn = XVidC_GetVideoModeId(widthIn, heightIn, StreamIn.FrameRate,
			FALSE);

  // Setup Video Processing Subsystem
  StreamIn.VmId = resIdIn;
  StreamIn.Timing.HActive = widthIn;
  StreamIn.Timing.VActive = heightIn;
  StreamIn.ColorFormatId = XVIDC_CSF_RGB;
  StreamIn.ColorDepth = csc_new_inst.Config.ColorDepth;
  StreamIn.PixPerClk = csc_new_inst.Config.PixPerClock;
  StreamIn.IsInterlaced = 0;

  status = XVprocSs_SetVidStreamIn(&csc_new_inst, &StreamIn);
  if (status != XST_SUCCESS) {
	xil_printf("Unable to set input video stream parameters correctly\r\n");
	return;
  }

  // Get resolution ID from frame size
  resIdOut = XVidC_GetVideoModeId(widthOut, heightOut, 60, FALSE);

  if (resIdOut != XVIDC_VM_1920x1080_60_P) {
	xil_printf("resIdOut %d doesn't match XVIDC_VM_1920x1080_60_P\r\n",
				resIdOut);
  }


  StreamOut.VmId = resIdOut;
  StreamOut.Timing.HActive = widthOut;
  StreamOut.Timing.VActive = heightOut;
  StreamOut.ColorFormatId = XVIDC_CSF_YCRCB_444;
  StreamOut.ColorDepth = csc_new_inst.Config.ColorDepth;
  StreamOut.PixPerClk = csc_new_inst.Config.PixPerClock;
  StreamOut.FrameRate = 60;
  StreamOut.IsInterlaced = 0;

  XVprocSs_SetVidStreamOut(&csc_new_inst, &StreamOut);
  if (status != XST_SUCCESS) {
	xil_printf("Unable to set output video stream parameters correctly\r\n");
	return;
  }

  status = XVprocSs_SetSubsystemConfig(&csc_new_inst);
  if (status != XST_SUCCESS) {
    xil_printf("xvprocss_SetSubsystemConfig failed %d\r\n", status);
    return;
  }

  //XVprocSs_ReportSubsystemConfig(&scaler_new_inst);
  XVprocSs_Start(&scaler_new_inst);


}


int main()
{

	XVtc_Timing vtcTiming;
	int Status;
	XVtc_SourceSelect SourceSelect;
    init_platform();

    print("Hello World\n\r");
    print("Successfully ran Hello World application");

    Status = IicLowLevelDynEeprom();
    if (Status != XST_SUCCESS) {
    	xil_printf("ADV7511 IIC programming FAILED\r\n");
    	return XST_FAILURE;
      }
    xil_printf("ADV7511 IIC programming PASSED\r\n");


    vtc_config = XVtc_LookupConfig(XPAR_VTC_0_DEVICE_ID);
    XVtc_CfgInitialize(&VtcInst, vtc_config, vtc_config->BaseAddress);

    video = VMODE_1920x1080 ;
	vtcTiming.HActiveVideo = video.width;	/**< Horizontal Active Video Size */
	vtcTiming.HFrontPorch = video.hps - video.width;	/**< Horizontal Front Porch Size */
	vtcTiming.HSyncWidth = video.hpe - video.hps;		/**< Horizontal Sync Width */
	vtcTiming.HBackPorch = video.hmax - video.hpe + 1;		/**< Horizontal Back Porch Size */
	vtcTiming.HSyncPolarity = video.hpol;	/**< Horizontal Sync Polarity */
	vtcTiming.VActiveVideo = video.height;	/**< Vertical Active Video Size */
	vtcTiming.V0FrontPorch = video.vps - video.height;	/**< Vertical Front Porch Size */
	vtcTiming.V0SyncWidth = video.vpe - video.vps;	/**< Vertical Sync Width */
	vtcTiming.V0BackPorch = video.vmax - video.vpe + 1;;	/**< Horizontal Back Porch Size */
	vtcTiming.V1FrontPorch = video.vps - video.height;	/**< Vertical Front Porch Size */
	vtcTiming.V1SyncWidth = video.vpe - video.vps;	/**< Vertical Sync Width */
	vtcTiming.V1BackPorch = video.vmax - video.vpe + 1;;	/**< Horizontal Back Porch Size */
	vtcTiming.VSyncPolarity = video.vpol;	/**< Vertical Sync Polarity */
	vtcTiming.Interlaced = 0;

    memset((void *)&SourceSelect, 0, sizeof(SourceSelect));
	SourceSelect.VBlankPolSrc = 1;
	SourceSelect.VSyncPolSrc = 1;
	SourceSelect.HBlankPolSrc = 1;
	SourceSelect.HSyncPolSrc = 1;
	SourceSelect.ActiveVideoPolSrc = 1;
	SourceSelect.ActiveChromaPolSrc= 1;
	SourceSelect.VChromaSrc = 1;
	SourceSelect.VActiveSrc = 1;
	SourceSelect.VBackPorchSrc = 1;
	SourceSelect.VSyncSrc = 1;
	SourceSelect.VFrontPorchSrc = 1;
	SourceSelect.VTotalSrc = 1;
	SourceSelect.HActiveSrc = 1;
	SourceSelect.HBackPorchSrc = 1;
	SourceSelect.HSyncSrc = 1;
	SourceSelect.HFrontPorchSrc = 1;
	SourceSelect.HTotalSrc = 1;
	XVtc_RegUpdateEnable(&VtcInst);
	XVtc_SetGeneratorTiming(&VtcInst, &vtcTiming);
	XVtc_SetSource(&VtcInst, &SourceSelect);
	XVtc_EnableGenerator(&VtcInst);

	XVtc_Enable(&VtcInst);

    u32 height,width,status;

    tpg_config = XV_tpg_LookupConfig(XPAR_XV_TPG_0_DEVICE_ID);
    XV_tpg_CfgInitialize(&tpg, tpg_config, tpg_config->BaseAddress);


    status = XV_tpg_IsReady(&tpg);
	printf("TPG Status %u \n\r", (unsigned int) status);

	XV_tpg_Set_height(&tpg, (u32) video.height);
	XV_tpg_Set_width(&tpg, (u32) video.width);
	height = XV_tpg_Get_height(&tpg);
	width = XV_tpg_Get_width(&tpg);
	XV_tpg_Set_colorFormat(&tpg,XVIDC_CSF_RGB);
	XV_tpg_Set_maskId(&tpg, 0x0);
	XV_tpg_Set_motionSpeed(&tpg, 0x4);

	//printf("info from tpg %u %u \n\r", (unsigned int)height, (unsigned int)width);
	XV_tpg_Set_bckgndId(&tpg,XTPG_BKGND_SOLID_GREEN); //);

	status = XV_tpg_Get_bckgndId(&tpg);
	//printf("Status %x \n\r", (unsigned int) status);
	XV_tpg_EnableAutoRestart(&tpg);
	XV_tpg_Start(&tpg);

	status = XV_tpg_IsIdle(&tpg);
	InitVprocSs_CSC();


    cleanup_platform();
    return 0;
}

Testing

Testing this on the board requires the creation of a debug configuration to download the FPGA Bit file and ELF file for execution

Testing this on the AC701 board with a HDMI display attached showed the correct test pattern.

To ensure the colours show correctly each solid test pattern of RED, GREEN, BLUE and WHITE to ensure the colours are correctly mapped.

Now I can take a look at setting up the camera input and completing the rest of the image processing chain.