Building a Camera / Imager Test Platform

Introduction

We have looked at creating image processing solutions previously which used image sensors which used a parallel interface or cameras which used HDMI interfaces.

One common interface for image sensors is MIPI, so let's take a look at how we can build an image processing design on the Zybo Z7 which is capable of using both HDMI input and the MIPI camera interface.

I also want to lay the ground work to be able to create an image processing system that allows me to quickly and easily test cameras and sensors. As such a future upgrade to this project will be to support a parallel Pmod camera as well.

We will then be able to select between which imager we wish to use for out algorithm. This has the added advantage that with pin out on the Zybo MIPI connector has been aligned with other commonly used maker cameras for example the Pi Cameras. We will look at how we can use the Pi camera in another project.

What is MIPI?

MIPI is the commonly used abbreviation for the Mobile Industry Processor Interface. This is an interface which is designed for uni-directional data transfer at high bandwidths using a number of high speed serial lanes.

MIPI has a number of different layers just like the OSI model, for this application we will be using the MIPI DPhy for the physical layer and MIPI Camera Serial Interface issue 2 (CSI-2) for the protocol which transfers the image data.

If you want to know more about MIPI have a look at this previous installment of the MicroZed Chronicles on MIPI

Zybo Z7 Interfacing

The Zybo Z7 interfaces with a MIPI camera using a Flat Flexible Cable (FFC) with a pin out as below. Using this interface we can use up to two MIPI lanes for image transfer. Configuration of the imager is provided using a I2C connection, while there are two GPIO provided.

These GPIOs provide different fucntionality depending upon what camera we are connected too. In the case of the PCam we only need to use GPIO on pin 11 this drives the power up signal to the imager.

In some applications we may need to provide a clock however, for the PCam we do not need too as the PCam board contains its own 12 MHz oscillator.

The actual pin out is as below

As the Zynq 7000 does not have DPhy capable IO, the designers of the Zybo Z7 have implemented a Dphy compatible approach which uses an external resistor network to provide the high speed and low speed MIPI interfaces.

FPGA Design

Our FPGA design is going to have two video inputs paths and a common video output path.

• HDMI Video In - Connected to J9
• MIPI Video In - Connected to J2
• HDMI Video Out - Connected to J8

Internally our FPGA architecture will use the following blocks from the Digilent Vivado Library.

• RGB2DVI - Conversion from the parallel output video format to HDMI / DVI
• DVI2RGB - Conversion from HDMI / DVI to a parallel video format
• MIPI D PHY RX - MIPI D Phy
• MIPI CSI-2 RX - MIPI CSI-2 IP block outputs AXI Stream Video
• AXI Dynamic Clock - Dynamic Clock Generator
• AXI Bayer to RGB - Converts the raw image to a full color image
• AXI Gamma Correction - Allows us to perform Gamma Correction

From the Vivado Library we will be using the following blocks

• Video In to AXIS - Conversion of the parallel video in to AXI Stream
• AXI Stream to Video Out - Conversion of AXI Stream to parallel video
• AXIS Switch - Allows switching between AXIS streams for HDMI and MIPI video feeds
• AXI VDMA - Stores the video frames in the PS DDR Memory
• Subset Converter - Remaps the colour channel correctly
• Video Timing Controller - Used to detect and generate video timing
• AXI GPIO - Provides the hot plug detection signal

We also need to configure the Zynq PS to provide the following

• I2C controller routed to EMIO this is the link we will use to configure the camera
• GPIO one bit wide which is routed to the EMIO, we use this to control the PCam power up and down.
• Clocking FS0 = 200 MHz, FS1 = 100 MHz, FS2 = 50 MHz

When it comes to clocking the AXI Lite network is clocked at 50 Mhz, the AXI Streaming network and the DPhy reference is connected to 200 MHz. While the Dynamic clock generator reference is connected to the 100 MHz clock.

To aid the integration and testing of the interfaces I placed several integrated logic analyzers within the design. Below you can see the initial capture of the MIPI input.

As we have two inputs we need to ensure both input paths work correctly as we desire.

Floor plan of the completed design.

Application Software

Once we have the FPGA design completed we need to write application SW to configure the image processing cores within the FPGA and configure the PCam for the correct operation.

Both cameras HDMI and MIPI will be configured to operate with a 720p resolution.

The application SW therefore needs to perform the following

• Configure the VTC to generate timing at 720p
• Configure the VDMA to read and write from the PS DDR memory
• Configure the GPIO to provide hot plug detection and power up the PCam
• Enable the DPhy and CSI-2 Receiver
• Set the dynamic clock generator to provide the correct pixel clock frequency

The largest part of the SW application is in configuration the PCam over the I2C link. The data sheet is quite comprehensive however the first thing we need to do is ensure we can communicate with the PCam.

I do this by reading the register 0x3100 and ensuring the response is 0x78 which is the I2C address of the device. IF I do not receive that value then I declare the camera is not connected, provided the camera is connected the next stage is to initialize the camera and then configure the camera for the mode we desire to operate in.

void detect_camera()
{
	XIicPs_Config *iic_conf;
	u32 Status;
	iic_conf = XIicPs_LookupConfig(IIC_cam);
   XIicPs_CfgInitialize(&iic_cam,iic_conf,iic_conf->BaseAddress);
   XIicPs_SetSClk(&iic_cam, IIC_SCLK_RATE);
   SendBuffer[0]= 0x31;
   SendBuffer[1]= 0x00;
   Status = XIicPs_MasterSendPolled(&iic_cam, SendBuffer, 2, IIC_CAM_ADDR);
	if (Status != XST_SUCCESS) {
		print("I2C write error\n\r");
		return XST_FAILURE;
	}
	Status = XIicPs_MasterRecvPolled(&iic_cam, RecvBuffer,1, IIC_CAM_ADDR);
	if (Status != XST_SUCCESS) {
		print("I2C read error\n\r");
		return XST_FAILURE;
	}
	if(RecvBuffer[0] != CAM_ID ){
		print("Camera not detected\n\r");
	}
	else{
		print("Camera detected \n\r");
	}
}

Once I had worked out what we need to configure in the PCam I created, a simple function which would read through an array and output the configuration data to the PCam.

int  Initial_setting_1 ( u32 *cfg_init , int cfg_init_QTY  )
{
	s32  Status , byte_count;
   int i ;
   u8 SendBuffer[10];
   for(i=0;i<(cfg_init_QTY*2);i+=2){
		SendBuffer[1]=  *(cfg_init + i);
		SendBuffer[0]=  (*(cfg_init + i))>>8;
		SendBuffer[2]=  *(cfg_init + i + 1);
	Status = XIicPs_MasterSendPolled(&iic_cam, SendBuffer, 3, IIC_CAM_ADDR);
	if (Status != XST_SUCCESS) {
		print("I2C read error\n\r");
		return XST_FAILURE;
	}
  usleep(1000);
  }
return XST_SUCCESS;
}

The software allows me to switch between the different input sources using the AXIS Switch.

Testing on the Bench

When I put all this together and ran the application on my bench it seemed to work pretty well providing me with a flexible platform for future testing of cameras and image sensors.

With the software all written we can use both video input formats. We will also be able to see differences in the performance.

Future Road Map

I intend to come back to this board in a future MicroZed chronicles blog and add in support for Pmod camera and expand the MIPI camera to work with the PI low light camera. Over the next few weeks and months but at the moment it is good sufficient for the testing I have in mind, and provides a very flexible platform.

You can find the files associated with this project here:

https://github.com/ATaylorCEngFIET/Hackster

See previous projects here.

More on on Xilinx using FPGA development weekly at MicroZed Chronicles.