This application report describes the work flow of tuning the ISP on TI's AM6xA vision processor family for a raw camera to obtain the best possible image quality. The tuning procedure provided in this report is accompanied with examples using the IMX219 sensor (RGB-only) and the OX05B1S sensor (RGB-IR) on the AM62A Starter Kit EVM.
Linux® is a registered trademark of Linus Torvalds.
Python® is a registered trademark of Python Software Foundation.
Microsoft® and Windows® are registered trademarks of Microsoft Corporation.
All trademarks are the property of their respective owners.
The AM6xA vision processors have a hardware accelerated Image Signal Processor (ISP) which is also referred as the Vision Pre-processing Accelerator (VPAC). With configurable image processing parameters, VPAC is designed to support a wide variety of raw camera modules (a typical raw camera module includes a lens, a filter, a raw image sensor, and sometimes a serializer). To obtain the best image quality for a specific raw camera module at run-time, the parameters of VPAC needs to be computed and then applied to process the raw sensor images frame by frame. To achieve that, the best VPAC parameters are typically prepared by engineers in an imaging lab under various controlled lighting conditions. Then at run-time, the prepared parameters are referenced and interpolated to fit the run-time lighting environment with the help of software imaging algorithms of Auto Exposure (AE), Auto White Balance (AWB), and dynamic ISP parameter control. The procedure of preparing the best VPAC parameters in an imaging lab is referred as ISP tuning in this application report.
The ISP tuning procedure described in this report applies to all SoCs in the AM6xA vision processor family, including AM62A, AM68A, and AM69A. Examples using the AM62A Starter Kit EVM are provided in the report.
For technical details of the ISP (VPAC) on a specific system-on-chip (SoC), see the Technical Reference Manual (TRM) of that SoC.
The ISP (VPAC) on the AM6xA family SoC is configured through the Dynamic Camera Configuration (DCC) binary files. In Linux® based applications, these binary files are provided to VPAC through the popular GStreamer pipeline. The processing blocks of VPAC are encapsulated by the GStreamer pipeline elements (tiovxisp, tiovxldc, tiovxmultiscaler) while all configurable parameters of VPAC are provided as properties.
For example, the following GStreamer pipeline performs video streaming from an IMX219 camera to an High-Definition Multimedia Interface (HDMI) display, using VPAC to process the raw images before sending them to the display:
gst-launch-1.0 v4l2src device=/dev/video-imx219-cam0 io-mode=dmabuf-import ! \
video/x-bayer, width=1920, height=1080, framerate=30/1, format=rggb10 ! \
tiovxisp sink_0::device=/dev/v4l-imx219-subdev0 \
sensor-name="SENSOR_SONY_IMX219_RPI" \
dcc-isp-file=/opt/imaging/imx219/dcc_viss_10b.bin \
sink_0::dcc-2a-file=/opt/imaging/imx219/dcc_2a_10b.bin format-msb=9 ! \
video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! \
kmssink driver-name=tidss sync=false
In the above pipeline, GStreamer element tiovxisp interfaces VPAC hardware and TI's reference software for imaging algorithms of AE and AWB (2A) and ISP parameter control. The VPAC configurations for IMX219 are provided through the following two binary files which are among the properties of tiovxisp:
These binary files are the output of the ISP tuning, and are also referred as the Dynamic Camera Configuration (DCC) profiles.
As an overview, the AM6xA ISP tuning procedure includes the following steps (using TI's reference imaging software and tuning tool as an example):
This application note uses the AM62A Starter Kit EVM with an IMX219 camera and an OX05B1S camera to demonstrate the above listed tuning steps. The principles and procedure of the tuning apply to any custom board and raw camera.