SPRAD74 March   2023 AM62A3 , AM62A3-Q1 , AM62A7 , AM62A7-Q1

 

  1.   Abstract
  2.   Trademarks
  3. 1Smarter Cameras at the Edge
  4. 2AM6xA Scalable Portfolio and the AM62A
  5. 3Smart Camera Use Cases
    1. 3.1 Security Camera Example
  6. 4Deep Learning on the AM62A
    1. 4.1 Deep Learning Accelerator
    2. 4.2 Edge AI Software
  7. 5VPAC Vision Accelerator and ISP
  8. 6Low-Power Performance
  9. 7Call to Action
  10. 8References

5 VPAC Vision Accelerator and ISP

The VPAC-3L is a 7th generation image sensor processor (ISP) from TI. This hardware accelerator performs image processing functions to support raw output image sensors.

The Vision Imaging Sub-System (VISS) is a traditional ISP, and supports the following features: de-noising, de-mosaicing, defective pixel correction, gain and white balancing (manual and automatic), chromatic aberration correction (CAC), and wide dynamic range (HDR) encoding. The de-mosaicing feature supports RGB-Ir, allowing high resolution capture of infrared frames without a secondary camera and frame alignment.

VPAC-3L includes a Lens Distortion Correction subsystem for dewarping images that are captured through high field-of-view lens. Cameras with wide angle lenses like fish-eye require correction to bring the image back into a rectangular form and reduce the non-linear stretching effects closer to the edges of the camera frame.

GUID-20230118-SS0I-130W-71J3-4KT9DH3CP5PX-low.jpg Figure 5-1 Lens Distortion Correction: A) The unprocessed frame from a fisheye lens, B) Front perspective output image C) Intermediate mesh image - warp and interpolate to project image into full-resolution rectangluar format using equisolid fisheye projection model

VPAC-3L also includes a multiscalar (MSC) engine for downscaling images. The MSC allows two inputs and 10 outputs, such that inputs can be copied and scaled independently. One use-case of this is to downscale an image for machine learning and maintain the full-size version so that the output of machine learning can be overlaid on the full resolution frame. The MSC supports up to 4x reduction in each dimension and automatically interpolates where needed, including stretching/shrinking to accommodate changes in aspect ratio. Figure 4-2 makes use of this, in which the "tiovxmultiscalar" gstreamer plugin produces two output paths that are joined again at the"tiovxdldrawbox" plugin for drawing bounding boxes from object detection onto the original image.

TI supports a number of camera sensors and modules and partners with 3rd parties to help developers add new camera sensors with driver support and ISP tuning. The ISP tuning tools also help more experienced ISP tuners perform this work themselves.