The imaging subsystem (ISS) deals with the processing of the pixel data coming from memory (image format encoding and decoding can be done to and from memory). With its subparts, such as interfaces and interconnects, image signal processor (ISP), Camera Adapter Layer (CAL), and still image coprocessor (SIMCOP), the ISS is a key component for the following use cases:
- Machine vision
- Robots with cameras
- Appliances with cameras
Figure 9-1 shows an overview of the ISS.
The direction of the arrows shows the command flow direction from the master (initiator) to the slave (target). The following color conventions are used for the connections:
- Blue: Bidirectional, 128-bit-wide interface data connection.
- Green: Write (ISS → system memory) data connection. Either 64-bit interface, 128-bit interface, or 32-bit MTC (inside ISP).
- Red: Read (system memory to ISS) data connection. 128-bit interface port or 32-bit MTC (inside ISP).
- Gray: 32-bit interface configuration connection.
- Solid black: Video port and camera interface related signals.
- Dotted black: Data flow stall control signal. Used to slow down ISP for memory-to-memory operation.
The ISS is mainly composed of CAL_B, an ISP, and a block-based imaging accelerator (SIMCOP).
The ISS is designed to reach high throughput and low latency with large image sensors. In high-performance mode, the ISP supports a pixel throughput of up to 532 MPix/s.
The ISS targets the following major use cases:
- Video / Preview
- Up to 1080p60
- Up to 2x 1080p30
- Stereo Video / Preview
- Multi-camera use cases:
- Up to 4 simultaneous cameras. ISP is time-shared.
Note: For a detailed list of features of a certain submodule, see the related subsection.
The ISS offers the following features:
- ISP:
- Memory-to-memory processing
- Up to 532 MPix/s throughput
- Statistic data collection
- Image pipe interface front-end raw data processing
- High-ISO video noise filtering (NSF3V)
- Global and local contrast enhancement accelerator (GLBCE)
- RGB and YUV data processing through ISIF and IPIPE
- Two image continuous real-time resizers
- Chroma noise filter
- CAL:
- Memory-to-memory functional mode. Pixel data is read from memory, processed and then sent back to the system memory and/or ISP
- SIMCOP:
- Memory-to-memory operation
- Warping accelerator
- Temporal video noise filter (VTNF)
- Direct memory access (DMA) controller
- Hardware sequencer
- Mesh based lens distortion and perspective correction
- The ISS relies on the centralized memory management unit (MMU) within the SoC. See Chapter 22, Memory Management Unit.