SPRADE7 November   2023 AM62A3 , AM62A3-Q1 , AM62A7 , AM62A7-Q1 , AM62P-Q1

 

  1.   1
  2.   Abstract
  3.   Trademarks
  4. 1Introduction
  5. 2AM62A Processor
  6. 3AM62A Video Processing Unit (VPU) Capability
  7. 4AM62A JPEG Encode Capability
    1. 4.1 Open Source GStreamer Overview
    2. 4.2 TI-Provided V4L2 Drivers for Multimedia
    3. 4.3 Hardware-Accelerated GStreamer Plugins
  8. 5Software Driver Architecture
  9. 6Performance Measurements
  10. 7Multimedia Use Cases
    1. 7.1 Smart AI Box
    2. 7.2 Surveillance
  11. 8Summary
  12. 9References

7.2 Surveillance

In many surveillance applications like traffic monitoring, event recognition systems, and drones it is desired to process a single camera stream at 8MP. AM62A’s ISP has a 4k linewidth and is therefore able to support high-resolution camera sensors. The internal ISP then downscales the image to lower resolutions and corrects for camera lens distortion effects before processing by the deep learning accelerator. This is illustrated in Figure 7-2.

GUID-20231120-SS0I-DP8L-HWWG-TMSCR02ZLPCH-low.svg Figure 7-2 Smart Surveillance Single Encode Use Case
Table 7-3 Resource Utilization for 8MP Surveillance Camera With Encode and Analytics Running
Main IP Loading
Encoder 8MP@ 15 fps = 120MP/s
VPAC3L (ISP) 8MP@ 15fps = 120MP/s
ARM loading @ 1.25GHz 16%
Deep Learning C7/MMA @850MHz 25%
DDR BW 1860 MB/s (2410 – 550(Display)) MB/s
Power Consumption Est (85c) using PET ~1.5W
  • Gstreamer Pipeline used:
    gst-launch-1.0 -v v4l2src device=/dev/video3  io-mode=dmabuf-import ! \
video/x-bayer, width=3280, height=2464, framerate=15/1, format=rggb ! \
tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name="SENSOR_SONY_IMX219_RPI" dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss.bin sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a.bin format-msb=7 ! \
video/x-raw, format=NV12, width=3280, height=2464, framerate=15/1 ! \
tiovxmultiscaler  name=msc \
msc. ! queue ! video/x-raw, format=NV12, width=1920, height=1080 ! kmssink driver-name=tidss  \
msc. ! queue ! video/x-raw, format=NV12, width=3280, height=2464, framerate=15/1 ! v4l2h264enc ! \
rtph264pay ! udpsink host=192.168.65.187 port=5002

In many examples where the surveillance camera does not have a physical network connection like ethernet, the video stream needs to be encoded and sent through Wi-Fi. In this case, some applications need to encode the stream simultaneously at different resolutions and select the encode stream for Wi-Fi transmission based on what the network traffic is capable of supporting. This is illustrated in Figure 7-3.

GUID-20231120-SS0I-B9LB-W7MG-X9PNTXK7DCMV-low.svg Figure 7-3 Smart Surveillance Multiple Encode Use Case
Table 7-4 Smart Surveillance Multiple Encode Use Case Resource Utilization
Main IP Loading
Decoder+ Encoder (2MP+0.9M+0.3MP)@ 30 fps = 96MP/s
VPAC3L (ISP) 2MP@ 30 fps = 60MP/s
ARM loading @ 1.25GHz 34.5%
Deep Learning C7/MMA @850MHz 25%
DDR BW 1310 MB/s (1860-550(Display))
Power Consumption Est (85c) using PET
  • Gstreamer Pipeline used:
    gst-launch-1.0 -v v4l2src device=/dev/video-rpi-cam0 io-mode=dmabuf-import ! \
video/x-bayer, width=1920, height=1080, framerate=30/1, format=rggb10 ! \
tiovxisp sink_0::device=/dev/v4l-rpi-subdev0 sensor-name="SENSOR_SONY_IMX219_RPI" \
dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_10b.bin \
sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_10b.bin format-msb=9 ! \
video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! tiovxmultiscaler name=msc \
msc. ! queue ! video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! \
v4l2h264enc output-io-mode=dmabuf-import extra-controls="controls,h264_i_frame_period=60,video_gop_size=60"! \
rtph264pay ! udpsink host=192.168.65.187 port=5001 \
msc. ! queue ! video/x-raw, format=NV12, width=1280, height=720, framerate=30/1 ! \
v4l2h264enc output-io-mode=dmabuf-import extra-controls="controls,h264_i_frame_period=60,video_gop_size=60"! \
rtph264pay ! udpsink host=192.168.65.187 port=5002 \
msc. ! queue ! video/x-raw, format=NV12, width=640, height=480, framerate=30/1 ! tee name=t1 \
t1. ! queue ! v4l2h264enc output-io-mode=dmabuf-import extra-controls="controls,h264_i_frame_period=60,video_gop_size=60"! \
rtph264pay ! udpsink host=192.168.65.187 port=5003 \
t1. ! queue ! kmssink driver-name=tidss