Smart AI Box is a cost-effective way of adding intelligence to existing non-analytics-based cameras present in retail stores, traffic roads, factories, and buildings. Such a system receives live video streams from multiple cameras, decodes them, and does intelligent video analytics at the edge relieving the burden of transferring large video streams back to the cloud for analysis. The applications of AI Box include security surveillance system with anomaly or event detection, workplace safety systems that verifies workers wear personal protective equipment (PPE) such as goggles, safety vests, and hard hats before entering a hazardous zone.

Figure 7-1 illustrates the data flow for a system where there are two camera inputs coming in via ethernet which need to be depacketized and then decoded. Smart Analytics via deep learning is run on the decoded image stream. TI provides a model analyzer tool [7], where AI analytics designers can select the deep learning model of their choice based on accuracy, inference time and DDR bandwidth consumption. An example of one such utilization is given for SSDLite-MobDet-EdgeTPU-coco network in Table 7-1.

The targeted objects that are detected have bounding boxes drawn around them and the images are then re-encoded. This type of setup is used to bring analytics to legacy FHD cameras which do not have an analytics processor attached to them.

Table 7-2 provides the final resource utilization table that a systems engineer can construct. Power Estimation is done via the PET tool provided in [8].

In order to measure the DDR bandwidth a gstreamer based pipeline was constructed that emulated the data flow in Figure 7-1. From the server side 4 encoded streams where put on the network and the client side is used to decode the streams, go through the data flow and re-encode

• Server side Gstreamer pipeline:

  gst-launch-1.0 -v v4l2src device=/dev/video-rpi-cam0 io-mode=dmabuf-import ! \
  video/x-bayer, width=1920, height=1080, framerate=60/1, format=rggb10 ! \
  tiovxisp sink_0::device=/dev/v4l-rpi-subdev0 sensor-name="SENSOR_SONY_IMX219_RPI" dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_10b.bin sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_10b.bin format-msb=9 ! \
  video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! \
  tiovxmultiscaler name=msc \
  msc. ! queue ! video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! v4l2h264enc ! rtph264pay ! udpsink host=192.168.1.134 port=6001  \
  msc. ! queue ! video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! v4l2h264enc ! rtph264pay ! udpsink host=192.168.1.134 port=6002  \
  msc. ! queue ! video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! v4l2h264enc ! rtph264pay ! udpsink host=192.168.1.134 port=6003  \
  msc. ! queue ! video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! v4l2h264enc ! rtph264pay ! udpsink host=192.168.1.134 port=6004

• Client side Gstreamer pipeline:

  gst-launch-1.0 -v udpsrc port=6001 ! 'application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96' ! \
  rtph264depay ! h264parse ! v4l2h264dec capture-io-mode=4 ! tiovxmultiscaler name=msc \
  msc. ! queue ! video/x-raw, width=640, height=480, format=NV12 ! kmssink driver-name=tidss \
  msc. ! queue ! video/x-raw, format=NV12, width=1920, height=1080 ! v4l2h264enc output-io-mode=5 extra-controls="controls,h264_i_frame_period=60,video_gop_size=60"  ! rtph264pay ! udpsink host=128.247.75.190 port=6001 &
  gst-launch-1.0 -v udpsrc port=6002 ! 'application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96' ! \
  rtph264depay ! h264parse ! v4l2h264dec capture-io-mode=4 ! queue ! v4l2h264enc output-io-mode=dmabuf-import extra-controls="controls,h264_i_frame_period=60,video_gop_size=60"   ! rtph264pay ! udpsink host=128.247.75.190 port=6002 & 
  gst-launch-1.0 -v udpsrc port=6003 ! 'application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96' ! \
  rtph264depay ! h264parse ! v4l2h264dec capture-io-mode=4 ! queue ! v4l2h264enc output-io-mode=dmabuf-import extra-controls="controls,h264_i_frame_period=60,video_gop_size=60"  ! rtph264pay ! udpsink host=128.247.75.190 port=6003 &
  gst-launch-1.0 -v udpsrc port=6004 ! 'application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96' ! \
  rtph264depay ! h264parse ! v4l2h264dec capture-io-mode=4 ! queue ! v4l2h264enc output-io-mode=dmabuf-import extra-controls="controls,h264_i_frame_period=60,video_gop_size=60"   ! rtph264pay ! udpsink host=128.247.75.190 port=6004 &