Peter Labaziewicz
In October of 2002, Cameron Gulbransen was killed by his father, Greg Gulbransen, who was slowly backing his SUV out their driveway and did not see his son run behind the car due to the large blind spot behind his SUV. Little Cameron ran into that blind spot and was accidentally run over.
This was a turning point in the effort to install electronic cameras in automobiles to help improve safety.
Greg promised himself that what happened to his little boy would never happen again to anyone else. He joined KidsAndCars.org, a child safety advocacy group, and started a campaign to eliminate the blind spot behind cars by installing an electronic backup camera. After a 12-year campaign, he finally succeeded. The U.S. Dept. of Transportation’s (DOT) National Highway Traffic Safety Administration (NHTSA) announced on March 31, 2014 the requirement for rear visibility technology cameras in all new vehicles under 10,000 pounds by May 2018. “Rear visibility requirements will save lives, and will save many families from the heartache suffered after these tragic incidents occur,” said NHTSA Acting Administrator David Friedman.
The DOT estimates that 210 deaths and 15,000 injuries are caused every year by back-over accidents in the U.S., the majority of which involve children and senior citizens. KidsAndCars.org estimates that 1,126 children lost their lives due to back-over accidents between 1991 and 2012. A majority of these deaths could have been prevented by backup cameras.
Many manufacturers did not wait for the mandate to be in place, and started installing backup cameras as an option. By 2012, nearly half of all automobiles sold in the U.S. were already equipped with this feature, and many aftermarket manufacturers started making kits to retrofit older cars. This was the start of the “imaging revolution” in cars.
Now camera based active safety systems are rapidly multiplying in cars. In addition to backup cameras, your typical 2017 model year car may also have the following:
Many new models from 2017 onwards will have more than 12 cameras to help enhance safety.
A typical electronic digital camera consists of a lens to capture light, and an image sensor that converts the light into electrical signals on an array of photo-sensors called pixels. These pixel signals are then fed into an image signal processor (ISP) for processing into the desired image, and then sent out to a display for viewing, compressed for storage or sent out for further computer vision analysis.
Cameras on automobiles can be divided into two categories – those for visual display and those for machine vision. Cameras for visual display present an image to the driver on a display screen in the cockpit. Examples of these applications are:
Cameras for vision or analytics applications generate an image for use by a vision processor which runs analytics on pixel data to perform Advance Driver Assist Systems (ADAS) functions. The analytics functions run on a custom TI heterogeneous architecture called Vision Acceleration Pac consisting of custom built embedded vision engines (EVEs), working in tandem with industry leading TI C6000™ DSP and ARM® cores. EVE is a specialized fully programmable TI vector processor designed to efficiently process low-level and mid-level computer vision algorithms at very high speed. It complements the TI C6000 DSP which excels at processing high level vision algorithms. Examples of analytics applications are:
Requirements for image processing for visual use-cases are significantly different than those for vision/analytics use cases. Visual use-cases focus on providing the best perceptual image quality to a human viewer, emphasizing visually pleasing color and tone, lower perceptible level of noise, increased dynamic range and edge enhancement to sharpen the image. These requirements map directly to the perception of the human visual system.
Analytics use-cases on the other hand, do not care about perceptual image quality as they focus on a class of image processing algorithms which aid computer vision. Consequently, algorithms which increase perceptual visual quality, for example color correction and white balance correction, are not needed when processing for analytics.
ADAS applications rely on cameras to provide a digital video stream for computer vision algorithms to help with safety and driver comfort. The video stream must be of high quality, free of defects, real-time and have low latency. It must be able to handle typical automotive challenges of wide field of view and high dynamic range. Finally, the camera system should be able to operate within a low power and thermal budget, and maintain signal integrity over a wide variety of temperature and weather conditions.
Texas Instruments (TI) has a long history of in electronic imaging and digital signal processing. In 1972, the very first patent in the world for an electronic camera system was issued to TI. Since then TI has leveraged its many years of image signal processing experience into the design of ISPs for many applications such as digital still cameras, mobile phones, surveillance cameras, traffic cameras, and most recently now automotive visual and vision cameras.
The TDA3x System-on-Chip (SoC) has a 6th generation ISP that performs advanced image processing on image signals coming from up to six cameras. It can either output those images to a display or feed them into its programmable DSP/EVE compute engines to execute machine vision algorithms. The heterogeneous image processing architecture on the TDA3x is flexible and powerful enough to solve any Automotive Imaging Problem, including new unexpected ones. This is very important, as automotive imaging is still a new and developing field with many new applications and requirements being presented.
The automotive imaging environment is very challenging. Cameras have to work reliably under temperature extremes and in all weather conditions, including high dynamic range situations with extreme brightness and darkness. New technologies such as LED lighting systems bring on new challenges such as LED flicker. In addition, many of these functions can be safety critical. If the camera does not perform correctly, safety may be compromised. It’s a much more challenging situation than that of your typical mobile phone camera.
To help further enhance safety, reliability and robustness under a wide variety of conditions, more than one type of sensor is often needed to view the same scene. As an example, visible light imaging can be combined with radar imaging to work together to help improve safety. Radar and imaging sensors are very complementary. Imaging brings the advantages of high resolution, ability to identify and classify objects, as well as providing vital intelligence. Radar, on the other hand, can see in darkness, through fog rain or snow, and can measure distance and motion very quickly and effectively. The visual and radar images can be processed and fused together on the TI TDA3x or TDA2x processors to provide a solution that is much more robust.
Automotive imaging is a new and rapidly developing field that is helping to make cars safer, and ultimately in the not too distant future, enabling them to drive themselves. Automotive Imaging applications have to be robust enough to work in a wide range of challenging real world conditions. TI has developed a heterogeneous signal processing architecture on the TDA3x that is flexible and powerful enough to address any automotive imaging challenge, including new unexpected ones.
The TDA3x is currently being designed into the full range of automotive imaging applications.
TI PROVIDES TECHNICAL AND RELIABILITY DATA (INCLUDING DATASHEETS), DESIGN RESOURCES (INCLUDING REFERENCE DESIGNS), APPLICATION OR OTHER DESIGN ADVICE, WEB TOOLS, SAFETY INFORMATION, AND OTHER RESOURCES “AS IS” AND WITH ALL FAULTS, AND DISCLAIMS ALL WARRANTIES, EXPRESS AND IMPLIED, INCLUDING WITHOUT LIMITATION ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT OF THIRD PARTY INTELLECTUAL PROPERTY RIGHTS.
These resources are intended for skilled developers designing with TI products. You are solely responsible for (1) selecting the appropriate TI products for your application, (2) designing, validating and testing your application, and (3) ensuring your application meets applicable standards, and any other safety, security, or other requirements. These resources are subject to change without notice. TI grants you permission to use these resources only for development of an application that uses the TI products described in the resource. Other reproduction and display of these resources is prohibited. No license is granted to any other TI intellectual property right or to any third party intellectual property right. TI disclaims responsibility for, and you will fully indemnify TI and its representatives against, any claims, damages, costs, losses, and liabilities arising out of your use of these resources.
TI’s products are provided subject to TI’s Terms of Sale (www.ti.com/legal/termsofsale.html) or other applicable terms available either on ti.com or provided in conjunction with such TI products. TI’s provision of these resources does not expand or otherwise alter TI’s applicable warranties or warranty disclaimers for TI products.
Mailing Address: Texas Instruments, Post Office Box 655303, Dallas, Texas 75265
Copyright © 2023, Texas Instruments Incorporated