Introduction

Artificial Intelligence (AI) and Machine Learning (ML) has garnered significant attention for the potential to effect and expand many industries globally. Using machine learning in applications ranging from robotics, security, research and a variety of industries in building automation, factory automation and personal electronics. The implementation and development of machine learning presents several challenges, including achieving autonomous decision making, and creating complex systems capable of adapting to unstructured environments.

Radar-based sensor integrated circuits (ICs) are becoming a popular technology for position and proximity sensing designs because of the sensors long-range capabilities, high-motion sensitivity, and privacy features. With high accuracy, radar sensors are also popular in the automotive and industrial markets for applications in robotics, home security, personal electronics, people presence and motion detection. mmWave radar sensors are one of the leading sensing technologies to use machine learning to provide more reliable and accurate classification for identification of objects closer to that of more expensive vision systems.

Classification of Objects Using Radar

A crucial component of any sensing technology is the ability to correctly understand and identify the objects around the object. mmWave is not only able to precisely detect the position of an object in a 3D environment, but through machine learning mmWave is also able to classify of the type of object. Every object living or non-living has a different micro-doppler signature in the movement, through which mmWave is able to accurately classify objects using machine learning. Various micro-doppler signatures of different objects are show in Figure 1.

Figure 1 Different Micro-Doppler Signatures of Objects

mmWave provides the ability to avoid any unwanted false errors based on unwanted movement (for example, movement of trees, leaves, small animals, robotics lawn mowers and so on). Using angle information from multi-receiver system to determine height and range objects can be filtered based on size, distance, and Doppler. This can help a home security system classify objects as a human, animal, or identify cars to limit any unwanted notifications or only notify users when certain objects have been detected.

Figure 2 Radar Classification Output to Differentiate Between Fan and Human

Figure 3 Scene View to Differentiate Between Fan and Human

This accurate form of classification is not just limited to only objects, mmWave can also use the micro-doppler signature of the ground beneath the micro-doppler to determine what surface the mmWave sensor is pointed at. Helping a robotic lawnmower understand if the robot is on grass, concrete, or even helping a home robot to determine if an area is wet (the ground has a puddle) to notify the homeowner or if the surface is dry.

Figure 4 Surface Classification of Grass vs Not Grass

mmWave sensors provide the designed for navigation and perception while still allowing for applications to be contactless and private. Classification of objects and surfaces can be used in a variety of end equipments. Enhancing the user experience by allowing for devices to become more accurate and smarter, opening the possibility to tailoring interactions or actions with the end user based off of these determinations.

Gesture Recognition With Radar

Classifying objects isn’t the only thing that mmWave radar sensors can utilize machine learning for, gesture recognition is one of the most anticipated technology features in many applications. Removing the need of a remote to change the channel on a TV, instead using a swipe of a hand, or turning up the volume on your mobile speaker. Many applications can utilize gesture recognition to expand the product offering and open the door to many new and exciting features.

Figure 5 Radar using Gesture Classification to Determine Movement as Down to Up

By using machine learning mmWave is able to capture a variety of different movements such as swiping left or right, moving your hand up or down and even unique movements like moving your hand in a clockwise or counter-clockwise motion to enable an action. Imagine no longer having to type in a passcode to unlock your front door and instead having the option to perform a variety of gestures to open the door, or presenting a slideshow and using gestures to move to the next slide instead of using buttons. Machine learning allows for all of these applications to be possible while maintaining all of the low-cost, low-power, and high-performance metrics that mmWave is already known for.

Benefits of mmWave Sensors Over LiDAR and Camera

mmWave Radar technology is often chosen over LiDAR, cameras and other optical sensors not only for the cost savings but also due to radar’s ability to work well in adverse weather conditions while cameras can be affected by poor lighting and weather. Radar also has a broad range and coverage allowing sensors to detect objects over a hundred meters away. With mobile robotic applications needing to save on power consumption as well often times customers use radar which can reach as low as 1 mW for presence detection. However, in combination radar being paired with a LiDAR, camera or other optical sensor can help applications reach nearly every corner case scenario your application can need.

From a functional safety standpoint, TI has designed a non-contact radar sensors such as the IWR6843 following an extensive hardware and software development process, and with device certification from (TÜV) SÜD. All TI radar sensors come with several built-in functional safety mechanisms that provide the necessary diagnostic coverage required by IEC 61508 to meet hardware capabilities as high as Safety Integrity Level (SIL) 2 at the component level.

TI provides this functional safety collateral available online through diagnostic software library, compiler qualification kits, third-party operating systems, development tools, and additional documentations allowing engineers to streamline the safety design process and the system-level certification.

Summary

Machine Learning provides a variety of new use cases radar can cover such as: surface classification, human vs non-human, gesture recognition and more. Now machine learning can be made easy with TI’s open-source PyTorch model where all models run on-chip and were created using open-source libraries and tools. Texas Instruments provides a complete step-by-step flow covering everything needed to deploy a model of your own. Using Python for data collection, PyTorch and Jupyter for any model-based training and Code Composer Studio for any TVM and C Integration needed. Providing anything you can need for data manipulation, model generation, and eventual target integration.

Figure 6 Machine Learning Flow using TI Radar

Texas Instruments provides a variety of learning resources when implementing machine learning into applications to make getting started today even easier than before. With the adverse benefits of mmWave sensors such as low cost, low power consumption and the ability to accurately sense in all weather conditions while being extremely accurate makes the mmWave Radars the top sensing design for applying machine learning in applications.

Resources

• Texas Instruments, Radar Toolbox for mmWave Sensors.
• Texas Instruments, Radar Camera Fusion.
• Texas Instruments, IWR6843AOP Evaluation Module for Single-Chip 60GHz Antenna-on-Package (AoP) mmWave Sensor.