Bringing Intelligent Autonomy to Fine Motion Detection and People Counting with mmWave Sensors
Robust, high-accuracy RF-based sensing has teamed with industrial-strength on-chip processing to remove the costs and inefficiencies of false detection.
Architects and urban planners of smart buildings, factories, and cities need increasingly intelligent sensors to address resource conservation, alleviate growing security concerns, and smooth human-machine interactions. Specifically, detecting the position and tracking the movement of people will enable the autonomous operation of systems. As shown in Figure 1, these systems can include indoor/outdoor security and surveillance; automated doors; factory safety scanners for machinery; and automation equipment for control of lighting, heating, ventilation, and air conditioning (HVAC) and elevators.
mmWave sensors are unique in that they embed processing cores onboard the sensor in order to process the range, velocity, and angle data in real time.
Millimeter wave (mmWave) sensing can detect, localize, and track people in these systems. mmWave sensors are unique in that they can sense the range, velocity, and angle of objects regardless of environment while providing on-chip processing for high-level algorithms. These features serve occupancy and movement sensors in building, factory, and city automation by reducing false detection, providing high accuracy for location and direction of travel, and maintaining privacy, all in a single chip for at-the-edge processing.
What Challenges Are Ahead?
Today’s sensors for occupancy and people tracking use technologies such as passive infrared (PIR), optical cameras, active infrared such as LIDAR and 3-D time of flight (ToF), and 10GHz-to-24GHz microwave. Table 1 compares these technologies and lists their pros and cons. However, with rising expectations for security, safety, and efficiency, the next generation of sensors must deliver accurate and reliable sensing while overcoming common sensing problems.
False detection, seen in Figure 2, is the response or failure to respond—whether through an alarm, system trigger, etc.—when a sensing event happens. False detection can take two distinct forms, either a false positive or a false negative, and occurs because of particular sensing failures or sensitivities of a technology.
False negatives are the failure of a sensing system to respond to an event that would be considered important. False positives occur when a sensing system responds to an event that would not be considered important. Depending on the greater system to which it’s tied, a sensor false positive can have a result that’s as harmless as a light turning on, or perhaps more seriously, a result that requires a security guard to investigate.
Sensitivity to Motion
Have you ever been working late at your desk in an office, only to have the lights shut off on you because you were not moving enough for the lighting sensors to register your presence? This example of a false negative is common with PIR sensors because of their poor sensitivity to motion. People are generally stationary indoors except for their finer motions, such as typing on a computer, adjusting their position on a couch or just breathing. For accurate occupancy sensing, being sensitive to these types of very fine motions is mandatory.
One common cause of false positives is the environment, where ambient conditions such as lighting, precipitation, temperature, humidity, or airflow can cause the inadvertent trigger of a sensor. An example of this is with a camera or PIR sensor outdoors, where direct sunlight or precipitation can “blind” the sensor and cause it to register a motion event that isn’t there.
Direction of Motion
Another example of a false positive comes from not being able to accurately detect the direction of a person in motion. Consider an automatic door. How many times have you just walked past an automatic door at a convenience store or warehouse only to have it open? This wastes energy by running the motor of the door and letting air-conditioned air out. Being able to infer the direction a person is intending to go, rather than responding based just on their proximity will be critical to improving the efficiency of tomorrow’s sensing systems.
Detection based on location can also be a false positive. Consider an optical camera system used for surveillance and motion detection inside a secure perimeter. An example of a false positive would be the camera alerting a security guard to movement outside the secure perimeter, not just inside it. Accurately determining the position of a moving object can be critical to understanding if the moving object is inside or outside an area of concern.
Detection of Non-Humans
For most intelligent automation systems, the principal concern is the detection and localization of people, not other objects. Unfortunately, moving objects such as swaying trees, scurrying animals or passing vehicles can trick motion detection systems into believing there is a person present. In order to overcome this problem, a sensor should be able to filter or classify objects based on their size and movement characteristics.
With automation systems increasingly moving toward high levels of connectivity and intelligence, implementing sensors in public and private spaces will alert the public to the potential recognition of personal identity. Having a sensor that can provide meaningful data while still maintaining anonymity will be an important advantage.
Solution complexity can thwart a sensing solution’s adoption for building automation. Ensuring straightforward hardware and software design can greatly reduce the cost of bringing the technology to market and covering all the corner cases.
Having processing and decision-making at the edge also simplifies hardware and software design for building automation systems. With decisions at the sensor edge, users can realize simpler system designs and cost savings by minimizing data transfer, data storage, and the need for a centralized system or person to make decisions.
A significant challenge to sensors is that objects in the environment can often occlude line-of-sight vision to objects of interest. Walls, foliage, optically opaque glass, and other objects block sensors based on optical technologies and ultimately limit the installation, placement and use of these sensors. Sensors that use radio frequency (RF) and other penetrating technologies have an advantage in that they can see through certain materials, opening up new ways to implement these sensors.
Tangible Benefits for Building Automation
TI’s mmWave technology and IWR family of sensing devices have a number of key features that translate into tangible benefits for building automation applications, including the reduction of false detection.
mmWave is the only sensing technology that can provide three unique sets of data: range, velocity and angle. It is with this data combination that mmWave sensors can accurately determine the location of people as well as their direction of travel. Figure 3 shows how these data sets could be used to trigger the activation of a system when a person enters a specific region. The velocity data enables mmWave sensors to ignore objects that are not moving in the environment. Because people are always moving—even if just a small amount when they fidget or breathe—mmWave can pick them up.
Figure 4 shows how to use velocity data to infer a person’s direction of travel and speed. mmWave sensors are unique in that they embed processing cores onboard the sensor in order to process the range, velocity, and angle data in real time. The sensors also implement advanced algorithms to enable capabilities such as tracking the history of a person’s movement, triggering systems based on location or direction of travel, or classifying objects based on their size and motion.
Conducting embedded processing onboard the sensor means that mmWave sensors can perform all operations on a single chip, with no external processor necessary.
The unique data set and on-chip processing of mmWave sensors enables building automation systems to reduce false detection. mmWave sensors can detect very fine motions such as typing, talking, or breathing in order to prevent false negatives on occupancy, while ignoring objects that are static. Static objects can also be a source of false detection depending on size and shape; Figure 5 shows an example of how mmWave can ignore these static objects using an algorithm known as static clutter removal.
mmWave sensors transmit and receive RF signals and are by nature very resilient to environmental effects that can be common sources of false detection. They can sense accurately regardless of ambient lighting, temperature, humidity, and airflow and can even continue sensing in the presence of precipitation. This makes them especially strong for indoor or outdoor applications where sensing must be constant across a variety of environmental conditions. In addition, this resilience means that mmWave sensors do not require any complex software to cover environmental corner cases such as shadows or weather.
In applications where privacy is important—such as in bathrooms, locker rooms or gymnasiums—there may be sensitivities over the use of cameras and other optical-based solutions. In contrast, an mmWave sensor’s use of RF signals means that the sensors do not provide any personally identifiable information. The sensor signals can transmit through different types of materials such as drywall, plywood, and plastic, enabling unique installation options that can be hidden behind walls and other objects to avert system damage or maintain a clean industrial design.
mmWave sensors are innovating building automation applications by providing a robust, high-accuracy RF-based sensing medium with an unprecedented data set of range, velocity and angle and powerful on-chip processing. mmWave sensors bring additional value by seeing in challenging environments, such as bright sunlight, darkness, through walls and in rain. These features make mmWave the obvious choice for sensing and help to combat today’s challenges of false detection, privacy and solution complexity in order to enable the next generation of building automation sensors with intelligence and at-the-edge decision-making.
Keegan Garcia is Marketing Manager, Industrial Radar Sensors, Texas Instruments.