KR102268677B1 - Radar Signal Processing Method and Apparatus for Detecting a human inside a room - Google Patents

Abstract

Disclosed is a radar signal processing technology for detecting a position of a person indoors using a pulse radar. A target is detected from a two-dimensional time-distance image signal obtained by accumulating signals obtained by buffering a pulse radar signal for each pulse period over time. For example, the time-distance image signal may be processed by a convolutional neural network (CNN) circuit. A one-dimensional target region may be calculated by extracting a continuous region having a high degree of activation from the activation map of the convolutional neural network processing unit. A two-dimensional target image may be generated by rearwardly projecting a plurality of one-dimensional target regions obtained from a plurality of radar sensors.

Description

Radar Signal Processing Method and Apparatus for Detecting a human inside a room

Radar technology, a radar signal processing technology for detecting the position of a person indoors using a patented pulse radar is disclosed.

A technique for imaging the presence or location of a person by detecting a respiratory movement of a person by using an impulse-radio ultra-wideband radar is known. A conventional frequency analysis method for detecting a person by converting a radar signal into a frequency domain and analyzing a breathing frequency component thereof is known. Frequency analysis has difficulty in distinguishing objects that move with frequencies similar to breathing, such as curtains blowing in the wind, rotating fans, and moving toy trains, from human breathing.

A new advanced method for detecting the biological movement of a target from a pulsed radar signal is presented.

In addition, a method for detecting the biological motion of a target that is simpler and more reliable than the existing one is proposed.

An effective radar signal processing method for detecting and locating a person indoors and imaging it is proposed.

According to an aspect, a target is detected from a two-dimensional time-distance image signal obtained by accumulating signals obtained by buffering a pulse radar signal for each pulse period over time.

According to another aspect, in the detection process, an activation map indicating a degree to which a certain area is used as a basis for determining that a target exists may be calculated, and a one-dimensional target area may be calculated therefrom.

According to a further aspect, the time-distance image signal may be processed by a convolutional neural network (CNN) circuit.

According to an additional aspect, a one-dimensional target region may be calculated by extracting a continuous region having a high activation degree from the activation map of the convolutional neural network processing unit.

According to a further aspect, a plurality of pulsed radar sensors are prepared, and a target is detected from a two-dimensional time-distance image signal obtained by buffering the pulsed radar signal for each pulse period for each pulsed radar sensor, and in the activation map One-dimensional target regions may be calculated. A two-dimensional target image may be generated from the plurality of obtained one-dimensional target regions.

According to a dependent aspect, the 2D target image may be generated by back-projecting 1D target regions.

According to the proposed invention, it is possible to effectively detect and image a person in an indoor space using a radar sensor.

1 is a block diagram illustrating a configuration of a radar signal processing apparatus according to an exemplary embodiment.
2 is a block diagram illustrating a configuration of a radar signal processing apparatus according to another embodiment.
3 is a flowchart illustrating a configuration of a radar signal processing method according to an embodiment.
4 is a flowchart illustrating a configuration of a radar signal processing method according to another embodiment.
5 shows an exemplary radar signal of a radar sensor arranged in a detection target space and an activation map thereof.

The foregoing and additional aspects are embodied through the embodiments described with reference to the accompanying drawings. It is understood that various combinations of elements of each of the embodiments are possible as long as there is no contradiction between them or other mentions. Hereinafter, these aspects will be described in detail through embodiments described with reference to the accompanying drawings.

1 is a block diagram illustrating a configuration of a radar signal processing apparatus according to an exemplary embodiment. In an embodiment of the proposed invention, an Impulse-radio Ultra-Wideband radar (IR UWB Radar) sensor is disposed in the detection target space. In one embodiment, the IR UWB radar is a pulsed radar. UWB radar emits extremely narrow pulses and receives reflected waves from the target. Since the wavelength is a very narrow pulse wave, the received radar signal can include information about not only the position of the target but also its physiological movement, such as respiration or heartbeat, and provides information about the biological structure non-invasively. can do. The proposed invention aims at arranging an IR-UWB radar sensor and identifying how many breathing people exist in a target space and/or where they exist by using the received signal.

FIG. 5A shows an exemplary radar signal of a radar sensor arranged in a detection target space. In the example shown, 12 pulses per second radar signal is transmitted and its reflected waves, i.e., radar signal frames, are received. Each of the radar signal frames includes distance information on the time axis because the pulse is received by being reflected from the target. The resolution of the distance in each radar signal is 0.65 cm. The signal shown is a collection of 770 distance samples at a distance of 1-6 m from the receiving frame of the radar sensor for 20 seconds and stacking them up. That is, 12×20=240 radar signal frames each including 770 distance samples are stacked on the time axis and displayed in two dimensions. Here, samples between 0m and 1m were removed because they contained inaccurate information.

1 is a block diagram illustrating a configuration of a radar signal processing apparatus according to an exemplary embodiment. As shown, the radar signal processing method according to an embodiment receives and processes a radar signal from the pulse radar sensor 113 . A UWB radar sensor chip based on CMOS technology has already been commercialized and provided. In the illustrated embodiment, the pulse radar sensor 113 includes a UWB radar sensor chip, and further includes circuit elements such as a low noise amplifier and a power amplifier connected between the antenna 111 and the UWB radar sensor chip. can do. However, it goes without saying that it can also be implemented using individual control circuits and analog devices.

The radar signal processing apparatus according to an embodiment includes an image-based target detection unit 150 and a target position calculation unit 170 . The image-based target detection unit 150 and the target position calculation unit 170 may be implemented as a program in a digital signal processor provided inside the UWB radar sensor chip. As another example, it may be implemented as a combination of dedicated hardware and software in a separate digital signal processor or artificial intelligence processor connected to the UWB radar sensor chip. Various implementation methods are known and may be appropriately selected according to design specifications.

The image-based target detector 150 detects a target by receiving a two-dimensional time-distance image signal obtained by accumulating a signal obtained by buffering a pulse radar signal for each pulse period over time. An example of an input of the image-based target detection unit 150 is shown in FIG. 5A . Such a signal is referred to herein as a “time-distance image signal”. Although the pulse radar signal is cut to a certain length and arranged, it has redundancy characteristics similar to that of the image signal, so it will be referred to as such. The buffer 130 generates a two-dimensional time-distance image signal by accumulating a signal obtained by buffering a digital radar signal output from the pulse radar sensor 113 over time under the control of a not-shown memory controller. The image-based target detection unit 150 detects a target by receiving such a time-distance image signal. The proposed invention uses the entire time-distance image of a radar signal as an input to increase utilization of input information and reduce processing time.

According to an additional aspect, the image-based target detection unit 150 may include a convolutional neural network (CNN) processing unit that receives a time-distance image signal and outputs whether a target exists. Convolutional neural network circuits are known as artificial intelligence algorithms that are effectively used for image recognition. The proposed invention consists of a two-dimensional time-distance image signal having the characteristics of an image signal by accumulating a signal buffered with a pulse radar signal over time, and processing it with such a convolutional neural network circuit to determine whether a target exists or the number of targets can be calculated. In order to make the characteristics of the acquired time-distance image more prominent, image normalization processing may be further performed before input to the convolutional neural network circuit.

When the convolutional neural network circuit outputs the presence or absence of a target, it can learn from a time-distance image and whether the target exists. As another example, when the convolutional neural network circuit outputs the number of targets, learning may be performed using a time-distance image and the number of targets present in the image.

The target position calculator 170 calculates an activation map of the image-based target detector 150 and calculates a one-dimensional target area therefrom. According to an aspect, the image-based target detector 150 determines based on a specific region of an image, without analyzing a frequency characteristic when determining whether a target exists. The activation map is a map in which degree values indicating the degree to which a certain area is utilized as a basis for determining that the target exists are displayed for all areas. A region having a high value in the activation map means that the target position calculator 170 has determined that the target exists due to the region. Therefore, by discriminating a region showing a high value in the activation map, the location of the target can be known.

According to an additional aspect, the target location calculator may include an activation map calculator 171 and a one-dimensional location calculator 173 . The activation map calculation unit 171 calculates an activation map of the convolutional neural network circuit. In the illustrated embodiment, the activation map calculating unit 171 calculates an activation map according to a GradCAM (Gradient-weighted Class Activation Mapping) algorithm. In general, deep learning algorithms such as CNNs show very good characteristics, but have the disadvantage of not being able to explain the results. To solve this problem, XAI (Explainable Aritificial Intelligence) technology is being developed. GradCAM extracts the entire filter of the last layer of CNN, calculates the weight of how much each filter affects the result, applies it to the last layer, and then transforms it to output a positive value through one filter that averages multiple filters. . However, the proposed invention is not limited thereto, and other algorithms for calculating the activation map of the neural network circuit may be applied.

In respiration detection, the area where the CNN has a great influence on identifying the target is highly likely to be the area where the respiration is detected in the image, that is, the area where the target exists. FIG. 5(b) shows the Grad-CAM output image of the time-distance image of FIG. 5(a) in the same scale as the original image. The activation map is output according to the breathing signal pattern that exists at a distance of about 3.2m.

The one-dimensional position calculator 173 extracts a continuous region having a high activation degree from the activation map and calculates it as a one-dimensional target region. For example, when regions having an activation degree equal to or greater than a predetermined value are merged in the activation map, and when the size of the continuous region, that is, the number of pixels per region is equal to or greater than a predetermined value, it may be determined that a person is present in the corresponding distance.

In the embodiment shown in FIG. 1 , the output unit 190 outputs the one-dimensional target area information obtained by the target position calculation unit 170 . For example, the output unit 190 may output whether the target exists. As another example, the output unit 190 may output the number of targets existing in the corresponding space. As another example, the output unit 190 may output the distance from the radar sensor to the target existing in the space. As another example, the output unit 190 may visualize a target existing in a corresponding space and output it as an image.

2 is a block diagram illustrating a configuration of a radar signal processing apparatus according to another embodiment. As shown, the radar signal processing apparatus according to another exemplary embodiment includes a plurality of one-dimensional target area detectors 160-1, 160-2, ..., 160-n and a two-dimensional image generator 180. As shown in FIG.

Although not shown for convenience, each of the plurality of one-dimensional target area detection units 160-1, 160-2, ..., 160-n includes an image-based target detection unit 150-1, 150-2, ..., 150-n; and target position calculation units 170-1, 170-2, ..., 170-n. The image-based target detection unit 150-1, 150-2, ..., 150-n buffers the pulse radar signals from the pulse radar sensors 113-1, 113-2, ..., 113-n for each pulse period in time. A target is sensed by receiving a two-dimensional time-distance image signal that is accumulated according to the input.

According to an additional aspect, each image-based target detector 150 may include a convolutional neural network (CNN) processing unit that receives a time-distance image signal and outputs whether a target exists. In order to make the characteristics of the acquired time-distance image more prominent, image normalization processing may be further performed before input to the convolutional neural network circuit.

The target position calculation unit 170-1, 170-2, ..., 170-n calculates an activation map of the image-based target detection unit 150-1, 150-2, ..., 150-n and calculates an activation map therefrom. Count areas. In the illustrated embodiment, the activation map calculating unit 171 calculates an activation map according to a GradCAM (Gradient-weighted Class Activation Mapping) algorithm. Since their configuration is similar to that of FIG. 1 , a detailed description thereof will be omitted.

The 2D image generator 180 generates and outputs a 2D target image from outputs of the plurality of 1D target area detectors. The one-dimensional target areas output by the plurality of one-dimensional target area detection units 160-1, 160-2, ..., 160-n include distance information from the corresponding pulse radar sensor to the target. Since the same target is detected by a plurality of pulsed radar sensors, the two-dimensional position of the target can be known by combining them.

According to an additional aspect, the 2D image generator 180 is configured to back-project the plurality of 1D target areas calculated by the plurality of 1D target area detectors 160-1, 160-2, ..., 160-n. ) to generate a two-dimensional target image. For example, it is possible to draw an arc having a radius as a radius of the distance to the one-dimensional target area calculated based on each pulse radar sensor, and identify that the target is present at the intersection points. Since a method of generating a two-dimensional image by such a rear projection is known, a detailed description thereof will be omitted.

3 is a flowchart illustrating a configuration of a radar signal processing method according to an embodiment. The radar signal processing method according to an embodiment receives and processes a radar signal from the pulse radar sensor 113 . A UWB radar sensor chip based on CMOS technology has already been commercialized and provided. A radar signal processing method according to an embodiment includes an image-based target detection step 350 and a target position calculation step 370 . The image-based target detection step 350 and the target position calculation step 370 may be implemented as a program in a digital signal processor provided inside the UWB radar sensor chip. As another example, it may be implemented as a combination of dedicated hardware and software in a separate digital signal processor or artificial intelligence processor connected to the UWB radar sensor chip. Various implementation methods are known and may be appropriately selected according to design specifications.

First, a two-dimensional time-distance image signal is generated by accumulating a signal obtained by buffering a digital radar signal output from a pulse radar sensor over time under the control of a not-shown memory controller (step 330). In the image-based target detection step 350, such a time-distance image signal is received and the target is detected. The proposed invention uses the entire time-distance image of a radar signal as an input to increase utilization of input information and reduce processing time.

According to an additional aspect, the image-based target detection step 350 may include a convolutional neural network (CNN) processing step of receiving a time-distance image signal and outputting the presence or absence of a target. Convolutional neural network circuits are known as artificial intelligence algorithms that are effectively used for image recognition. The proposed invention consists of a two-dimensional time-distance image signal having the characteristics of an image signal by accumulating a signal buffered with a pulse radar signal over time, and processing it with such a convolutional neural network circuit to determine whether a target exists or the number of targets can be calculated. In order to make the characteristics of the acquired time-distance image more prominent, image normalization processing may be further performed before input to the convolutional neural network circuit.

When the convolutional neural network circuit outputs the presence or absence of a target, it can learn from a time-distance image and whether the target exists. As another example, when the convolutional neural network circuit outputs the number of targets, learning may be performed using a time-distance image and the number of targets present in the image.

In the target position calculation step 370 , an activation map of the process in the image-based target detection step 350 is calculated, and a one-dimensional target area is calculated therefrom. According to an aspect, the image-based target detection step 350 determines based on a specific region of an image without analyzing frequency characteristics when determining whether a target exists. The activation map is a map in which degree values indicating the degree to which a certain area is utilized as a basis for determining that the target exists are displayed for all areas. A region having a high value in the activation map means that the target location calculation step 370 determines that the target exists due to the region. Therefore, by discriminating a region showing a high value in the activation map, the location of the target can be known.

According to a further aspect, calculating the target location 370 may include calculating an activation map 371 and calculating a one-dimensional location 373 . In the activation map calculation step 371, an activation map of the convolutional neural network circuit is calculated. In the illustrated embodiment, the activation map calculation step 371 calculates the activation map according to the GradCAM (Gradient-weighted Class Activation Mapping) algorithm. In general, deep learning algorithms such as CNNs show very good characteristics, but have the disadvantage of not being able to explain the results. To solve this problem, XAI (Explainable Aritificial Intelligence) technology is being developed. GradCAM extracts the entire filter of the last layer of CNN, calculates the weight of how much each filter affects the result, applies it to the last layer, and then transforms it to output a positive value through one filter that averages multiple filters. . However, the proposed invention is not limited thereto, and other algorithms for calculating the activation map of the neural network circuit may be applied.

In respiration detection, the area where the CNN has a great influence on identifying the target is highly likely to be the area where the respiration is detected in the image, that is, the area where the target exists. FIG. 5(b) shows the Grad-CAM output image of the time-distance image of FIG. 5(a) in the same scale as the original image. The activation map is output according to the breathing signal pattern that exists at a distance of about 3.2m.

In the one-dimensional position calculation step 373, a continuous region having a high activation degree is extracted from the activation map and calculated as a one-dimensional target region. For example, when regions having an activation degree equal to or greater than a predetermined value are merged in the activation map, and when the size of the continuous region, that is, the number of pixels per region is equal to or greater than a predetermined value, it may be determined that a person is present in the corresponding distance.

In the embodiment shown in FIG. 3 , in the output step 390 , the one-dimensional target area information obtained in the target position calculation step 370 is output. For example, in the output step 390, the presence or absence of the target may be output. As another example, in the output step 390 , the number of targets existing in the corresponding space may be output. As another example, in the output step 390, the distance from the radar sensor to the target existing in the space may be output. As another example, in the output step 390 , a target existing in a corresponding space may be visualized and output as an image.

4 is a flowchart illustrating a configuration of a radar signal processing method according to another embodiment. As shown, the radar signal processing method according to another embodiment includes detecting a plurality of one-dimensional target regions (360-1, 360-2, ..., 360-n) and generating a two-dimensional image (380). .

First, each of the pulse radar signals from the pulse radar sensors is buffered and rearranged into a two-dimensional time-distance image signal (steps 330-1, 330-2, ..., 330-n). Thereafter, each buffered time-distance image signal is processed in a one-dimensional target region detection step to calculate each one-dimensional target region. Although not shown for convenience, each of the plurality of one-dimensional target area detection steps 360-1, 360-2, ..., 360-n includes an image-based target detection step 350-1, 350-2, ..., 350-n and , target position calculation steps 370-1, 370-2, ..., 370-n may be included. In the image-based target detection step 350-1, 350-2, ..., 350-n, a two-dimensional time-distance image signal obtained by accumulating signals obtained by buffering pulsed radar signals from pulsed radar sensors for each pulse period over time. Detects the target by receiving input.

According to an additional aspect, each image-based target detection step 350 may include a convolutional neural network (CNN) processing step of receiving a time-distance image signal and outputting the presence or absence of a target. In order to make the characteristics of the acquired time-distance image more prominent, image normalization processing may be further performed before input to the convolutional neural network circuit.

In the target position calculation step (370-1, 370-2, ..., 370-n), an activation map of the image-based target detection step (350-1, 350-2, ..., 350-n) is calculated, and the one-dimensional target is obtained therefrom. Count areas. In the illustrated embodiment, the activation map calculation step 371 calculates the activation map according to the GradCAM (Gradient-weighted Class Activation Mapping) algorithm. However, the proposed invention is not limited thereto, and other algorithms for calculating the activation map of the neural network circuit may be applied. Since their configuration is similar to that of FIG. 1 , a detailed description thereof will be omitted.

In the 2D image generation step 380, a 2D target image is generated and output from the outputs of the plurality of 1D target area detectors. The one-dimensional target areas output in the plurality of one-dimensional target area detection steps 360-1, 360-2, ..., 360-n include distance information from the corresponding pulse radar sensor to the target. Since the same target is detected by a plurality of pulsed radar sensors, the two-dimensional position of the target can be known by combining them.

According to an additional aspect, in the 2D image generating step 380, the plurality of 1D target areas calculated in the detecting the plurality of 1D target areas 360-1, 360-2, ..., 360-n are back-projected. projection) to generate a two-dimensional target image. For example, it is possible to draw an arc having a radius as a radius of the distance to the one-dimensional target area calculated based on each pulse radar sensor, and identify that the target is present at the intersection points. Since a method of generating a two-dimensional image by such a rear projection is known, a detailed description thereof will be omitted.

Although the present invention has been described above with reference to the accompanying drawings, the present invention is not limited thereto, and it should be construed to encompass various modifications that can be apparent from those skilled in the art. The claims are intended to cover such modifications.

111: antenna 113: pulse radar sensor
130: buffer 150: image-based target detection unit
160: one-dimensional target area detection unit 170: target position calculation unit
171: activation map calculation unit 173: one-dimensional position calculation unit
180: one-dimensional image generating unit 190: output unit

Claims (14)

Image-based including a convolutional neural network (CNN) processing unit that receives a two-dimensional time-distance image signal obtained by accumulating a pulse radar signal buffered for each pulse period over time and outputs the presence or absence of a target target detection unit;
a target location calculator comprising: an activation map calculator for calculating an activation map of the convolutional neural network processor; and a one-dimensional location calculator for calculating a one-dimensional target region by extracting a continuous region having an activation degree higher than a predetermined value from the activation map;
A radar signal processing device comprising a. delete delete delete A plurality of convolutional neural network (CNN) processing units each including a convolutional neural network (CNN) processing unit that receives a two-dimensional time-distance image signal obtained by accumulating a pulse radar signal buffered for each pulse period over time and outputs the presence or absence of a target of the image-based target detection unit, a plurality of activation map calculation units each calculating an activation map of the convolutional neural network processing unit, and extracting a continuous region having an activation degree higher than a certain value from each activation map to 1 a plurality of one-dimensional target area detection units each including a target position calculation unit including a one-dimensional position calculation unit for calculating a dimensional target area;
a two-dimensional image generator for generating and outputting a two-dimensional target image from outputs of the plurality of one-dimensional target region detectors;
A radar signal processing device comprising a. The radar signal processing apparatus of claim 5 , wherein the 2D image generator generates a 2D target image by back-projecting the plurality of 1D target areas calculated by each 1D target area detector. delete delete An image-based target detection step of detecting a target by processing a two-dimensional time-distance image signal obtained by accumulating a signal obtained by buffering a pulse radar signal for each pulse period over time by a convolutional neural network (CNN);
an activation map calculation step of calculating an activation map of the convolutional neural network;
a one-dimensional position calculation step of extracting a continuous region having an activation degree higher than a predetermined value from the activation map and calculating it as a one-dimensional target region;
A radar signal processing method comprising a. delete delete delete A two-dimensional time-distance image signal obtained by accumulating signals buffered for each pulse period from each of the pulse radar sensors from a plurality of pulse radar sensors is processed by a convolutional neural network (CNN) to obtain a target. An image-based target detection step of sensing, an activation map calculation step of calculating an activation map of each convolutional neural network, and a one-dimensional target area by extracting a continuous region with an activation degree higher than a certain value from each activation map a one-dimensional target area detection step including a one-dimensional position calculation step;
a two-dimensional image generating step of generating and outputting a two-dimensional target image from the one-dimensional target regions calculated in each one-dimensional position calculation step;
A radar signal processing method comprising a. The method of claim 13 , wherein the generating of the 2D image comprises back-projecting the plurality of 1D target areas calculated in the step of detecting the 1D target area to generate a 2D target image.

Images