JP2019148428A - Human detection device and human detection method - Google Patents

Abstract

To perform human detection effectively by using only CSI amplitude information.SOLUTION: There are provided: a plurality of antennas; a plurality of transmitters and a plurality of receivers coupled to the plurality of antennas and configured to communicate in compliance with wireless LAN standards including channel state information (CSI); a detector that determines whether there is a person in the vicinity on the basis of the amplitude of CSI received by the plurality of antennas.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to an apparatus and method for detecting a person.

  There are various ways to detect the presence of a person. Non-Patent Document 1 performs human detection using phase information of CSI (Channel State Information).

Liangyi Gong, Wu Yang, Dapeng Man, Guozhong Dong, Miao Yu and Jiguang Lv, “WiFi-Based Real-Time Calibration-Free Passive Human Motion Detection,” Sensors, 2015

  The inventors of the present application have found that human detection can be performed effectively using only CSI amplitude information.

  The human detection device includes a plurality of antennas, a plurality of transmitters coupled to the plurality of antennas and configured to perform communication based on a wireless LAN standard including channel state information (CSI), and a plurality of receivers; A detector for determining whether there is a person in the vicinity based on the amplitude of the CSI received by the plurality of antennas.

  The human detection method is a method using a plurality of transmitters and a plurality of receivers coupled to a plurality of antennas and configured to perform communication based on a wireless LAN standard including channel state information (CSI), Determining whether there is a person in the vicinity based on the amplitude of the CSI received by the plurality of antennas.

  Human detection can be performed effectively using only the amplitude information of CSI.

It is a figure which shows the structure of a person detection apparatus. It is a figure which shows an example of the threshold value produced in each subcarrier. It is a figure which shows the structure of a person detection apparatus. It is a figure which shows an example of the threshold value produced in each subcarrier, and the instantaneous value of received data. It is a figure which shows an example of the data for about 15 minutes. It is a figure which shows the data which took the movement sum with respect to the data of FIG. It is a figure which shows the plot of the value which totaled the difference with the minimum value of the plot of FIG. 6 for 5 minutes, and divided by the minimum value. It is a figure which shows the plot of the 2nd component of a principal component analysis (PCA) in case a person exists. It is a figure which shows the plot of the 2nd component of PCA when there is no person. It is the figure which plotted the value of the root mean square of the value of FIG.8 and FIG.9. It is a figure which shows the structure of the human detection apparatus using machine learning.

  In human detection according to the present disclosure, a radio wave of WiFi (registered trademark) which is a standard of a wireless LAN (local area network) is used. Specifically, human detection is performed using CSI, which is originally used for beamforming technology.

  In the wireless LAN standards IEEE 802.11n and IEEE 802.11ac that are widely used as WiFi, the communication speed is improved by applying MIMO (Multiple Input Multiple Output) transmission using a plurality of antennas at the transmitter and the receiver. Beam forming is possible by using a plurality of antennas. Beamforming uses the property of radio waves that strengthen each other when they are the same phase and cancel each other when they are opposite phases. In an actual environment, radio waves are reflected, scattered, and diffracted in a complex manner by various objects, so the transmitter does not know how the signal propagates to the receiver. Therefore, in the beam forming technique, efficient transmission is realized by using information on a propagation path such as the amplitude and phase of a signal received from a receiver, that is, CSI. When modulated by OFDM (Orthogonal Frequency Division Multiplexing), data is divided into subcarriers having a plurality of frequencies. CSI represents the state of each subcarrier and is expressed by the following Equation 1.

H (f _k ) = || H (f _k ) || e ^{j∠H (fk)} −−− (1)
Here, H (f _k ) is represented by a complex number, and represents CSI in a subcarrier having a frequency f _k , k∈ [1,30]. || H (f _k ) || and ∠H (f _k ) represent the amplitude and phase of H (f _k ), respectively.

  FIG. 1 is a diagram illustrating the structure of the human detection device 100. The human detection device 100 communicates with the access point 190 in accordance with a wireless LAN standard (for example, IEEE 802.11n or IEEE 802.11ac of WiFi).

  The transmitter group 110 transmits data toward the access point 190 at regular intervals via the antenna group 112. The transmitter group 110 includes a plurality of transmitters, and may include, for example, three transmitters. The antenna group 112 includes a plurality of antennas, and may include, for example, three antennas. The frequency band of the radio wave used by the transmitter group 110 is a 2.4 GHz band or a 5 GHz band, which is a WiFi standard.

  The access point 190 that has received the data transmits the data to the human detection device 100. The receiver group 120 receives data via the antenna group 122. The receiver group 120 includes a plurality of receivers, and may include, for example, three receivers. The antenna group 122 includes a plurality of antennas, and may include, for example, three antennas. The data received by the receiver group 120 includes CSI that is information regarding the propagation path.

  In the present embodiment, the receiver group 120 acquires CSI data in the absence of a person in advance. The storage circuit 130 stores this data.

  The detector 140 detects a person by comparing the signal 124 from the receiver group 120 with the signal 134 stored in the storage circuit 130.

In the present embodiment, human detection is performed by detecting a difference between CSI without a person (referred to as “background data”) and CSI with a person moving. Here, the background data is data having a certain length of time acquired in advance. The amplitude of each subcarrier of CSI varies greatly when a person moves. This is mainly because radio waves are reflected and scattered by the human body. Therefore, by calculating the difference between the amplitude of the current frame and the previous frame, a change in amplitude can be detected, that is, the presence of a person can be detected. If CSI at time t is H _t (f _k ), the difference D _t (f _k ) is expressed by the following Equation 2.

D _t (f _k ) = √ ((|| H _t (f _k ) ||-|| H _t-1 (f _k ) ||) ² ) −−− (2)
A difference value D _t (f _k ) is calculated for each subcarrier of the background data, and a standard deviation δ representing the degree of variation in time series is calculated for each subcarrier. Assuming that the degree of variation of values in each subcarrier can be approximated by a normal distribution, an average value ± 3δ of background data is adopted as a threshold value. ± 3δ in the normal distribution represents 99.7% of the entire region.

  FIG. 2 is a diagram illustrating an example of threshold values created in each subcarrier. In FIG. 2, the vertical axis represents a change in amplitude, and the horizontal axis represents a subcarrier. Plots Max and Min in FIG. 2 are obtained by taking the average difference between the amplitude of the current frame and the previous frame for 5 to 10 minutes in the background data registration phase. For example, when the user purchases and installs the human detection device 100, the background data may be registered only once. The plots Max and Min in FIG. 2 correspond to changes in subcarrier amplitude between adjacent frames.

The threshold value based on background data acquired in advance is compared with the difference D _t (f _k ) at the current time t. If the difference D _t (f _k ) is larger than the threshold value, it is determined that a person exists. For example, when the plot D _t (f _k ) completely falls between the plots Max and Min in FIG. 2, it is determined that there is no person. Conversely, when at least a part of the plot D _t (f _k ) comes out between the plots Max and Min in FIG. 2, it is determined that a person exists.

  The CSI has information on a plurality of (30 to 128) subcarriers, but the number of usable data varies depending on the number of antennas of the transmitter and the receiver. When there are three transmitters and three receivers, 3 × 3 × 30 = 270 and a maximum of 270 data can be used. The above threshold determination process is performed for all subcarriers, and if one subcarrier exceeds the threshold, it is determined that there is a person. That is, the detector 140 determines whether there is a person in the vicinity based on the amplitude of the CSI received by the plurality of antennas.

  The controller 150 determines the period of data transmitted by the transmitter group 110. The controller 150 changes the transmission cycle to, for example, 20 Hz, 30 Hz, etc. according to the result of human detection by the detector 140. For example, when a person is detected, the controller 150 may shorten the transmission cycle, and when no person is detected, the controller 150 may increase the transmission cycle. Thereby, when a person is not detected, the power consumption of the person detection apparatus 100 can be reduced. The controller 150 may not change the transmission cycle.

  The detector 140 outputs a signal representing the detection result from the output node 142 to, for example, a connected external device. In response to this signal, the external device can change the state of the device, for example. For example, when the human detection device 100 is connected to a television, an operation such as turning off the power of the television is performed when no people are detected (that is, when no people are present from the front of the television).

  In the prior art, human detection is performed using RSSI (Received Signal Strength Indication), which is the radio wave reception strength. However, in the prior art, human detection succeeds only when a person passes between a device that transmits a WiFi radio wave and an access point. On the other hand, in this embodiment, by using the CSI that is information on the propagation path, the distance between the WiFi device (human detection device 100) and the access point is as close as 1 m, and the person is separated from the two devices. People can be detected even when in a location.

  FIG. 3 is a diagram illustrating the structure of the human detection device 300. The human detection device 100 registered data in the absence of a person in advance as background data. The human detection device 300 performs human detection without using background data. Except for the detector 340 and the memory circuit 330, the human detection device 100 is almost the same.

  FIG. 4 is a diagram illustrating an example of the threshold value created in each subcarrier and the instantaneous value of the reception data 324. In FIG. 4, the vertical axis represents changes in amplitude, and the horizontal axis represents subcarriers. Here, the number of subcarriers is 56, but is not limited thereto, and may be different depending on the WiFi chip to be used. In the human detection device 300, two transmission antennas Tx1 and Tx2 are used as the antenna group 112, and two reception antennas Rx1 and Rx2 are used as the antenna group 122. The number of antennas is not limited to this, and any appropriate number of antennas may be used for transmission and reception.

  A plot Rx1 represents a change (instantaneous value) in amplitude in a path (represented as Tx1: Rx1) from the transmission antenna Tx1 to the reception antenna Rx1. The plot Rx2 represents the change (instantaneous value) of the amplitude in the path (represented as Tx1: Rx2) from the transmission antenna Tx1 to the reception antenna Rx2. The distance between the human detection device 300 and the access point 190 is 1 m. A person who is an object to be detected by the human detection device 300 performs an operation with relatively little movement such as watching a television or reading a newspaper in a place away from these two devices.

  The plots Rx1max and Rx1min represent the maximum value and the minimum value in a predetermined period of the amplitude change of the radio wave received by the receiving antenna Rx1, respectively. Plots Rx2max and Rx2min represent the maximum value and the minimum value of the amplitude change of the radio wave received by the receiving antenna Rx2 in a predetermined period, respectively. The amplitude change is obtained by taking the difference between the current frame amplitude and the previous frame amplitude.

  The detector 340 removes noise from the received signal 324. In the noise removal, for example, a median filter or an averaging process is performed on data of a predetermined number of frames in the past in time series. Thereby, spike noise can be removed. In addition, the detector 340 calculates an average value of the subcarrier values 1 to 56. The detector 340 sends the average value to the storage circuit 330 as a signal 334. The storage circuit 330 stores the signal 334 as a representative value in the frame. When two transmitting antennas and two receiving antennas are used, since there are 4 × 2 data, the signal 334 represents 4 data per frame. The storage circuit 330 stores data over a long period of time (for example, 5 minutes or 15 minutes).

  FIG. 5 is a diagram showing an example of data for about 15 minutes. In FIG. 5, the horizontal axis represents the number of frames corresponding to time (1 frame = 50 ms), and the vertical axis represents the amplitude. Four data for each antenna are displayed. In the paths Tx1: Rx1 and Tx2: Rx1, the plot with people has a larger amplitude than the plot without people. In the paths Tx1: Rx2 and Tx2: Rx2, the plot with the person has the same amplitude as the plot without the person. Therefore, when there is an amplitude difference for a certain path, it can be determined that there is a person.

  FIG. 6 is a diagram showing data obtained by taking a moving sum with respect to the data of FIG. In FIG. 6, the sum of the amplitude values for one minute is obtained and plotted so that the difference between when the person is present and when the person is absent can be emphasized. In this case, since the period of the amplitude data is 20 Hz, the sum of the amplitude values for 1200 frames is plotted. In FIG. 6, the presence or absence of a person can be determined using the threshold value represented by a broken line. In the case of FIG. 6, since there are plots that exceed the threshold values in the paths Tx1: Rx1 and Tx2: Rx1, it can be determined that there is a person.

  FIG. 7 is a diagram showing a plot of values obtained by summing the difference from the minimum value of the plot of FIG. 6 for 5 minutes and dividing by the minimum value. In the process of FIG. 7, the difference due to the presence or absence of a person is further emphasized in the graph of FIG. In the data of FIG. 6, the undulation of the data is greater when there is a person, and the area of the portion that looks like a mountain is larger. The difference is emphasized by calculating the area of this mountain part. The data in FIG. 6 is obtained by integrating the data for 1 minute, and the minimum integrated value is acquired from the data for 15 minutes. Since the minimum value is acquired for each antenna, there are four in this case. The difference between the data of FIG. 6 and its minimum value is taken, the total value of the number of data for 5 minutes is calculated, and the value obtained by dividing the total value by the minimum value is plotted on a graph in sequence as shown in FIG.

  In the graph of FIG. 6, Tx1: Rx2 and Tx2: Rx2 could not be separated by a threshold value, but in FIG. 7, the presence or absence of a person can be determined by the threshold values for all four antennas. Furthermore, the same threshold value can be used for the four antennas, and there is an effect that the complexity of the threshold adjustment is eliminated. According to the processing for emphasizing the difference in FIG. 7, since the motion is emphasized, it is easy to determine the presence or absence of a person using a threshold value.

  FIG. 8 is a diagram showing a plot of the second component of the principal component analysis (PCA) when there is a person. FIG. 9 is a diagram showing a plot of the second component of PCA when no one is present. 8 and 9, the vertical axis represents the magnitude of the second component, and the horizontal axis represents the number of frames corresponding to time.

  To obtain the data of FIG. 8, detector 340 performs dimensional compression using PCA. Thereby, human detection can be performed without acquiring prior background data. The detector 340 uses 15 minutes of data for each antenna. For the input data, the number of subcarriers was 56, and the number of dimensions was 19000 corresponding to the number of data for 15 minutes. According to this, the difference by the presence or absence of a person appeared in the 2nd component of PCA.

  As shown in FIG. 8, when there is a person, the value of the second component of PCA changes depending on the movement of the person. On the other hand, when there is no person, it is almost constant and does not change.

  Since the values in FIGS. 8 and 9 vary, the root mean square is calculated using Equation 3.

RMS [x] = √ ((Σ (xi) ² ) / N) −−− (3)
Where Σ is the sum of i = 1, ..., N. Since averaging is performed using data for one minute, N in Equation 3 is 1200.

  FIG. 10 is a diagram in which the root mean square values of the values in FIGS. 8 and 9 are plotted. In FIG. 10, when there is no person, the value does not change so much and the graph is flat. On the other hand, when there are people, there are many places where the value changes, and the range of the change is also large. When the standard deviation of these values for 15 minutes is calculated, it is 0.0051 when there is a person and 0.0012 when there is no person, and a clear difference is seen. By distinguishing this by a threshold value, the presence or absence of a person can be determined.

  Although the second component of the principal component analysis is used in this embodiment, the first component, the third component, the fourth component, or the fifth component may be used. Further, although data for 15 minutes was used as data for principal component analysis, the time may be shorter or longer than 15 minutes. When performing threshold processing, the average gradient of the graph of FIG. 10 may be used instead of the standard deviation. The average slope has a high value for a rough graph, and a low value for a flat graph, so it can be used to determine the presence or absence of a person.

  FIG. 11 is a diagram illustrating a structure of a human detection device 1100 that uses machine learning. The human detection device 1100 performs human detection using machine learning such as deep learning. Learning is performed before performing human detection. During pre-learning, the learning unit 1130 receives data from the receiver group 120 and performs learning based on teacher data when there is a person and teacher data when there is no person. When actually detecting a person, the detector 140 detects the person based on the filter or feature quantity generated by the learning unit 1130. The deep learning method uses CNN (Convolution Neural Networks) or RNN (Recurrent Neural Networks). When CNN is used, the received signal is treated as an image signal by plotting it on a graph as shown in FIG. By using an image as an input, it is possible to use a CNN that is widely used in image recognition. When RNN is used, the signal value is input as it is. By using machine learning, more accurate human detection is possible.

  Various functional blocks in the above-described embodiments can be realized by hardware, software, or a combination of hardware and software.

100 human detection device 110 transmitter group 112 antenna group 120 receiver group 122 antenna group 124 signal 130 storage circuit 134 signal 140 detector 142 output node 150 controller 190 access point

Claims (10)

Multiple antennas,
A plurality of transmitters coupled to the plurality of antennas and configured to perform communication conforming to a wireless LAN standard including channel state information (CSI); and a plurality of receivers; and the plurality of antennas received by the plurality of antennas A human detection device including a detector that determines whether there is a person in the vicinity based on the amplitude of CSI.   The human detector according to claim 1, wherein the detector makes the determination by taking a difference from background data.   The human detector according to claim 1, wherein the detector makes the determination by taking an average value of each subcarrier.   The human detector according to claim 1, wherein the detector makes the determination by principal component analysis.   The human detector according to claim 1, wherein the detector makes the determination by deep learning. A human detection method using a plurality of transmitters and a plurality of receivers coupled to a plurality of antennas and configured to perform communication conforming to a wireless LAN standard including channel state information (CSI),
A person detection method including determining whether or not there is a person in the vicinity based on the amplitude of the CSI received by the plurality of antennas.   The person detection method according to claim 6, wherein the determination includes taking a difference from background data.   The person detection method according to claim 6, wherein the determination includes taking an average value of each subcarrier.   The human detection method according to claim 6, wherein the determination includes performing principal component analysis.   The person detection method according to claim 6, wherein the determination includes deep learning.