JP6289027B2 - Person detection device and program - Google Patents

Description

  The present invention relates to a person detection device that detects the presence of a person and a program executed by the person detection device.

  Conventionally, there is a technique for detecting a person from a camera image. Patent Document 1 discloses an apparatus for detecting a person based on information on a face area included in a camera video. Patent Document 2 discloses an apparatus that extracts a region where a person exists from an input image and estimates the presence of the person based on the extraction result. Patent Document 3 describes a method for detecting a person using an infrared camera.

JP 2005-234947 A JP 2012-108785 A Japanese Patent Laid-Open No. 2002-202197

  With existing technologies, it is not possible to detect a person for various postures and behaviors of the person in the daily life space.

An object of this invention is to provide the person detection apparatus which can detect a person irrespective of a person's attitude | position etc.
Another object of the present invention is to provide a program executed in a person detection device.

  The present invention is a person detection device that detects the presence of a person based on a moving image, an imaging unit that acquires the moving image, and a feature point detection unit that detects a feature point from each frame of the moving image acquired by the imaging unit Acquiring a feature quantity related to the trajectory of the feature point detected by the feature point detection section, describing the feature quantity in a fixed dimension, a feature quantity described in the fixed dimension by the description section, and a person in advance The present invention relates to a person detection device including a person detection unit that detects the presence of a person based on a feature amount learned about the presence or absence of a person.

  The description unit includes, from the trajectory of the feature point, a motion feature based on the direction and length of the motion vector of the trajectory, an appearance feature based on a luminance or color histogram around each feature point constituting the trajectory, and a total trajectory movement. To extract a shape feature based on any of the following: the amount, the distance from the origin of the trajectory to the disappearance point, the area and aspect of the rectangle encompassing all the feature points, and the time from the occurrence of the trajectory to the disappearance It is preferable that the feature amounts of the motion feature, the appearance feature, and the shape feature are acquired based on the feature feature.

  The feature point detector preferably detects a plurality of feature points from one frame. In this case, the person detection unit detects the presence of a person based on a machine learning framework.

  The person detection device preferably includes a display unit that performs a predetermined display when the presence of a person is detected by the person detection unit.

  In addition, the present invention is a program executed in a person detection device that detects the presence of a person based on a moving image, the first step detecting a feature point from each frame of the moving image, and the detection in the first step A second step of acquiring a feature amount related to the trajectory of the feature point and describing the feature amount in a fixed dimension; a feature amount described in a fixed dimension in the second step; And a third step of detecting the presence of a person based on the amount.

  According to the present invention, it is possible to provide a person detection device that can detect a person regardless of the posture of the person. According to the present invention, it is possible to provide a program that is executed in the person detection device.

It is a block diagram which shows the structure of the person detection apparatus which concerns on one Embodiment. It is a figure which shows an example of the locus | trajectory of a feature point. It is a figure for demonstrating the example in the case of obtaining a motion characteristic. It is a figure for demonstrating the labeling which concerns on a modification. It is a figure for demonstrating the example in the case of obtaining an appearance feature. It is a figure for demonstrating the method of obtaining a fixed dimension. It is a figure for demonstrating detecting the presence of a person.

  Hereinafter, an embodiment of the present invention will be described. FIG. 1 is a block diagram illustrating a configuration of a person detection device 1 according to an embodiment. FIG. 2 is a diagram illustrating an example of a trajectory of feature points.

The person detection device 1 detects the presence of a person based on a moving image. That is, the person detection device 1 detects a person from the video. As shown in FIG. 1, the person detection device 1 includes an imaging unit 11, a control unit 12, a storage unit 13, and a display unit 14.
The imaging unit 11 acquires a moving image. The imaging unit 11 is a camera that can acquire a moving image.

  The control unit 12 controls the person detection device 1. The control unit 12 may be a CPU (Central Processing Unit). The control unit 12 includes a feature point detection unit 121, a description unit 122, and a person detection unit 123.

  The feature point detection unit 121 detects a feature point from each frame of the moving image acquired by the imaging unit 11. The feature point detection unit 121 detects a plurality of feature points from one frame. The feature point is, for example, the edge of the subject recorded in the frame. The feature point detection unit 121 detects feature points using an existing image analysis method. As an example, the feature point detection unit 121 detects feature points in a frame at high speed using a Harris operator or the like. As an example, the feature point detection unit 121 can detect about 200 feature points per frame. Note that the method of detecting feature points is not limited to the above method.

  The description unit 122 acquires a feature amount related to the trajectory of the feature point detected by the feature point detection unit 121, and describes the feature amount in a fixed dimension. That is, the description unit 122 tracks each feature point detected by the feature point detection unit 121 over a plurality of frames. As an example, the description unit 122 performs matching of feature points in adjacent frames using an optical flow calculation method typified by the Lucas-Kanade method, and tracks feature points from generation to disappearance of feature points. This method is commonly used as a Kanade-Lucas-Tomasi (KLT) tracker. Thereby, when the feature point 21 moves over a plurality of frames, the description unit 122 obtains a trajectory 22 of the feature point (see FIG. 2). The number of feature point trajectories obtained by the description unit 122 also changes in accordance with a parameter change for detecting the feature point in the feature point detection unit 121. Note that the method of acquiring the trajectory of the feature point is not limited to the above method.

  The description unit 122 extracts motion features, appearance features, and shape features from the trajectory of feature points. The description unit 122 acquires the feature amounts of the motion feature, the appearance feature, and the shape feature based on the extraction of the motion feature, the appearance feature, and the shape feature.

  The motion feature is a feature based on the direction and length of the motion vector of the trajectory. FIG. 3 is a diagram for explaining an example of obtaining a motion feature. The description unit 122 divides the trajectory of feature points shown in FIG. 3A into motion vectors in units of frames (see FIG. 3B). The description unit 122 then labels each of the divided motion vectors according to the direction and length (see FIG. 3C). In the case illustrated in FIG. 3C, there are 8 directions and 4 lengths (including 0 length). In the case illustrated in FIG. 3C, 25 motion vectors are labeled (“8 ways” × “3 ways in length” + “1 way in length 0”). The description unit 122 creates a frequency histogram based on the labeling (see FIG. 3D). In the case shown in FIG. 3, the histogram has 25 dimensions. The description unit 122 uses the histogram of each label as a feature amount related to the movement of the trajectory of the feature point. The motion feature is converted into a histogram with a fixed number of bins.

  The description unit 122 sets the number of divisions using the average length and variance of the motion vectors in each trajectory as indices, in order to avoid changing the length of the motion vector depending on the size of one frame (imaging size of the imaging unit 11). Is possible. The description unit 122 can arbitrarily set the number of dimensions according to the direction and length division number of the motion vector. FIG. 4 is a diagram for explaining labeling according to the modification. For example, it is possible to perform labeling with four motion vector directions (D) and three motion vector lengths (L) (including length 0) (see FIG. 4A). In the case shown in FIG. 4A, a nine-dimensional histogram is obtained. Further, for example, it is possible to perform labeling with 16 motion vector directions (D) and 5 motion vector lengths (L) (including length 0) (see FIG. 4B). ). In the case shown in FIG. 4B, a 65-dimensional histogram is obtained. Further, the description unit 122 can collect a plurality of histograms created from different divided areas and convert them into feature quantities, thereby making it possible to obtain a feature quantity that takes into account both a global movement of feature points and a minute movement.

  The appearance feature is a feature based on a luminance or color histogram around each feature point constituting the trajectory. The description unit 122 creates a histogram related to the luminance or color of each pixel in the vicinity of each feature point, for example, a 16 × 16 pixel region. The method of dividing the luminance or color space is arbitrary. FIG. 5 is a diagram for describing an example of obtaining appearance features. As illustrated in FIG. 5, the description unit 122 applies the color around the feature point (square area) to any area divided into 27 in the RGB color space. In 27 divisions, there are 3 ways for R, 3 ways for G, and 3 ways for B. The description unit 122 creates a frequency histogram based on the fitted result. In the case illustrated in FIG. 5, the histogram has 27 dimensions. The description unit 122 uses the histogram as a feature amount regarding the appearance of the trajectory of the feature point. The appearance feature is converted into a histogram with a fixed number of bins.

  Further, the description unit 122 may create a luminance gradient feature histogram such as a Local Binary Pattern by paying attention to luminance gradient information instead of the color information histogram. Histograms relating to appearance are created for the number of trajectory frames of feature points. By averaging the histograms in units of trajectories, the amount of feature is reduced while suppressing the influence of noise in a specific frame. Moreover, the description part 122 may average by the histogram of only several points on a locus | trajectory, and may save calculation amount.

  A shape feature is a feature based on any of the total amount of movement of the trajectory, the distance from the origination point of the trajectory to the disappearance point, the area and aspect of the rectangle encompassing all the feature points, and the time from the occurrence of the trajectory to the disappearance. It is. The total movement amount of the trajectory is obtained based on the sum of the motion vector lengths of the trajectories. The distance from the generation point to the disappearance point is obtained based on the distance from the initial detection position for the feature point to the final detection position for the feature point.

The total movement amount and distance are affected by the size of one frame (the imaging size of the imaging unit 11). For this reason, normalization is performed by the size of the frame (image width of the camera video) as shown in the following equation (1).
Li = li / W (1)
Here, li is the movement amount before and after Li normalization, and W is the image width of the camera video. The description unit 122 can calculate the shape feature without being influenced by the imaging size when using the imaging unit 11 by normalization.

  FIG. 6 is a diagram for explaining a method of obtaining a fixed dimension. When the description unit 122 acquires each feature amount (trajectory feature) as described above, the description unit 122 projects each feature amount within a predetermined time onto the feature space. In the feature space, an arbitrary number of codeword centers effective for creating a distribution are determined. The description unit 122 assigns the feature amount projected to the feature space to the nearest code word, counts the number of feature amounts for each code word, and creates a frequency histogram. The frequency histogram becomes a feature expression and a fixed dimension feature amount. Note that the time length of the trajectory of each feature point is indefinite. In the Bag-of-features framework used by the person detection unit 123 described later, it is necessary to align each feature amount in a fixed dimension.

  FIG. 7 is a diagram for explaining the detection of the presence of a person. The person detection unit 123 detects the presence of a person based on the feature amount described in a fixed dimension by the description unit 122 and the feature amount for which the presence / absence of a person has been learned in advance. The person detection unit 123 detects the presence of a person based on a machine learning framework. In the person detection unit 123, for example, a discriminator is created in a supervised machine learning framework such as Support vector machine (SVM) or AdaBoost. In the learning phase, learning is performed by giving correct data of “existence” or “absence” to the feature expression. A feature amount that is a determination criterion for detection in the person detection unit 123 is stored in the storage unit 13. In the operation phase, the person detection unit 123 determines the presence or absence of a person based on the input fixed-dimension feature value and the learning in the learning phase.

  The display unit 14 preferably performs a predetermined display when the presence of a person is detected by the person detection unit 123. As an example, the display unit 14 displays a character or an image indicating that a person exists.

[Comparative example]
Next, a comparative example will be described. The first comparative example is a detection device (first detection device) using a camera and a depth sensor. The first detection device detects a person based on detecting the face of the subject by the subject and the depth sensor. The second comparative example is a detection device using a camera (second detection device). The second detection device detects a person based on detecting the face of the subject by the camera. The second detection device uses a general face detection method. Table 1 is a comparison for the case where a person is detected by the first comparative example, the second comparative example, and the person detection device 1 of the present embodiment.

  In the first comparative example and the second comparative example, the matching rate is 100%. For this reason, in the first comparative example and the second comparative example, when a face is detected, it is determined that a person is present accurately. However, the reproducibility of the first comparative example and the second comparative example is lower than the reproducibility of the present embodiment, and many detection omissions occur. In the first comparative example and the second comparative example, since the person is detected based on the face detection, it is often overlooked when the person turns sideways or backward, and in an environment where the person acts at various angles. The high reproducibility was not obtained. On the other hand, in this embodiment, high values are obtained for both reproducibility and precision. Furthermore, in the present embodiment, the F value, which is a harmonic average of the precision and reproducibility, is higher than those in the first comparative example and the second comparative example. In the present embodiment, unlike the first comparative example and the second comparative example, face detection is not required, and therefore a person can be detected in any orientation of the subject. In addition, since the present embodiment learns in units of trajectories along which the feature points move, it can deal with various appearances. In the present embodiment, since the minute movement of the feature points contributes to the detection of the person, it is possible to detect the person with high accuracy.

  The person detection device 1 described above is detected by the imaging unit 11 that acquires a moving image, the feature point detection unit 121 that detects a feature point from each frame of the moving image acquired by the imaging unit 11, and the feature point detection unit 121. The feature amount related to the trajectory of the feature point is acquired, and the description portion 122 describing the feature amount in a fixed dimension, the feature amount described in the fixed dimension by the description portion 122, and the feature amount in which the presence / absence of a person is learned in advance And a person detection unit 123 that detects the presence of a person. Thus, since the person detection apparatus 1 determines the presence of a person or the absence of a person based on the trajectory of feature points, the person detection apparatus 1 can detect a person regardless of the posture of the person. That is, the person detection device 1 can detect a person even if the person is located at an arbitrary angle with respect to the imaging unit 11. Moreover, since the person detection device 1 uses feature points, even if the person moves slightly, the person detection apparatus 1 can detect the person with high accuracy. In addition, the person detection apparatus 1 can detect a person using an inexpensive camera device that is generally widely available.

Such a person detection device 1 can be used for the following purposes, for example. That is, the person detection device 1 can be used for detecting a person from an image taken by a surveillance camera, visual display terminals (VDT) work at work, detecting a person when analyzing a television viewing situation at home, and the like.
In addition, the person detection device 1 can detect not only a person but also a person's motion by learning. For example, the person detection device 1 can also detect that a person is eating or reading a newspaper. The person detection device 1 can also detect, for example, whether a person is playing baseball (throwing) or playing basketball (shooting).
Moreover, the person detection apparatus 1 can detect not only a person but also an animal by learning.

  The present invention may be configured as a program. The program is executed in the above-described person detection device 1, that is, a computer. The program sequentially executes a first step, a second step, and a third step. The first step detects feature points from each frame of the moving image. In the second step, a feature amount related to the trajectory of the feature point detected in the first step is acquired, and the feature amount is described in a fixed dimension. In the third step, the presence of a person is detected based on the feature amount described in the fixed dimension in the second step and the feature amount learned in advance. The program may be recorded on the person detection device 1, that is, a computer-readable recording medium. The recording medium is, for example, a flexible disk, an optical disk, a memory, a hard disk, or the like.

DESCRIPTION OF SYMBOLS 1 Person detection apparatus 11 Imaging part 14 Display part 121 Feature point detection part 122 Description part 123 Person detection part

Claims (5)

A person detection device that detects the presence or absence of a person based on an arbitrary video,
An imaging unit for acquiring a video;
A feature point detection unit that detects a feature point from each frame of the moving image acquired by the imaging unit;
Get the trajectories of the feature points detected by the feature point detecting unit, based on the trajectory of the feature point, which is characteristic motion characteristic on the time axis of the trajectory, an image characteristic of the periphery of the track visible features and described description unit in the fixed dimension of the feature quantity of shape features is and envelope characteristics of the track,
A human detection unit for detecting presence / absence of a person by inputting feature amounts of motion features, appearance features, and shape features described in a fixed dimension by the description unit;
With
The person detection unit
Supervised by using the motion feature, appearance feature, and shape feature feature values created in advance as a moving image in which a person is present or a moving image in which a person is not present as input data, and the presence or absence of a person in the moving image as correct data Including a classifier created by machine learning,
Human detection device. From the trajectory of feature points, the description section
Motion features based on the direction and length of the motion vector of the trajectory;
Appearance features based on luminance or color histograms around each feature point constituting the trajectory;
A shape feature based on any of the total amount of movement of the trajectory, the distance from the occurrence point of the trajectory to the disappearance point, the area and aspect of the rectangle encompassing all the feature points, and the time from the occurrence of the trajectory to the disappearance,
The person detection apparatus according to claim 1, wherein the feature amount of each of the motion feature, the appearance feature, and the shape feature is acquired based on extracting the motion feature. The feature point detection unit detects a plurality of feature points from one frame,
The person detection device according to claim 1, wherein the person detection unit detects the presence of a person based on a machine learning framework.   The person detection apparatus according to claim 1, further comprising a display unit configured to perform a predetermined display when the presence of a person is detected by the person detection unit. The program for functioning a computer as each function part of the person detection apparatus of any one of Claim 1 thru | or 4.

Images