JP2005084985A - Individual recognition device and passing controller - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an individual recognition device by which the face of a history image is stationed at the mid section of the image, and to provide the probability that the face turns to the front is high to clearly identify a recognized person. <P>SOLUTION: The individual recognition device involves: continuously inputting a plurality of frames of the face images of the recognized person by using a video camera; successively checking feature information on the face specific to the person obtained from the respective input face images on the plurality of the successively inputted frames against dictionary information previously stored on a dictionary storage means; obtaining the degrees of similarity in both information, respectively; and recognizing whether the person is the subject by referring to each of the obtained similarities. The input face image on the frame obtained by the maximum value of the degrees of similarity is saved as the history information. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention provides, for example, a person recognition device that recognizes whether or not a person is the person by comparing a face image input by the person to be recognized with dictionary information registered in advance, and the person recognition The present invention relates to a traffic control device that controls traffic of a passerby (for example, entry / exit to a security area) using the device.

  Generally, this type of person recognition device inputs a face image of a person to be recognized using a video camera, and dictionary storage means using the face characteristic information unique to the person obtained from the input face image as dictionary information. Is stored (registered) in advance.

  At the time of recognition, using a video camera, a face image of a person to be recognized is continuously input in a plurality of frames, and the face unique to the person obtained from the input face images of the plurality of frames input continuously. The feature information is sequentially collated with dictionary information stored in advance in the dictionary storage means to determine the similarity between the two information, and it is determined whether or not the person is the person based on the obtained similarity.

In such a person recognition apparatus, a face image of a person using the apparatus may be stored as a history image. In that case, conventionally, in order to simplify the process, the face image of the first frame from which collation is started is acquired, and the face image of the first frame is stored as a history image (see, for example, Patent Document 1).
JP 2002-279424 A

  However, in the conventional method in which the face image of the first frame from which collation is started is used as the history image, the face is detected from the input face image, but the face may not be facing the front. However, there is a problem that it is difficult to determine whether the person is the person.

  Therefore, the present invention provides a person recognition device and traffic control that can clearly identify the person to be recognized by increasing the probability that the face of the history image is arranged in the center of the image and the face is facing the front. An object is to provide an apparatus.

  The person recognition apparatus of the present invention includes an image input unit that inputs at least a plurality of face images of a person to be recognized, and a first image storage unit that temporarily stores a plurality of input face images input by the image input unit. The information based on the plurality of input face images input by the image input means is sequentially compared with dictionary information registered in advance to obtain the similarity between the two information, and the person is identified by the obtained similarity. A recognizing means for recognizing whether or not the similarity is obtained, a similarity storing means for temporarily storing each similarity obtained in the recognizing means, and the similarity storing means based on the completion of the recognition processing by the recognizing means. First selection means for selecting a similarity indicating the highest value from among the stored similarities, and an input face when the similarity of the highest value selected by the first selection means is obtained A second selection unit that selects an image from the first image storage unit, and a second selection unit that stores, as a history image, an input face image when the highest similarity selected by the second selection unit is obtained. 2 image storage means.

  Further, the traffic control device of the present invention is an image input means for inputting at least a plurality of face images of a passerby to be recognized, and a first image for temporarily storing a plurality of input face images input by the image input means. The information based on a plurality of input face images input by the storage means and the image input means is sequentially compared with dictionary information registered in advance, and the similarity between the two information is obtained. Recognizing means for recognizing whether or not the passer is the person, passage control means for controlling the passage of the passer based on the recognition result of the recognition means, and each similarity obtained by the recognizing means are temporarily stored. Based on the completion of the recognition processing by the similarity storage means and the recognition means, a first degree of similarity indicating the highest value is selected from the similarities stored in the similarity storage means Selection means, second selection means for selecting an input face image from the first image storage means when the highest similarity selected by the first selection means is obtained, and the second selection means Second image storage means for storing the input face image when the similarity of the highest value selected by the selection means is obtained as a history image.

  According to the present invention, for example, by saving the input face image when the highest degree of similarity is obtained as a history image, the face of the history image is arranged in the center of the image, and the face faces the front. Therefore, it is possible to provide a person recognition device and a traffic control device that can clearly identify the person to be recognized.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
First, the first embodiment will be described.
FIG. 1 schematically shows a configuration of a person recognition apparatus according to the first embodiment. The person recognition apparatus includes a television camera 101 as an image acquisition unit that captures and inputs a face image (an image including at least a face) of a person to be recognized (hereinafter also referred to as a person) 100, from the vicinity of the camera 101. Illumination unit 102 as an illuminating unit that emits light at a certain illuminance toward at least the face of recognition target person 100, a face image input to recognition target person 100, various types of guidance, and the like, A display unit for inputting a personal ID (identification information) of 100, an input / output unit 103 as an input unit, and a processing unit for processing a face image input from the camera 101 and performing a recognition process of the face image 104.

  In the following description, image input data, extracted feature quantities, subspaces, eigenvectors for constructing the subspace, correlation matrix, registration time, date and time, status information such as location, password, ID number, etc. Each piece of personal information appears. And when it says recognition data, it contains the eigenvector for comprising a partial space or a partial space, and when it says registration information (dictionary information), it is image input data, the extracted feature-value, partial space, partial space. Includes an eigenvector, a correlation matrix, situation information, and personal information. Accordingly, the recognition data is included in the registration information.

  The camera 101 captures and inputs a face image of the person 100 to be recognized from below at a predetermined angle. For example, the camera 101 includes a television camera using an image sensor such as a CCD sensor.

  The illuminating unit 102 is an illuminating device such as a fluorescent lamp that emits light at a constant illuminance from the vicinity of the camera 101 (for example, the side of the camera 101) toward the face of the person 100 to be recognized.

  The input / output unit 103 displays the face image of the recognition target person 100 photographed by the camera 101, various guidance, processing results, and the like, and inputs the personal ID and password of the recognition target person 100. The liquid crystal display unit and the touch panel are integrated, and are provided in the vicinity of the camera 101, for example, in the upper part of the camera 101 at a position corresponding to the face portion of the person 100 to be recognized. Note that not only the guidance by display but also a voice guidance function may be added.

  The processing unit 104 uses an image input unit 105 as an image input unit for inputting a face image from the camera 101, and a human face image obtained from the image input unit 105, in comparison with a pattern dictionary prepared in advance. A face detection unit 106 serving as a feature extraction unit that detects the position of facial parts such as the face, eyes, and nose, and a feature extraction unit that extracts facial feature amounts such as grayscale information or partial space information based on the output of the face detection unit 106 When the dictionary information is registered, the feature extraction unit 107 as a dictionary storage unit that stores (registers) the feature amount extracted by the feature extraction unit 107 as dictionary information in association with the input image (face image) at the time of registration. In the dictionary information storage unit 108, during person recognition (face matching), the feature amount (input feature information) extracted by the feature extraction unit 107 and the dictionary registered in the dictionary information storage unit 108 The similarity between the two information is obtained by collating with the information, and the recognition unit 109 as a recognition means for recognizing whether or not the person is the person based on the obtained similarity, every predetermined time interval when inputting the face image Control means for controlling the image storage unit 110, the illumination unit 102, the input / output unit 103, the image storage unit 110, and the like as image storage means for temporarily storing inputted face images for a plurality of frames or storing history images The control part 111 is comprised and is comprised.

The image input unit 105 inputs the face image from the camera 101, digitizes it by A / D conversion, and sends it to the face detection unit 106.
First, the face detection unit 106 detects the face area of the recognition target person 100 from the face image input by the image input unit 105. For example, by obtaining a correlation value while moving a template prepared in advance in an image, a place having the highest correlation value is set as a face region. In addition, face detection means such as a face region extraction method using an eigenspace method or a subspace method may be used.

  Next, the face detection unit 106 detects the position of the facial parts such as eyes, nose, and mouth from the detected face area. The detection method is described in, for example, literature (Kazuhiro Fukui, Osamu Yamaguchi: “Face Feature Point Extraction by Combination of Shape Extraction and Pattern Matching”, IEICE Transactions (D), vol. J80-D-II, No. 8, pp2170-2177 (1997)).

  FIG. 2 shows an example of face detection results and eye / nose detection results. In FIG. 2, (fx, fy) is the center coordinates of the face, (fw, fh) is the size of the face, (EyeLx, EyeLy), (EyeRx, EyeRy) are the position coordinates of the left and right eyes, (NoseLx, NoseLy) , (NoseRx, NoseRy) indicate the position coordinates of the left and right nostrils. The eye position indicates the center point of the black eye, and the nose position indicates the center point of the nostril.

  The feature extraction unit 107 cuts the face area into a certain size and shape based on the position of the face part detected by the face detection unit 106, and uses the shading information as a feature amount. Here, for example, the gray value of an area of m pixels × n pixels is used as information as it is, and m × n-dimensional information is used as a feature vector.

  Further, as will be described later, the partial space itself may be used as input data for recognition. Therefore, the result of calculating the partial space is sent to the recognition unit 109 and the dictionary information storage unit 108.

  The dictionary information storage unit 108 holds dictionary information such as a partial space (or correlation matrix) used for recognizing or identifying the person 100 to be recognized. Also, attribute information such as date and time, time, and place at the time of registration can be held. The dictionary information is not limited to the partial space, and an input face image or an extracted face feature amount itself may be used.

  The recognizing unit 109 collates (compares) the recognition data (partial space) stored in the dictionary information storage unit 108 with the feature amount (shading information or partial space information) obtained by the feature extracting unit 107, thereby It recognizes who the recognition target person 100 who is photographing at 101 is, or identifies whether the person is a corresponding person. In order to recognize a person, it suffices to determine which person's data is most similar, and the person corresponding to the data having the maximum similarity may be used as the recognition result.

  In addition, when performing facial recognition using a card, ID number, personal identification number, key, etc., the degree of similarity with recognition data corresponding to personal information such as each person's card or ID number is calculated. The threshold is set, and if the threshold is exceeded, the person is identified.

  Means for inputting such a card, an ID number, a personal identification number, a key, and the like are necessary. When an ID number or a password is used, key input means may be provided.

  As a recognition method, when information of the feature extraction unit 107 is used, a method such as a subspace method or a composite similarity method may be used. The recognition method in the present embodiment is, for example, a literature (Kenichi Maeda, Sadaichi Watanabe: “Pattern matching method introducing a local structure”, IEICE Transactions (D), vol. J68-D, No. 3, pp 345-352 (1985)), a mutual subspace method which is a known recognition method is used.

と対角化し、固有ベクトルΦｉｎを求める。２つのΦｉｎ、Φｄで表わされる部分空間の部分空間類似度（０．０〜１．０）を求め、それを認識するための類似度とする。 In this method, recognition data in dictionary information stored in advance and input data are expressed as partial spaces, and an “angle” formed by two partial spaces is defined as a similarity. The subspace input here is referred to as an input subspace. Similarly, a correlation matrix Cin is obtained for the input data string,

And the eigenvector Φin is obtained. The subspace similarity (0.0 to 1.0) of the subspace represented by two Φin and Φd is obtained and used as the similarity for recognizing it.

  For example, the recognition unit 109 operates as shown in the flowchart of FIG. First, the operation differs depending on whether recognition work (one-to-one collation) is performed or identification work (one-to-one collation) is performed (step ST1). When performing the identification operation, first, the ID number of the target person to be recognized is read (step ST2). Next, dictionary information corresponding to the target ID number is read from the dictionary information storage unit 108 (step ST3).

  Next, as described above, in order to perform recognition by the subspace method or the like, the similarity between the subspace of each dictionary information and the input vector (eigenvector from the feature extraction unit 107) or the input subspace is calculated. (Step ST4). Next, the similarity is compared with a preset threshold value (steps ST5 and ST6), and the identification result is output (step ST7).

  When performing the recognition work, all the data to be recognized is read from the dictionary information storage unit 108 (step ST8). Then, the similarity with each dictionary information is calculated (step ST9). Next, the maximum similarity is selected from the calculated similarities (step ST10), and is output as a recognition result (step ST12).

  Note that it is also possible to verify whether the recognition result is correct by determining the maximum similarity as a threshold value as in step ST11 surrounded by a broken line in FIG. 3 (step ST13). For example, when the degree of similarity is too low, it can be determined that it is not any recognition target.

  As described above, the image storage unit 110 temporarily stores a plurality of frames of face images input when inputting a face image, or stores a face image selected from the temporarily stored face images of a plurality of frames as a history image. To do.

Next, a first operation example of a recognition process that mainly stores history images will be described with reference to a flowchart shown in FIG.
When the recognition target person 100 stands in front of the present apparatus (camera 101) and is detected (step S21), the camera 101 captures an image of the recognition target person 100, and receives a face detection unit via the image input unit 105. 106. That is, the image input unit 105 first acquires an image of the first frame from the camera 101 (step S22) and sends it to the face detection unit 106 and the control unit 111. The control unit 111 temporarily stores the first frame image in the image storage unit 110 (step S23). Here, the processing in step S23 corresponds to the first image storage means in the present invention.

  The face detection unit 106 detects a face area from the first frame image captured by the camera 101, and detects the position of a facial part such as eyes, nose, and mouth from the detected face area. Next, the feature extraction unit 107 extracts a facial feature amount (information based on the input face image) based on the position of the face part detected by the face detection unit 106, and sends it to the recognition unit 109.

  The recognizing unit 109 obtains the similarity between both pieces of information by collating the facial feature amount extracted by the feature extracting unit 107 with the dictionary information stored in the dictionary information unit 108 (step S24). The degree is temporarily stored in a memory (not shown) (step S25), and the obtained similarity is compared with a preset threshold (step S26). If the obtained similarity is smaller than the threshold, the degree is set in advance. It is checked whether or not the acquisition of the number of frames (for example, 50) has been completed (step S27), and if it has not been completed yet, the process returns to step S22 and the same processing as described above is repeated for the next frame. Here, the process of step S25 corresponds to the similarity storage means in the present invention.

  If the obtained similarity is greater than the threshold value in step S26, the collation is successful here, and the recognition unit 109 ends the collation process (recognition process). If the obtained similarity does not exceed the threshold value and the number of captured frame images exceeds 50, collation fails and the collation process (recognition process) ends.

  When the collation process ends, the recognition unit 109 selects the similarity indicating the highest value from the similarities temporarily stored in a memory (not shown) (the similarities from the maximum first frame to the 50th frame). (Step S28). Here, the process of step S28 corresponds to the first selection means in the present invention.

  Next, the control unit 111 receives the similarity indicating the highest value from the recognizing unit 109, and thereby each input face image temporarily stored in the image storage unit 110 (each input from the maximum first frame to the 50th frame). From the face image), the input face image when the similarity is the highest value is selected, and the input face image when the selected similarity is the highest value is selected as the history image in the history image area of the image storage unit 110. Is stored (stored) (step S29). Here, the processing in step S29 corresponds to the second selection means and the second image storage means in the present invention.

  Next, the control unit 111 clears each similarity temporarily stored in the memory of the recognition unit 109 and each input face image temporarily stored in the image storage unit 110 (step S30), and performs the recognition. The process ends.

Next, a second operation example of the recognition processing that mainly stores history images will be described with reference to the flowchart shown in FIG.
When the recognition target person 100 stands in front of the apparatus (camera 101) and is detected (step S41), the camera 101 captures an image of the recognition target person 100, and the face detection unit via the image input unit 105 106. That is, the image input unit 105 first acquires an image of the first frame from the camera 101 (step S42), and sends it to the face detection unit 106 and the control unit 111. The control unit 111 temporarily stores the first frame image in the image storage unit 110 (step S43). Here, the processing in step S43 corresponds to the first image storage means in the present invention.

  The face detection unit 106 detects a face area from the first frame image captured by the camera 101, and detects the position of a facial part such as eyes, nose, and mouth from the detected face area. Next, the feature extraction unit 107 extracts a facial feature amount based on the position of the face part detected by the face detection unit 106 and sends it to the recognition unit 109.

  The recognizing unit 109 obtains the similarity between the information by collating the facial feature amount extracted by the feature extracting unit 107 with the dictionary information stored in the dictionary information unit 108 (step S44). Next, the recognizing unit 109 acquires the eye and nose position information previously obtained for extracting the facial feature amount (step S45). Here, the processing in step S47 corresponds to the face feature point detecting means in the present invention.

  Next, the control unit 111 checks whether or not the history image has already been saved in the image storage unit 110 (step S46). If the history image has not been saved, the position information of the eyes and nose acquired in step S45. Is compared with a preset reference value, for example, when the eye position is within the reference value range and the nose position is within the reference value range, the image storage unit that is the face image at that time The input face image temporarily stored in 110 is read, and the read input face image is stored (stored) in the history image area of the image storage unit 110 as a history image (step S47). Here, the processing in step S47 corresponds to the second image storage means in the present invention.

  When the history image has been saved in the check in step S46 or when the history image has been saved in step S47, the recognition unit 109 compares the similarity obtained in step S44 with a preset threshold value (step S44). S48) If the obtained similarity is smaller than the threshold value, it is checked whether or not image acquisition for a preset number of frames (for example, 50) has been completed (step S49), and has not been completed yet. In this case, the process returns to step S42, and the same processing as described above is repeated for the next frame.

  If the obtained similarity is larger than the threshold value in step S48, the collation is successful here, and the recognition unit 109 ends the collation process (recognition process). If the obtained similarity does not exceed the threshold value and the number of captured frame images exceeds 50, collation fails and the collation process (recognition process) ends.

  In the above description, the input face image is stored as a history image when both the detected eye and nose positions are within the range of the reference values. However, both the eye and nose positions are not necessarily the reference images. The input face image when the position is not within the range of the value and at least one of the positions is within the range of the reference value may be stored as the history image.

Next, a second embodiment will be described.
The second embodiment is an example in which the person recognition device described in the first embodiment is applied to a traffic control device that recognizes a passer's face image and controls the traffic of the passer.

FIG. 6 schematically shows the configuration of the traffic control device according to the second embodiment. This traffic control device, for example, manages entry / exit to important facilities (such as security-oriented rooms), recognizes the face image of the user (passerby), and enters the important facility based on the recognition result. Open / close control of the exit door is performed, and the entrance / exit door 202 of the important facility 201 is opened / closed according to the recognition results of the camera 101, the illumination unit 102, the input / output unit 103, the processing unit 104, and the recognition unit 109. The door control unit 112 is configured as a traffic control means for performing control.
In FIG. 6, the configuration other than the door control unit 112 is the same as that of the person recognition device in FIG. 1.

  When the recognition unit 109 determines that the recognition target person 100 is the person himself / herself (determined that the verification is successful), the recognition unit 109 outputs a “door open” signal to the door control unit 112 and determines that the recognition target person 100 is not the person himself / herself. If it is determined that the verification is successful, a “door closed” signal is output to the door control unit 112.

Upon receiving the “door open” signal from the recognition unit 109, the door control unit 112 controls the entrance / exit door 202 to be in an open state, and permits the recognition target person (passerby in this case) 100 to enter the room. When the “door closed” signal is received, the entrance / exit door 202 is kept closed, and the passer-by 100 is refused to enter the room.
Thus, according to the second embodiment, it is possible to control the passage of passers-by using the person recognition device described in the first embodiment.

As described above, according to the above embodiment, the input face image when the highest similarity is obtained, or the input face image when the detected eye and nose positions are within the reference value range. By saving as a history image, the following effects can be expected.
In other words, the purpose of the history image is to know which person uses the device at which date and time, and the history image at that time is very important information. If the image is not normal, the value as a history image is lost.
Conventionally, the probability that a person's face in the history image is facing the front is very low, but the input face image or detection when the highest similarity is obtained as in this embodiment. Using the input face image when the position of the eyes and nose within the reference value range is used, the probability that the face of the history image is placed in the center of the image and the face is facing the front is high, and recognition The identification of the target person can be made clear.

1 is a block diagram schematically showing the configuration of a person recognition device according to a first embodiment of the present invention. The figure for demonstrating the process of a face detection part. The flowchart for demonstrating the flow of a process of a recognition part. The flowchart for demonstrating the 1st operation example of the recognition process which mainly preserve | saved the log | history image. The flowchart for demonstrating the 2nd operation example of the recognition process which mainly preserve | saved the log | history image. The block diagram which shows schematically the structure of the traffic control apparatus which concerns on the 2nd Embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Recognition person (person, passerby), 101 ... Camera (image input means), 102 ... Illumination part (illumination means), 103 ... Input / output part (input means, display means), 104 ... Processing part, 105 ... Image input unit (image input unit), 106 ... face detection unit (face detection unit), 107 ... feature extraction unit (feature extraction unit), 108 ... dictionary information storage unit (dictionary storage unit), 109 ... recognition unit (recognition unit) , 110 ... Image storage unit (image storage unit), 111 ... Control unit (control unit), 112 ... Door control unit (traffic control unit), 201 ... Important facility, 202 ... Door for entrance / exit.

Claims (4)

An image input means for inputting at least a plurality of face images of a person to be recognized;
First image storage means for temporarily storing a plurality of input face images input by the image input means;
The information based on a plurality of input face images input by the image input means is sequentially compared with dictionary information registered in advance to determine the similarity between the two information. A recognition means for recognizing whether there is,
Similarity storage means for temporarily storing each similarity obtained by the recognition means;
First selecting means for selecting a similarity indicating the highest value from the similarities stored in the similarity storing means based on completion of the recognition processing by the recognizing means;
Second selection means for selecting an input face image from the first image storage means when the highest similarity selected by the first selection means is obtained;
Second image storage means for storing the input face image as the history image when the similarity of the highest value selected by the second selection means is obtained;
A person recognition device comprising: An image input means for inputting at least a plurality of face images of a person to be recognized;
The information based on the plurality of input face images input by the image input means is sequentially compared with dictionary information registered in advance, thereby obtaining the similarity of both information, and the person is identified by the obtained similarity. A recognition means for recognizing whether there is,
Face feature point detection means for detecting the position of at least one of eyes and nose from each of a plurality of input face images input by the image input means;
When the position of at least one of the eyes and nose detected by the face feature point detection means is within a preset reference value range, an input that detects the position of at least one of the eyes and nose A second image storage means for storing the face image as a history image;
A person recognition device comprising: Image input means for inputting at least a plurality of face images of passersby to be recognized;
First image storage means for temporarily storing a plurality of input face images input by the image input means;
The information based on a plurality of input face images input by the image input means is sequentially compared with previously registered dictionary information to obtain the similarity of both information, and the passer-by is identified by the obtained similarity. A recognition means for recognizing whether or not
A passage control means for controlling the passage of the passer based on the recognition result of the recognition means;
Similarity storage means for temporarily storing each similarity obtained in the recognition means;
First selecting means for selecting a similarity indicating the highest value from the similarities stored in the similarity storing means based on completion of the recognition processing by the recognizing means;
Second selection means for selecting an input face image from the first image storage means when the highest similarity selected by the first selection means is obtained;
Second image storage means for storing the input face image as the history image when the similarity of the highest value selected by the second selection means is obtained;
A traffic control device comprising: Image input means for inputting at least a plurality of face images of passersby to be recognized;
The information based on a plurality of input face images input by the image input means is sequentially checked against dictionary information registered in advance to obtain the similarity between the two information, and the passerby is identified by the obtained similarity. A recognition means for recognizing whether or not
A passage control means for controlling the passage of the passer based on the recognition result of the recognition means;
Face feature point detection means for detecting the position of at least one of eyes and nose from each of a plurality of input face images input by the image input means;
When the position of at least one of the eyes and nose detected by the face feature point detection means is within a preset reference value range, an input that detects the position of at least one of the eyes and nose A second image storage means for storing the face image as a history image;
A traffic control device comprising:

Images