JP2007156974A - Personal identification/discrimination system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a personal identification/discrimination device with a lower erroneous discrimination rate, for integrating a face image and voice to prevent impersonation using a face picture and recorded voice. <P>SOLUTION: The face image and voice of a user outputted from an imaging section 1 are separated with each other. A face image collation section 14 collates an amount of feature extracted from a static image of the face image and pre-registered personal face image data 15 by a face image feature amount extraction section 12, and outputs its likelihood α. A voice collation section 23 collates an amount of feature extracted from the voice and the preregistered personal voice data 24 by a voice feature amount extraction section 22, and outputs its likelihood β. A likelihood integrating section 4 outputs the integrated likelihood by integrating the likelihoods α and β, and an identification/discrimination determining section 6 identifies and/or discriminates the user on the basis of the integrated likelihoods. The likelihoods α and β can vary their thresholds on the basis of brightness of the face image from a brightness measuring section 16 and the magnitude of background noises from a background noise measuring section 25. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a personal authentication / identification system, and more particularly to a personal authentication / identification system capable of performing personal authentication and identification with high accuracy by integrating facial images and sound.

  Authentication methods using biometric information (biometrics) are being studied for entrance / exit management and access control to information systems. Examples of biometric information that can be used for personal authentication and identification include fingerprints, faces, voices, irises, retinas, and palm veins.

As one of conventional personal authentication devices, for example, there is one described in Patent Document 1 below. The personal authentication device described in this document collates a face image of a specific individual stored in advance with a captured face image, and authenticates if both patterns are approximated, and stores in advance Compare the waveform of the voice of the specified individual and the input voice, and verify the voice identification unit that authenticates if both are approximated, the keyword or ID number stored in advance, and the input keyword or ID number The ID authentication unit that authenticates if both match, and the fingerprint authentication unit that verifies the pattern of a specific personal fingerprint stored in advance and the input fingerprint, and authenticates if both match A combination of two or more of these is used for personal authentication, and when all used are authenticated, it is authenticated as a registered individual.
JP 2000-259828 A

  However, in the above-described prior art, the problem that the image identification unit can be misidentified when the brightness or the angle of the face changes or can be misrepresented by the person's face photograph, and the voice identification unit misidentifies that the background noise is large. There is a problem that it can be spoofed by the voice of the recorded person.

  SUMMARY OF THE INVENTION An object of the present invention is to solve the above-described problems of the prior art and provide a personal authentication / identification device that integrates face images and voices that have a low misidentification rate and that cannot be spoofed by facial photographs or recorded voices. There is.

  In order to achieve the above-described object, the present invention provides a personal authentication / identification system that integrates a face image and sound, an imaging unit that captures a moving image including at least a user's face image, and the moving image. A voice / image separation unit that separates audio and images, a personal face image registration data unit that registers personal face image data in advance, a personal voice registration data unit that registers personal voice data in advance, A facial image feature amount extracting unit that extracts a feature amount of a face image from a still image of the face image separated by the sound / image separation unit, and a sound feature from the sound data separated by the sound / image separation unit. A voice feature amount extraction unit that extracts a quantity, a lip movement detection unit that detects lip movement from the facial image feature amount extracted by the face image feature amount extraction unit, and a voice / image separation unit. Detecting speech segment from speech A voice segment detection unit; a voice feature amount extraction unit that extracts a voice feature amount from voice data detected by the voice segment detection unit; and lip movement time information detected by the lip movement detection unit; Voice segment / lip synchronization verification unit for verifying synchronization between lips and voice from time information of the voice segment detected by the voice segment detection, facial image feature amount extracted by the facial image feature amount extraction unit, and the individual A face image collation unit that collates personal face image data registered in the face image registration data unit and outputs a likelihood α representing personality, and a voice feature amount extracted by the voice feature amount extraction unit And the personal voice data registered in the personal voice registration data part, the voice collation part for outputting the likelihood β representing the identity, and the face image collation part and the voice collation part notified from the voice collation part Combines likelihood α and β It is characterized in that it comprises a likelihood integration unit for authentication and an authentication / identification determination unit for authenticating and identifying a user based on the integrated likelihood integrated by the likelihood integration unit.

  Further, when the movement of the lips is not detected by the voice section / lip synchronization detection unit, when there is no overlap between the movement of the lips and the voice section, or when the ratio of the lip movement section at the same time as the voice section is equal to or less than the threshold value, Another feature is that the authentication / identification determination unit does not perform personal authentication / identification processing.

  According to the present invention, both the authentication / identification by the face image and the authentication / identification by voice taking account of the movement of the lips are used in the entrance / exit management system, the access management to the information system, etc. High accuracy is obtained compared to authentication and identification.

  Further, even when used in a place where there is a change in brightness or where background noise is large, user authentication / identification can be performed with high accuracy. For this reason, the environment in which the authentication / identification system of the present invention can be used is expanded.

  In addition, since the moving image of the face can be recorded / saved in the moving image recording unit, there is an effect of psychologically suppressing spoofing. In addition, it is difficult to misrepresent a photograph or recorded sound, and the reliability can be improved.

  Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention.

  The video camera 1 is provided in a place where entrance / exit management, access management to an information system, or the like, or in an installation place of an automatic money deposit / payment machine or the like of a financial system. When performing personal recognition / identification, the user is given a predetermined instruction, for example, “Please say your name”. Then, the video camera 1 takes a picture of the user speaking his / her name.

  The audio / image separating unit 2 obtains still image data and audio data of the face from the moving image data output from the video camera 1. The still image data is sent to the face image authentication / identification unit 10, while the voice data is sent to the voice authentication / identification unit 20. The moving image data is recorded in the moving image recording unit 3.

  As shown in FIG. 2, the moving image recording unit 3 determines whether or not a moving image has been recorded for a predetermined time (step S1). If this determination is affirmative, an authentication / identification determination is made. A determination OK (step S2) is output to the unit 6, and if it is negative, a determination p (NG) signal p is output (step S3).

  The face image authentication / identification unit 10 includes a face still image acquisition unit 11, a face image feature amount extraction unit 12, a lip movement detection unit 13, a face image collation unit 14, an individual face image registration data 15, and a brightness measurement unit 16. It is configured. The voice authentication / identification unit 20 includes a voice section detection unit 21, a voice feature quantity extraction unit 22, a voice collation unit 23, personal voice registration data 24, and a background noise measurement unit 25.

  The face still image acquisition unit 11 of the face image authentication / identification unit 10 acquires a still image of the face from the sound / image separation unit 2. The face image feature amount extraction unit 12 extracts a feature amount from the still image of the face and sends it to the face image collation unit 14. The face image collation unit 14 collates with the personal face image registration data 15 registered in advance, and outputs a score (likelihood) α representing the identity. Further, the brightness measuring unit 16 measures the brightness of the still image data of the face and outputs data q related to the brightness to the likelihood integrating unit 4.

  Note that the face image matching unit 14, as shown in FIG. 3, when the lip movement detecting unit 13 determines that the lip movement is detected (step S 4 is affirmative) and / or the voice section detecting unit 21 If it is determined to be a section (step S5 is affirmative), face image matching is forgotten (step S6), and face image matching is performed using face still images of sections other than these sections (step S7).

  As a result, the face image matching unit 14 can perform the matching process using the feature amount of the completely stationary face image, so that the accuracy of the face image matching is improved.

  On the other hand, when voice is input from the voice / image separation unit 2, the voice section detection unit 21 of the voice authentication / identification unit 20 detects the voice section and sends the voice in the voice section to the voice feature quantity extraction unit 22. The voice feature amount extraction unit 22 extracts a voice feature amount of a voice section, and the voice collation unit 23 performs collation with the personal voice registration data 24 registered in advance, and a score (likelihood) representing personality. β is output.

  The voice section / lip synchronization verification unit 5 receives the voice section (time information) from the voice section detection unit 21 and the lip movement detection section (time information) from the lip movement detection unit 13 and compares the two sections. Then, the authentication / identification determination unit 6 is controlled as shown in FIG. First, in step S11, it is determined whether or not lip movement is detected. If this determination is affirmative, the determination in step S12 is performed. If the determination is negative, the process proceeds to step S14. In step S12, it is determined whether or not the ratio of the time overlapped with the lip movement detection section in the voice section is equal to or greater than a threshold value. If this determination is affirmative, the process proceeds to step S13, and an authentication / identification enable signal t is sent to the authentication / identification determination unit 6. On the other hand, when the determination in step S12 is negative, the process proceeds to step S14 to send an authentication / identification impossible (NG) signal t.

  As a result, the movement of the lips is not detected in the person's face photo, and the movement of the lips in the recorded person's voice is not a voice section or the movement of the lips and the voice section are not compatible. You can't misrepresent yourself with a photo of your face or your voice.

  In addition, if a lip position detecting means (not shown) is additionally provided to detect the position of the user's lips, an instruction can be given to bring the mouth closer to the position of the microphone (not shown). it can. In this way, the user's voice can be collected accurately.

  Next, the operation of the likelihood integration unit 4 will be described with reference to FIG. The likelihood integration unit 4 combines the score α from the face image authentication / identification unit 10 and the score β from the voice authentication / identification unit 20 in order to obtain a personality score used for determination of authentication / identification (step S21). . Next, it is determined whether or not the brightness is insufficient (step S22). When the brightness information q from the brightness measuring unit 16 indicates insufficient brightness (Yes in step S22), a predetermined threshold value Sa for the score α is decreased by a predetermined value, and the score β The predetermined threshold value Sb is increased by a predetermined value (step S23). Next, the scores α and β are compared with the decreased Sa and the increased Sb, respectively, and it is determined whether or not the scores α ≧ Sa and the score β ≧ Sb are satisfied (step S24). Returns the threshold values Sa and Sb to the original values (step S25). On the other hand, if not established, the integrated likelihood γ is set to 0, for example (step S30).

  Next, it is determined whether or not the background noise information r from the background noise measuring unit 25 is larger than a predetermined value (step S26). If it is larger, the threshold value Sa for the score α is set. The threshold value Sb for the score β is decreased by a predetermined value by increasing it by a predetermined value (step S27). The scores α and β are compared with the increased Sa and the decreased Sb, respectively, and it is determined whether or not the scores α ≧ Sa and β ≧ Sb are satisfied (step S28). If the integrated likelihood γ = 1 and not established, the process proceeds to step S30 where the integrated likelihood γ = 0 is output to the authentication / identification determining unit 6. Here, γ = 1 is OK, and γ = 0 is NG.

  If the brightness is not insufficient (determination in step S22 is negative) and the background noise is smaller than a predetermined value (determination in step S26 is negative), the process proceeds to step S28, where the scores α and β are the original values. Are compared with threshold values Sa and Sb. And if judgment of step S28 becomes affirmation, it will progress to step S29, and conversely, if it becomes negative, it will progress to step S30.

  The authentication / identification determination unit 6 determines personal authentication or identifies individuals from a plurality of users. As shown in FIG. 6, the authentication / identification determination unit 6 determines whether or not the NG input p has arrived from the moving image recording unit 3 in step S31. If this determination is affirmative, the process proceeds to step S36. The integrated likelihood is discarded (γ = 0). When this determination is negative, the process proceeds to step S32, where it is determined whether or not an NG input t has arrived from the voice interval / lip synchronization verifying unit 5, and when this determination is affirmative, the process proceeds to step S36. The integrated likelihood γ is discarded (γ = 0).

  Next, when the determination in step S32 is negative, the process proceeds to step S33 to determine whether to perform individual recognition or to identify individuals from multiple users. In the former case, the process proceeds to step S34, where it is determined whether or not the integrated likelihood γ is 1. If this determination is affirmative, the personal recognition result is accepted as valid. However, if the combined likelihood γ = 0, the process proceeds to step S36. Next, when the determination in step S33 is identification of individuals from a plurality of users, the process proceeds to step S37 to collect collation results with the registration data of the plurality of users, and in step S38, the most integrated likelihood (score). A user with a high ID is used as an identification result.

  The preferred embodiment of the present invention has been described above, but the present invention is not limited to this, and it is obvious that various modifications can be made within the scope of the gist of the present invention.

It is a block diagram which shows the structure of one Embodiment of this invention. It is a flowchart which shows the function of a moving image recording part. It is a flowchart which shows the function of a face image collation part. It is a flowchart which shows the function of an audio | voice area and lip synchronization verification part. It is a flowchart which shows the function of a likelihood integration part. It is a flowchart which shows the function of an authentication and identification determination part.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Video camera, 2 ... Audio | voice / image separation part, 3 ... Moving image recording part, 4 ... Likelihood integration part, 5 ... Voice section and lip synchronization verification part, 6. Authentication / identification determination unit, 10... Face image authentication / identification unit, 20.

Claims (8)

In a personal authentication / identification system that integrates facial images and audio,
An imaging unit that captures a moving image including at least a user's face image;
A sound / image separation unit for separating sound and image from the moving image;
A personal face image registration data section for registering personal face image data in advance;
A personal voice registration data section for registering personal voice data in advance;
A face image feature amount extraction unit that extracts a feature amount of a face image from a still image of the face image separated by the sound / image separation unit;
Lip movement detection unit for detecting lip movement from the feature value of the face image extracted by the face image feature value extraction unit;
A voice section detector for detecting a voice section from the voice separated by the voice / image separator;
A voice feature quantity extraction unit that extracts a voice feature quantity from the voice data detected by the voice section detection unit;
A voice interval / lip synchronization verification unit that verifies the synchronization of the lip and the voice from the time information of the lip movement detected by the lip movement detection unit and the time information of the voice interval detected by the voice interval detection;
The face which outputs the likelihood α representing personality by comparing the feature amount of the face image extracted by the face image feature amount extraction unit with the personal face image data registered in the personal face image registration data unit An image matching unit;
A speech collation unit that collates the speech feature amount extracted by the speech feature amount extraction unit with the personal speech data registered in the personal speech registration data unit, and outputs a likelihood β representing personality;
A likelihood integration unit that combines the likelihoods α and β notified from the face image verification unit and the voice verification unit;
A personal authentication / identification system comprising: an authentication / identification determination unit that authenticates / identifies a user based on the integrated likelihood integrated by the likelihood integration unit. The personal authentication / identification system according to claim 1,
When the movement of the lips is not detected by the voice section / lip synchronization detection unit, or when there is no overlap between the movement of the lips and the voice section, the authentication / identification determination unit does not perform the authentication / identification process. A personal authentication and identification system. The personal authentication / identification system according to claim 1,
The authentication / identification determination unit is configured to prevent the authentication / identification determination unit from performing personal authentication / identification processing when the ratio of the lip movement interval at the same time as the voice interval is equal to or less than a threshold in the voice interval / lip synchronization detection unit. Personal authentication / identification system. The personal authentication / identification system according to claim 1,
A personal authentication / identification system, wherein a face image collation unit excludes a face image of a lip movement segment and a voice segment detected by the voice segment / lip synchronization detection unit from a collation target. The personal authentication / identification system according to claim 1,
Furthermore, it has a lip position detection means,
A personal authentication / identification system characterized by instructing a user to correct the position of a lip. The personal authentication / identification system according to any one of claims 1 to 5,
Furthermore, a moving image recording unit for recording a moving image of a face is provided,
A personal authentication / identification system in which the authentication / identification determination unit does not perform personal authentication / identification processing when the time of a recorded moving image of a face is equal to or less than a threshold value. The personal authentication / identification system according to claim 1,
And a brightness measuring unit that measures the brightness of the still image of the face image,
If the brightness measured by the brightness measurement unit is less than or equal to a threshold value, the authentication / identification determination unit lowers the threshold value for the likelihood α of face image matching, and instead increases the threshold value for the likelihood β of speech matching. A personal authentication / identification system. The personal authentication / identification system according to any one of claims 1 to 6,
Furthermore, a background noise measuring unit for measuring the background noise of speech is provided,
If the background noise level measured by the background noise measuring unit is equal to or higher than the threshold, the authentication / identification determining unit lowers the threshold for the likelihood β of speech matching, and instead increases the threshold for the likelihood α of face image matching. A personal authentication / identification system.

Images