KR101444538B1 - 3d face recognition system and method for face recognition of thterof - Google Patents

Abstract

The present invention relates to a three-dimensional face recognition system capable of acquiring a three-dimensional face image by overlapping two cameras, and a face recognition method thereof.
For example, a first camera for photographing a visible light image of a face recognition object; An infrared ray pattern optical device for irradiating the face recognition object with infrared ray pattern light; A second camera for capturing an infrared image of a pattern irradiated on the face recognition target; A mirror for matching the optical axes of the first camera and the second camera; And a control unit for generating a three-dimensional face image of the face recognition object by taking a color image and depth information in which the visible ray image and the infrared image are matched in pixel units from the first camera and the second camera, System.

Description

FIELD OF THE INVENTION [0001] The present invention relates to a 3D face recognition system,

The present invention relates to a three-dimensional face recognition system and a face recognition method thereof.

Generally, authentication systems such as fingerprint recognition, iris recognition, face recognition, etc. are introduced in the home or company due to security problems. Among them, face recognition is performed by extracting feature points from a 2D face image and comparing the feature points with registered images. However, the recognition rate of the image obtained from the illumination of the angle different from that of the registered image and the illumination of the direction different from that of the registered image is considerably deteriorated. To solve this problem, 3D face recognition is being developed. The existing 3D face recognition system extracts feature points from two-dimensional face images and then uses two-dimensional images over the three-dimensional average face models based on the positions of the feature points. However, this method has a disadvantage in that the accuracy of the three-dimensional face is lowered because information on the face type of the object to be authenticated is not obtained. In addition, there is a method of acquiring three-dimensional image by matching the two images after photographing the depth camera and the color camera on the same horizontal axis. However, it is difficult to secure the matching accuracy, and a lot of time and efforts are required in the image processing for matching need. Therefore, a three-dimensional face recognition system that can increase the face recognition rate is required.

The present invention provides a three-dimensional face recognition system capable of acquiring a three-dimensional face image by superimposing two cameras, and a face recognition method thereof.

A three-dimensional face recognition system according to the present invention includes: a first camera for capturing a visible light image of a face recognition object; An infrared ray pattern optical device for irradiating the face recognition object with infrared ray pattern light; A second camera for capturing an infrared image of a pattern irradiated on the face recognition target; A mirror for matching the optical axes of the first camera and the second camera; And a controller for generating a three-dimensional face image of the face recognition object by taking a color image and depth information in which the visible ray image and the infrared image are matched in pixel units from the first camera and the second camera.

The mirror is positioned at an angle inclined 45 degrees from a center of the first camera and a center axis of the second camera vertically intersect. The mirror reflects the visible light of the first camera, May be a cold mirror that transmits infrared rays.

The mirror is disposed at an angle inclined by 45 degrees from a point where the central axis of the first camera intersects the central axis of the second camera vertically. The mirror transmits the visible light of the first camera, May be a hot mirror that reflects.

In addition, the distance from the first camera to the mirror and the distance from the second camera to the mirror may be the same.

In addition, the first camera and the second camera are cameras of the same type, the first camera is equipped with an infrared cut filter, and the second camera is equipped with a visible light cut filter.

The storage unit may further store a registered image of the face recognition target.

In addition, the controller may determine the degree of matching between the current three-dimensional image of the face recognition target and the registered image stored in the storage unit.

In addition, the controller may determine the degree of matching between the position of the feature point of the current 3D image and the position of the feature point of the registered image.

In addition, the controller can determine the degree of matching between the respective characteristics of the visible light ray image and the infrared ray image of the current three-dimensional image, and the respective characteristics of the visible light ray image and the infrared ray image of the registered image.

In addition, the controller may determine the degree of matching between the N 2-dimensional images of the current 3D image and the N 2-dimensional images of the registered image.

According to another aspect of the present invention, there is provided a face recognition method for a 3D face recognition system, comprising: acquiring a visible ray image from a first camera and acquiring an infrared ray image from a second camera; A feature extraction step of extracting a feature of the image acquired in the image acquisition step; A matching degree determining step of determining a matching degree between the feature extracted in the feature extracting step and the feature of the stored registered image; And an authentication step of indicating an authentication result according to the matching degree determined in the matching degree determination step.

In addition, the feature extraction step may extract the positions of the minutiae points from the visible light image and the infrared image. The matching degree determination step may determine a degree of matching between the position of the minutiae point extracted in the feature extracting step and the position of the minutiae point of the stored registered image.

Also, the feature extraction step may extract the features of the visible light image and the features of the infrared image, respectively. The matching degree determination step may determine a matching degree between the characteristic of the visible light ray image extracted in the characteristic extracting step and the characteristic of the infrared ray image, the characteristic of the visible light ray image of the stored registered image, and the characteristic of the infrared ray image.

The feature extracting step may generate a three-dimensional face image from the visible light image and the infrared image, and rotate the three-dimensional face image into N angles to generate N two-dimensional face images. The matching degree determination step may determine the degree of matching between the N 2-dimensional face images extracted at the feature extraction step and the N 2-dimensional face images of the stored registered images.

A 3D face recognition system and a face recognition method thereof according to an embodiment of the present invention include a first camera that captures a visible light image and a second camera that captures an infrared image to obtain a color image and depth information at the same time, 3D face images can be acquired.

In addition, the 3D face recognition system and its face recognition method according to an embodiment of the present invention can improve the face recognition rate by using the feature information of the 3D face image.

1 is a block diagram illustrating a 3D face recognition system in accordance with an embodiment of the present invention.
Fig. 2 is a view showing the infrared ray pattern optical device shown in Fig.
FIGS. 3A and 3B are views showing a structure for matching the optical axes of the first camera and the second camera shown in FIG. 1. FIG.
4 is a flowchart illustrating a face recognition method of a 3D face recognition system according to an embodiment of the present invention.
FIG. 5 is a flowchart illustrating a face recognition method of a 3D face recognition system according to another embodiment of the present invention.
FIG. 6 is a flowchart illustrating a face recognition method of a 3D face recognition system according to another embodiment of the present invention.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention.

1 is a block diagram illustrating a 3D face recognition system in accordance with an embodiment of the present invention. Fig. 2 is a view showing the infrared ray pattern optical device shown in Fig. FIGS. 3A and 3B are views showing a structure for matching the optical axes of the first camera and the second camera shown in FIG. 1. FIG.

1, a 3D face recognition system 100 according to an embodiment of the present invention includes a first camera 110, an infrared pattern optical device 120, a second camera 130, a mirror 140, A control unit 150 and a storage unit 160.

The first camera 110 captures a visible light image of the face recognition object 10. The color image of the face recognition object 10 can be obtained through the visible light image photographed by the first camera 110. That is, the color image acquired through the first camera 110 is a two-dimensional face image. The first camera 110 may transmit the visible light image of the face recognition object 10 through the mirror 140. The first camera 110 may reflect the visible light image of the face recognition object 10 by being reflected by the mirror 140. The first camera 110 may be equipped with a visible ray camera or a general camera with an infrared cut filter.

The infrared pattern optical device 120 irradiates the face recognition object 10 with infrared ray pattern light. 2, the infrared ray pattern light irradiated by the infrared ray pattern optical device 120 is irradiated to the front face of the face recognition object 10. [ Here, the infrared pattern light may be a horizontal line pattern, a vertical line pattern, a lattice pattern, or the like at equal intervals. When the infrared pattern light is irradiated on the face recognition object 10, the interval, thickness, and brightness of the pattern are different depending on the bending of the face. The infrared ray pattern light emitted from the infrared ray pattern optical device 120 is photographed through the second camera 130.

The second camera 130 captures an infrared image of the face recognition object 10. Here, the second camera 130 captures an infrared image of a pattern irradiated on the face recognition object 10 from the infrared pattern optical device 120. In the infrared image captured through the second camera 130, information such as a pattern distribution and brightness according to the face shape of the face recognition target 10 is stored. At this time, since the interval and thickness of the pattern are inversely proportional to the distance, the distance d to each part of the face recognition object 10 can be measured from the infrared pattern optical device 120 through the following equation.

[Mathematical Expression]

d = 1 / (am + b)

Here, m represents the interval or thickness of the pattern, and a and b represent coefficients. As described above, the depth information of the face recognition object 10 can be obtained through the infrared image captured by the second camera 130. The second camera 130 may transmit the infrared ray image of the face recognition object 10 through the mirror 140. [ The second camera 130 may reflect the infrared image of the face recognition target 10 by being reflected by the mirror 140. The second camera 130 may use an infrared ray camera or a general camera with a visible ray blocking filter.

The mirror 140 serves to align the optical axes of the first camera 110 and the second camera 130. The mirror 140 is positioned at an angle of 45 degrees with respect to a center axis of the first camera 110 and a center axis of the second camera 130 perpendicularly intersect with each other. The distance D1 from the mirror 140 to the first camera 110 is the same as the distance D2 from the mirror 140 to the second camera 130 (D1 = D2).

Referring to FIG. 3A, a method of matching the optical axis of the mirror 140 between the first camera 110 and the second camera 130 will be described.

3A, the second camera 130 is positioned on the front face of the face recognition object 10, and a mirror 140 is positioned between the second camera 130 and the face recognition object 10 Position at an angle of 45 degrees. Next, the first camera 110 is positioned below the mirror 140. Here, the mirror 140 may be a cold mirror that transmits infrared light and reflects visible light. Accordingly, the cold mirror 140 transmits the infrared rays of the horizontally disposed second camera 130 and reflects the visible light of the vertically disposed first camera 110, 2 camera 130 can be made to coincide with each other. In addition, the first camera 110 and the second camera 130 use cameras of the same type, and the first camera 110 is equipped with an infrared cut filter and the second camera 130 is provided with a visible light cut filter The first camera 110 and the second camera 130 can have the same resolution and the same angle of view.

In addition, as shown in FIG. 3B, a cold mirror 140 'may be replaced with a hot mirror 140' that reflects infrared light and transmits visible light. In this case, the positions of the first camera 110 and the second camera 130 should be changed. That is, the first camera 110 is positioned on the front face of the face recognition object 10, the hot mirror 140 'is positioned between the first camera 110 and the face recognition object 10, And the second camera 130 is positioned below the hot mirror 140 '. Accordingly, the hot mirror 140 'transmits the visible light of the horizontally arranged first camera 110 and reflects the infrared rays of the vertically arranged second camera 130, The optical axis of the second camera 130 can be matched.

The control unit 150 generates a three-dimensional face image of the face recognition object 10 by combining the visible light image and the infrared image captured by the first camera 110 and the second camera 130. Here, the visible light image and the infrared image are registered in units of pixels. The control unit 150 may obtain a color image through the visible light image captured by the first camera 110 and may obtain the depth information through the infrared image captured by the second camera 130. That is, the controller 150 takes the color image and the depth information, which are obtained by matching the visible light image and the infrared image from the first camera 110 and the second camera 130, in pixel units, A three-dimensional face image can be generated.

In addition, the controller 150 determines the degree of matching between the currently generated three-dimensional face image and the stored registered image to determine whether or not the user is authenticated. Hereinafter, the currently generated three-dimensional face image will be referred to as a current image. The controller 150 may determine whether the user is authenticated by determining the degree of matching between the position of the feature point of the current image and the position of the feature point of the registered image. Here, the feature point refers to a specific region such as an eye, a nose, a mouth, an ear, and the like. In addition, the controller 150 may determine whether the user is authenticated by determining the degree of matching between the respective characteristics of the visible light image and the infrared image in the current image and the respective characteristics of the visible light image and the infrared image in the registered image. In addition, the controller 150 rotates the current image and the registered image equally at N angles to create N 2-dimensional images, and calculates the matching between the N 2-dimensional images of the current image and the N 2-dimensional images of the registered images It is possible to determine whether or not the user is authenticated (N is a constant).

The storage unit 160 stores a registered image of the face recognition target 10. That is, the storage unit 160 previously registers and stores images of a plurality of face recognition objects 10 in advance. In addition, the storage unit 160 stores the feature points of the registered image, the characteristics of each of the visible light image and the infrared image, and N 2-dimensional images.

As described above, the three-dimensional face recognition system 100 according to an embodiment of the present invention includes a first camera 110 for capturing a visible light image and a second camera 130 for capturing an infrared image, By acquiring depth information at the same time, 3D face images can be obtained.

Next, a face recognition method of the 3D face recognition system will be described.

4 is a flowchart illustrating a face recognition method of a 3D face recognition system according to an embodiment of the present invention.

Referring to FIG. 4, the face recognition method of the 3D face recognition system includes an image acquisition step S10, a feature extraction step S20, a matching degree determination step S30, and an authentication step S40.

The image acquisition step S10 is a step of acquiring a visible ray image and an infrared ray image of the face recognition object 10 photographed from the first camera 110 and the second camera 130. [ In step S11, the control unit 150 acquires a visible light image from the first camera 110 (S11) and acquires an infrared image from the second camera 130 (S12). At this time, the controller 150 may extract depth information from the infrared image (S13).

The feature extracting step S20 is a step of extracting the positions of the minutiae points from the visible light ray image and the infrared ray image acquired in the image acquiring step S10. In step S21, the controller 150 extracts feature points such as an eye, a nose, a mouth, an ear, and the like, and normalizes the feature points to a three-dimensional position using a magnitude, an angle, . Accordingly, in the feature extraction step S20, the three-dimensional position of the feature point can be extracted.

The matching degree determination step S30 is a step of determining the matching degree between the feature extracted in the feature extracting step S20 and the feature of the stored registered image. In the matching degree determination step S30, the controller 150 determines how the positions of the minutiae points extracted in the feature extraction step S20 match the positions of the minutiae points of the registered image stored in the storage unit 160. [ Here, the storage unit 160 stores three-dimensional registered images of a plurality of face recognition objects 10.

The authentication step S40 is a step of indicating an authentication result according to the degree of matching determined in the matching degree determination step S30. In the authentication step S40, the controller 150 indicates that the authentication is successful if the matching degree is equal to or greater than the reference value, and indicates that the authentication is unsuccessful if the matching value is less than the reference value.

As described above, the face recognition method of the 3D face recognition system according to an embodiment of the present invention uses the feature information of the 3D face image, thereby improving the face recognition rate.

In addition, the face recognition method of the 3D face recognition system according to an embodiment of the present invention simultaneously acquires a visible light ray image and an infrared ray image through the first camera 110 and the second camera 130, The same forged face can be easily judged.

FIG. 5 is a flowchart illustrating a face recognition method of a 3D face recognition system according to another embodiment of the present invention.

Referring to FIG. 5, the face recognition method of the 3D face recognition system includes an image acquisition step S10, a feature extraction step S120, a matching degree determination step S130, and an authentication step S40. The face recognition method shown in FIG. 5 differs from the face recognition method shown in FIG. 4 only in the feature extraction step S120 and the matching degree determination step S130, and the remaining steps are the same. Therefore, only the feature extraction step S120 and the matching degree determination step S130 will be described below.

The characteristic extracting step S120 is a step of extracting characteristics of each of the visible light ray image and the infrared ray image acquired in the image acquiring step S10. In the feature extraction step S120, the controller 150 extracts features of the visible light image (S121), and extracts features of the infrared image (S122). Then, the controller 150 fuses the respective features with each other (S123).

The matching degree determination step S130 is a step of determining a matching degree between the feature extracted in the feature extracting step S120 and the feature of the stored registered image. In the matching degree determination step S130, the controller 150 determines whether the characteristic of the visible light ray image and the characteristic of the infrared ray image extracted in the characteristic extracting step S120 is the characteristic of the visible light ray image of the registered image stored in the storage unit 160 And the characteristics of the infrared image.

FIG. 6 is a flowchart illustrating a face recognition method of a 3D face recognition system according to another embodiment of the present invention.

Referring to FIG. 6, the face recognition method of the 3D face recognition system includes an image acquisition step S10, a feature extraction step S220, a matching degree determination step S230, and an authentication step S40. The face recognition method shown in FIG. 6 differs from the face recognition method shown in FIG. 4 only in the feature extraction step S220 and the matching degree determination step S230, and the remaining steps are the same. Therefore, only the feature extraction step S220 and the matching degree determination step S230 will be described below.

The feature extracting step S220 is a step of generating a three-dimensional face image from the visible light ray image and the infrared ray image acquired in the image acquiring step S10, and generating the two-dimensional two-dimensional face image again. In the feature extraction step S220, the controller 150 generates a three-dimensional face image from the visible light image and the infrared image (S221), rotates the three-dimensional face image at N angles, (S222).

The matching degree determination step S230 is a step of determining a matching degree between the feature extracted in the feature extracting step S220 and the feature of the stored registered image. In the matching degree determination step S230, the controller 150 compares the N 2-dimensional face images extracted in the feature extraction step S220 with the N 2-dimensional face images of the registered images stored in the storage unit 160 .

The present invention is not limited to the above-described embodiments, and various modifications and changes may be made by those skilled in the art without departing from the spirit and scope of the present invention. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined in the appended claims.

100: 3D face recognition system 110: First camera
120: infrared pattern optical device 130: second camera
140: mirror 150:

Claims (17)

A first camera for capturing a visible light image of a face recognition target;
An infrared ray pattern optical device for irradiating the face recognition object with infrared ray pattern light;
A second camera for capturing an infrared image of a pattern irradiated on the face recognition target;
A mirror for matching the optical axes of the first camera and the second camera; And
And a controller for generating a three-dimensional face image of the face recognition object by taking a color image and depth information obtained by matching the visible light image and the infrared image from the first camera and the second camera in pixel units,
Wherein the mirror is positioned at an angle inclined by 45 degrees at a point where the central axis of the first camera and the center axis of the second camera intersect vertically, and the mirror reflects the visible light of the first camera, Dimensional face recognition system. delete A first camera for capturing a visible light image of a face recognition target;
An infrared ray pattern optical device for irradiating the face recognition object with infrared ray pattern light;
A second camera for capturing an infrared image of a pattern irradiated on the face recognition target;
A mirror for matching the optical axes of the first camera and the second camera; And
And a controller for generating a three-dimensional face image of the face recognition object by taking a color image and depth information obtained by matching the visible light image and the infrared image from the first camera and the second camera in pixel units,
Wherein the mirror is positioned at an angle inclined by 45 degrees from a point at which the central axis of the first camera intersects with the center axis of the second camera vertically and transmits the visible light of the first camera, Wherein the first mirror is a hot mirror for reflecting light. The method according to claim 1 or 3,
Wherein the distance from the first camera to the mirror and the distance from the second camera to the mirror are the same. The method according to claim 1 or 3,
Wherein the first camera and the second camera are cameras of the same model,
The first camera is equipped with an infrared cut filter,
Wherein the second camera is equipped with a visible light blocking filter. The method according to claim 1 or 3,
And a storage unit for storing the registered image of the face recognition target. The method according to claim 6,
Wherein the control unit determines whether the user is authenticated by determining a degree of matching between the current three-dimensional image of the face recognition target and the registered image stored in the storage unit. 8. The method of claim 7,
Wherein the controller determines a degree of matching between a position of the feature point of the current 3D image and a position of the feature point of the registered image. 8. The method of claim 7,
Wherein the control unit determines the degree of matching between each feature of the visible light ray image and the infrared ray image of the current three-dimensional image, and the respective characteristics of the visible light ray image and the infrared ray image of the registered image. 8. The method of claim 7,
Wherein the control unit determines the degree of matching between the N 2-dimensional images of the current 3D image and the N 2-dimensional images of the registered image. An image acquiring step of acquiring a visible ray image from the first camera and acquiring an infrared ray image from the second camera;
A feature extraction step of extracting a feature of the image acquired in the image acquisition step;
A matching degree determining step of determining a matching degree between the feature extracted in the feature extracting step and the feature of the stored registered image; And
And an authentication step of indicating an authentication result according to the matching degree determined in the matching degree determination step,
Wherein the mirror captures a visible ray of the first camera at a position at which the center axis of the first camera and the center axis of the second camera intersect at right angles, Wherein the second camera is a cold mirror for transmitting infrared rays of the second camera. 12. The method of claim 11,
Wherein the feature extraction step extracts the positions of the minutiae points from the visible light image and the infrared image. 13. The method of claim 12,
Wherein the matching degree determining step determines a degree of matching between a position of a minutiae point extracted in the feature extracting step and a minutiae point of a stored registered image. 12. The method of claim 11,
Wherein the feature extraction step extracts features of the visible light image and features of the infrared image, respectively. 15. The method of claim 14,
Wherein the matching degree determination step determines the degree of matching between the characteristics of the visible light ray image extracted in the characteristic extraction step and the characteristics of the infrared ray image, the characteristics of the visible light ray image of the stored registered image, and the characteristics of the infrared ray image. Face Recognition Method of 3D Face Recognition System. 12. The method of claim 11,
Wherein the feature extraction step generates a three-dimensional face image from the visible light image and the infrared image, and rotates the three-dimensional face image at N angles to generate N two-dimensional face images, A method of face recognition of a system. 17. The method of claim 16,
Wherein the matching degree determination step determines the degree of matching between the N 2-dimensional face images extracted in the feature extraction step and N 2-dimensional face images of the stored registered images, .

Images