KR20230075770A - Face recognition system and method for controlling the same - Google Patents

Abstract

The present invention relates to a method and system for authenticating a user. A control method for a face recognition system according to the present invention may comprise the steps of: obtaining an image using a camera provided in a terminal that performs a face recognition function; recognizing a face from the image based on a preset face recognition algorithm; checking whether the recognized face in the image satisfies preset face conditions; requesting user authentication for the recognized face to a preset server in order to perform user authentication for the recognized face based on the result of the check; and performing control according to the user authentication result for the recognized face based on the user authentication result for the recognized face received from the server. Accordingly, a user can more conveniently and intuitively transmit his or her face to the terminal.

Description

Face recognition system and control method for face recognition system {FACE RECOGNITION SYSTEM AND METHOD FOR CONTROLLING THE SAME}

The present invention relates to a system for recognizing a user's face and a method for controlling the same in order to perform user authentication.

As technology develops, utilization of terminals is increasing, and in particular, terminals are used as a means of authenticating users in various situations.

For example, user identification at an access control device installed in an access control space (e.g. access control gate or speed gate), attendee verification at a conference, patient visit verification at a hospital, user verification for payment at a cafe As such, there is a need to authenticate genuine users in various situations. In this way, the use of the terminal in a situation requiring user authentication has the effect of reducing unnecessary labor and, furthermore, reducing the time consumed for user authentication.

There are various user authentication methods using terminals. For example, identification information (eg, phone number, e-mail address, etc.) or biometric information (eg, fingerprint information, etc.) There may be a method in which authentication is performed by inputting into a terminal.

Recently, active research is being conducted on a method for quickly and accurately recognizing a user's face in order to perform user authentication without the user directly applying information to the terminal.

An object of the present invention is to provide a face recognition system that more quickly and accurately recognizes the face of a user attempting user authentication, and a method for controlling the face recognition system.

Furthermore, the present invention is to provide a face recognition system and a method for controlling the face recognition system for recognizing a user's intention on whether to perform user authentication.

Furthermore, the present invention is to provide a face recognition system and a control method capable of completing user authentication for a user who truly intends to perform user authentication.

In order to solve the problems described above, a control method of a face recognition system according to the present invention includes acquiring an image using a camera provided in a terminal performing a face recognition function, based on a preset face recognition algorithm, Recognizing a face from an image, checking whether the recognized face in the image satisfies a preset face condition, and performing user authentication on the recognized face based on the result of the check, Requesting user authentication for the recognized face to a set server, and performing control according to the user authentication result for the recognized face based on the user authentication result for the recognized face received from the server. can include

Furthermore, the face recognition system according to the present invention recognizes a face from the image based on a camera unit acquiring an image, a storage unit storing the acquired image, and a preset face recognition algorithm, and the recognition is performed in the image. and a controller that checks whether the recognized face satisfies a preset face condition, wherein the controller performs user authentication on the recognized face based on a result of the check, so as to perform user authentication on the recognized face. User authentication for a face may be requested, and based on the user authentication result for the recognized face received from the server, control according to the user authentication result for the recognized face may be performed.

Furthermore, the face recognition system according to the present invention recognizes a face from the image based on a camera unit acquiring an image, a storage unit storing the acquired image, and a preset face recognition algorithm, and the recognition is performed in the image. A controller for checking whether the recognized face satisfies a preset face condition, including a server that performs user authentication on the recognized face when the recognized face satisfies the preset face condition, wherein the controller includes: Based on the user authentication result for the recognized face performed in , control according to the user authentication result for the recognized face may be performed.

Furthermore, the program, which is executed by one or more processes in the electronic device according to the present invention and stored in a computer-readable recording medium, acquires an image using a camera provided in a terminal that performs a face recognition function; Recognizing a face from the image based on a preset face recognition algorithm, checking whether the recognized face in the image satisfies a preset face condition, and based on a result of the check, the recognized face In order to perform user authentication for , requesting user authentication for the recognized face to a preset server and based on the user authentication result for the recognized face received from the server, It may include instructions for performing a step of performing a control according to a user authentication result.

As described above, the face recognition system and its control method according to the present invention acquires an image using a camera provided in a terminal that performs face recognition, and detects a face from the image based on a preset face recognition algorithm. Recognizable. As such, the present invention can accurately and quickly recognize a user's face based on the location of the user around the terminal even without receiving a direct input from the user. Accordingly, the user can more conveniently and intuitively transfer his/her face to the terminal by being located near the terminal.

Furthermore, the face recognition system and method for controlling the face recognition system according to the present invention may determine whether a face recognized in an image is suitable for performing user authentication. That is, in the present invention, by checking whether a face recognized in an image satisfies a preset face condition, when a face unsuitable for user authentication is recognized, user authentication is not performed, thereby improving data processing efficiency. there is.

Furthermore, the face recognition system and method for controlling the same according to the present invention perform user authentication only for users who satisfy a preset face condition, so that even when there are a plurality of users around the terminal, the user's face attempting user authentication is performed. can be accurately recognized to perform user authentication.

Furthermore, the face recognition system and method for controlling the face recognition system according to the present invention may request user authentication for a face recognized by a preset server based on whether a face recognized in an image satisfies a preset face condition. As described above, the face recognition system and method for controlling the face recognition system according to the present invention perform user authentication on the face of a user who truly wants user authentication, so that the user who has not attempted user authentication or the user's surroundings who have attempted user authentication It is possible to improve the accuracy and reliability of user authentication results by reducing errors in which user authentication is performed for other users in .

1 is a conceptual diagram for explaining a face recognition system according to the present invention.
2 is a block diagram for explaining a face recognition system according to the present invention.
3 is a flowchart for explaining a control method of a face recognition system according to the present invention.
4, 5, 6, 7a, 7b, 8, 9, 10, 11, 12, and 13 are conceptual diagrams for explaining a face recognition system and a control method thereof according to the present invention. .

Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings, but the same reference numerals will be assigned to the same or similar components regardless of reference numerals, and overlapping descriptions thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used together in consideration of ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves. In addition, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiment disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, the technical idea disclosed in this specification is not limited by the accompanying drawings, and all changes included in the spirit and technical scope of the present invention , it should be understood to include equivalents or substitutes.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as "comprise" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

The present invention, even without receiving a direct input from the user, accurately and quickly recognizes the user's face, and identifies the user's face that satisfies a preset face condition in order to determine the user's true intention regarding whether to perform user authentication. As for providing a face recognition system and a control method of the face recognition system that can be recognized, the present invention will be described in detail below with accompanying drawings. 1 is a conceptual diagram for explaining a face recognition system according to the present invention, FIG. 2 is a block diagram for explaining a face recognition system according to the present invention, and FIG. 3 describes a control method for a face recognition system according to the present invention. It is a flow chart for Furthermore, FIGS. 4, 5, 6, 7a, 7b, 8, 9, 10, 11, 12, and 13 are for explaining a face recognition system and a control method thereof according to the present invention. are concepts.

As shown in FIG. 1 , user authentication in the present invention may be performed through interworking with the face recognition system 100 and the server 200 . In the present invention, the server 200 may be located in the same place as the face recognition system 100 or in a separate place, and may be configured to process user authentication for a face recognized by the face recognition system 100.

Meanwhile, the server 200 may be implemented as one component of the face recognition system 100, and in this case, the face recognition system 100 may perform functions including the role of the server 200.

In addition, as shown in FIG. 1 , the face recognition system 100 and the server 200 may be configured separately, and in the present invention, for convenience of description, the face recognition system 100 and the server 200 to be separated and explained.

Meanwhile, regardless of whether the face recognition system 100 and the server 200 are physically separated, the functions and controls performed by the face recognition system 100 and the server 200 described in the present invention are "face recognition system" Of course, it can be described as being performed in .

In the present invention, a user's face may be recognized, user authentication may be performed, and an event or function related to user authentication (hereinafter, a target function) may be performed according to a user authentication result.

Here, the type of “target function” may be very diverse, and may mean a function that requires user authentication, such as a specific space access control function, payment function, attendance confirmation function, visit confirmation function in a hospital, and the like. For example, a specific space access control function may be performed by controlling the access control gate 20 (or access control device or speed gate) shown in FIG. 1 to operate in either an open state or a closed state. can

As such, the face recognition system 100 in the present invention may have various hardware configurations depending on the functions to be achieved through face recognition and user authentication (eg, access control, attendance check, payment acceptance, etc.), In the present invention, as shown in FIG. 1 as a representative example, user authentication in the access control gate 20 will be described. However, it goes without saying that the face recognition and user authentication methods described below can be applied to face recognition systems having various functions.

As shown in FIG. 1, the face recognition system 100 (hereinafter referred to as “system 100”) according to the present invention includes a sensor unit 110, a storage unit 120, a communication unit 130, It may be configured to include at least one of the display unit 140, the sound output unit 150, and the control unit 160. At this time, the system 100 according to the present invention is not limited to the above-described components, and may further include components performing the same or similar roles as the functions described herein.

Furthermore, the face recognition system 100 may be interlocked with the access control gate 20 or may include the access control gate 20 as a component. Furthermore, it goes without saying that the face recognition system 100 can be a component of the access control gate 20 . The sensor unit 110 senses (or senses) a user (or object) using a configuration of a sensing module (eg, a camera 111, a thermal imaging camera, a proximity sensor, an infrared sensor, etc.) provided in the system 100. or filming).

In the present invention, “sensing” can be interpreted in various ways depending on the sensing module. For example, when using the camera 111, sensing may be understood as capturing an image. As another example, when using a proximity sensor, it may be understood as sensing the presence of a user (or object). Hereinafter, “sensing” may be used interchangeably with “photographing (or obtaining) an image”.

The sensor unit 110 may capture an image (eg, at least one of an RGB image frame and a thermal image frame). At this time, the sensor unit 110 may capture an image in the form of at least one of a video and an image.

The sensor unit 110 may capture an image based on a preset time period or a user's request.

The storage unit 120 may be configured to store data related to face recognition and/or user authentication according to the present invention using a memory provided (or inserted) in the system 100 or a cloud server. For example, the storage unit 120 may store information sensed by the sensor unit 110 or acquired images or information obtained from them.

Furthermore, the communication unit 130 uses a configuration of communication modules (eg, a mobile communication module, a short-distance communication module, a wireless Internet module, a location information module, a broadcast reception module, etc.) provided in the system 100 to use a server ( 200), an external server (not shown), or a cloud server (not shown).

Furthermore, the display unit 140 is configured to output visual information, and under the control of the controller 160, it may be configured to output information related to at least one of face recognition and user authentication.

For example, the display unit 140 outputs information related to user authentication success (or failure) based on the user authentication result information for the user to be authenticated received from the server 200, or the sensor unit 110 may be controlled to output guide information for guiding the user's location so that the user can be photographed.

For example, based on the authentication result information for the user to be authenticated, the display unit 140 displays the authentication result or guide information related to performing the target function (eg, “Welcome, User 1 is welcome”, FIG. 8 810 of) may be output.

Furthermore, the audio output unit 150 may audibly output information related to the present invention.

For example, under the control of the control unit 160, based on authentication result information on the user to be authenticated, the audio output unit 150 may output an authentication result or guide information related to performing a target function in voice.

Furthermore, the controller 160 may utilize at least one of a CPU and a cloud server provided in the system 100 to perform control of each configuration and overall control related to the present invention.

The controller 160 may have control authority for at least one of the components of the system 100 in order to carry out the present invention.

The controller 160 may recognize a user's face and perform a process for user authentication based on information sensed by the sensor unit 110 or received through the communication unit 130 .

The controller 160 may include a face recognition unit 161, and the face recognition unit 161 may recognize a face in an image captured by the sensor unit 110 based on an algorithm having a face recognition function. . Furthermore, the face recognition unit 161 may perform a series of controls related to the recognized face, which will be discussed in more detail later. In the present invention, for convenience of description, the configuration of the face recognition unit 161 and the controller 160 is not distinguished, and the controller 160 is unified and described.

Meanwhile, the system 100 and the server 200 according to the present invention operate in conjunction with each other to perform user authentication for the target function and control the execution of the target function based on the user authentication result for the target function. can

The server 200 may serve to perform user authentication based on a user authentication request from the system 100 and transmit a user authentication result to the system 100 .

Meanwhile, in the present invention, when the function performed as a result of user authentication is the operation control of the access control gate 20, the door of the access control gate 20 is opened based on the user authentication result for the user to be authenticated. It can be controlled to open or not to open. The access control gate 20 may be controlled based on a control command transmitted from at least one of the server 200 and the system 100 .

Meanwhile, as shown in FIG. 1 , the server 200 according to the present invention may be configured to include at least one of a communication unit 210, a storage unit 220, and a control unit 230. At this time, the server 200 according to the present invention is not limited to the above-described components, and may further include components performing the same or similar functions as those described herein.

The communication unit 210 may transmit/receive data with at least one of the system 100, the access control gate 20, an external server (not shown), or a cloud server (not shown).

For example, the communication unit 210 receives image information including the face of the user to be authenticated or face information on the user to be authenticated from the system 100, and the user authentication result for the user to be authenticated is displayed in the system 100. ) can be transmitted.

The storage unit 220 may be configured to store various information related to user authentication.

The storage unit 220 may include at least one of a user DB 221 containing reference facial feature information related to a pre-registered face and a terminal DB 222 containing information related to the system 100 .

Here, the reference facial feature information may include at least one of a pre-registered face image and pre-registered feature information (eg, feature vector information).

User information for each person may be stored in the user DB 221 in units of individual persons (or persons). That is, in the user DB, user information corresponding to each different person may be matched and stored.

Here, the user information includes the name, date of birth, address, phone number, e-mail address, account information, employee number, ID, biometric information (fingerprint information, iris information, etc.), reference facial feature information, It may include at least one of schedule (schedule) related information.

The reference face image included in the reference face feature information is a face image of a specific person, and may be one or more. In the present invention, the user DB may also be referred to as "user related information", "user personal information" or "user identification information".

Meanwhile, information about the system 100 performing the user authentication process may be stored in the terminal DB 222 .

Here, the information on the terminal may include one of identification information (serial number) of each system 100 or access control gate 20, location, and information related to a target function linked with user authentication.

Meanwhile, the controller 230 may perform user authentication on the user to be authenticated using reference facial feature information stored in the user DB 221 .

At this time, user authentication may be performed through the user authentication unit 231 of the control unit 230. That is, it can be understood that the user authentication unit 231 performs a role of the control unit 230 . In the present invention, the control unit 230 and the user authentication unit 231 are used interchangeably without being separately distinguished.

Hereinafter, based on the configuration of the system 100 and the server 200 described above, a basic process of user authentication in the present invention will be described.

As shown in FIG. 2 , in the system 100 according to the present invention, a process of photographing an image and performing data processing on the photographed image using the sensor unit 110 (eg, a camera) is performed. It may be performed (S210). Images may be captured at preset time intervals.

In the sensor unit 110, an image frame (or image, ex: RGB image frame) and a thermal image frame (or image) may be captured respectively (S211 and S212).

In this way, through photographing through the sensor unit 110, an image (referred to as “video” in the description of the present invention) to be a face recognition target can be secured.

Next, the system 100 may perform a process of recognizing a face in an image, that is, a video (S220).

More specifically, the controller 160 may recognize a user's face in the image (S221) and determine whether the recognized user's face satisfies a preset face condition (S222).

Here, the preset face condition is whether or not the recognized face meets conditions suitable for performing user authentication, and the preset face condition may be set in various ways. For example, preset face conditions include i) a location in which a recognized face is included in an image, ii) a size of an area occupied by the recognized face in an image, iii) an angle of a face recognized in an image (ex: user face It may be related to at least one of various conditions, such as whether the user is looking at the front), iv) whether the user's entire face is recognized, and v) whether the user is wearing a mask.

When a face that satisfies a preset face condition is recognized in an image acquired through the camera 111, the controller 160 performs user authentication on a face that satisfies the preset face condition using the corresponding image. process can be performed.

The controller 160 may specify a face that satisfies a preset face condition among faces recognized in the image as a face to be authenticated. For example, in the image shown in FIG. 4(b), the controller 160 selects a face 2000a that occupies the widest area without being cropped among a plurality of user faces 2000a, 2000b, 2000c, 2000d, and 2000e. It can be specified as a face to be authenticated.

In this way, when a face to be authenticated, which is a face to be authenticated, is specified from the image, the controller 160 may additionally perform a spoofing check and a temperature check (S230).

The controller 160 may request user authentication for an image that has passed the spoofing check and the temperature check.

The controller 160 may transmit a user authentication request for the user to be authenticated to the server 200 (S250, see FIG. 2).

The control unit 160 may transmit information about the face of the user to be authenticated (such as a user's face image or facial feature information) to the server 200 along with a user authentication request.

The server 200 may perform a similarity determination between face information of the user to be authenticated included in the image and reference facial feature information stored in the user DB 221 of the server 200 (S240).

As an example of a similarity determination method, the server 200 may perform user authentication by comparing a face of a user to be authenticated included in an image with a reference face image stored in the user DB 221 .

In this case, comparing the reference face image with the face of the user to be authenticated included in the image may mean comparing feature point information corresponding to the reference face image with feature point information on the face of the user recognized in the image. there is. That is, feature point information in which reference face images of users are defined according to features of eyes, noses, and mouths included in the reference face images may exist in the user DB.

As another example of a similarity determination method, the server 200 performs user authentication by comparing similarity between a feature vector of a face of a user to be authenticated extracted from a face image and a feature vector included in reference facial feature information. can In this case, feature vectors for various reference faces may be stored in the user DB. The server 200 searches (or extracts) a feature vector most similar to the feature vector of the face of the user to be authenticated from the user DB, and the similarity between the retrieved feature vector and the feature vector of the face of the user to be authenticated is a preset value. (threshold) or more, the authentication of the user can be processed as a success.

In addition, the server 200 provides (or transmits) the user authentication result to the system 100, and the system 100 may provide the user authentication result to the user based on the authentication result received from the server 200. Yes (S260). For example, the authentication result may be provided to the user through an output unit (eg, a display unit, a sound output unit (speaker), etc.) provided in the system 100 . For example, the controller 160, when user authentication for the face of the user to be authenticated succeeds, displays the authentication result or guide information related to performing the target function on the display unit 140 (for example, “Welcome, Welcome to User 1”). It is possible to control the display unit 140 so that “, see 810 of FIG. 8 ) is output.

Furthermore, the controller 160 may perform various controls based on the user authentication result for the user to be authenticated received from the server 200 . As an example, when the system 100 is interlocked with or included in an access control device, the control unit 160, when user authentication for a user to be authenticated succeeds, allows the user who has succeeded in authentication to enter a specific space. It is possible to control the access control gate 20 so as to be able to. The access control gate 20 maintains a closed state in a default driving state, and may be controlled to be in an open state based on control by the control unit 160 or a control signal received from the server 200 .

Furthermore, in the present invention, the system 100 and the access control gate 20 or payment terminal to which the result of user authentication is applied can be driven independently of each other. In this case, the server 200 may transmit the user authentication result to the system 100 as well as the access control gate 20 or the payment terminal (S270).

In this case, the operation of the access control gate 20 or the payment terminal may be controlled based on a user authentication result received from the server 200 .

For example, when the user authentication result is used to control the operation of the access control gate 20, the server 200 determines the gate (or door) of the access control gate 20 based on the user authentication result. A control signal to be in an open state may be transmitted to the access control gate 20 .

Hereinafter, a method of quickly and accurately recognizing a user's face using each component included in the system 100 and the server 200 will be described in detail.

In the control method of the face recognition system according to the present invention, a process of obtaining an image using the camera 111 provided in the system 100 performing the face recognition function may proceed (S310, see FIG. 3).

As shown in (a) of FIG. 4 , various users may be located in a specific space 1 . For example, in a specific space 1, users 1000a, 1000b, 1000c who want to pass through the access control gate 20, a user 1000d stationary at a specific location, or a user who freely moves in the specific space 1 A user 1000e may exist.

Hereinafter, for convenience of description, a user 1000a standing in the first row based on the access control gate 20 or system 100 is referred to as a first user 1000a, and a user 1000b standing in the second row is referred to as a second row. The user 1000b is named, and the user 1000c standing in the third row is named the third user 1000c. Furthermore, a user 1000d stationary at a specific location is named a fourth user 1000d, and a moving user 1000e is named a fifth user 1000e.

On the other hand, the system 100 or the camera 111 provided in the system 100, for the execution of a function to be performed through user authentication (eg, a door operation function of the access control gate 20), It may be arranged so that an image of the space 1 is taken. The camera 111 may be disposed at an appropriate location capable of recognizing a user's face.

For example, as shown in (a) of FIG. 4 , the camera 111 captures a specific space where users who want to pass through the access control gate 20 are located, around or around the access control gate 20. It may be disposed in one area of the access control gate 20 .

As shown in FIG. 4( b ), the image 400 photographed by the camera 111 may include information about a space in which the camera 111 corresponds to a field of view. When users are located within the field of view of the camera 111, as shown in (a) of FIG. 4, as shown in (b) of FIG. 4, the graphic object 2000a corresponding to the user is included in the image 400. , 2000b, 2000c, 2000d, 2000e) may be included. As illustrated, the graphic objects 2000a, 2000b, 2000c, 2000d, and 2000e may include an image corresponding to a user's face.

As shown in (a) of FIG. 4, when a plurality of users are located around the camera 111, in (b) of FIG. , 2000b, 2000c, 2000d, 2000e) may be included.

In the present invention, in order to avoid confusion of terms, a graphic object corresponding to a user's face may be named "face" or "face image". For example, a graphic object corresponding to the face of the first user 1000a is converted to a first face 2000a or a first face image 2000a, and a graphic object corresponding to the face of the second user 1000b is converted to a second face. (2000b) or the second face image (2000b). Furthermore, in the drawings of this specification, only the user's face is highlighted in the image 400 for convenience of description.

Meanwhile, the above process S310 may correspond to the process S210 described above in FIG. 2 , and the process S310 may include the detailed processes described in the process S210.

Next, in the present invention, based on a preset face recognition algorithm, a process of recognizing a face from an image may proceed (S320, see FIG. 3).

The controller 160 may recognize a face in the image 400 obtained in (b) of FIG. 4 based on a face recognition algorithm. An algorithm for recognizing a face may be performed using various image object recognition techniques for recognizing an object in an image. For example, the algorithm for recognizing faces in the image 400 may be at least one of the techniques for recognizing users' faces in an image using an algorithm (eg, YOLO algorithm, CNN algorithm) learned for face recognition. there is.

As shown in (a) of FIG. 5 , the controller 160 sets bounding boxes (3000a, 3000b, 3000c, 3000d, 3000e) for each of the recognized faces 2000a, 2000b, 2000c, 2000d, and 2000e. can be assigned. Each of the bounding boxes 3000a, 3000b, 3000c, 3000d, and 3000e may be assigned to each face 2000a, 2000b, 2000c, 2000d, and 2000e included in the image.

The controller 160 may allocate different identification information to each recognized face. Meanwhile, the identification information assigned to each recognized face can be understood as the identification information for the bounding boxes 3000a, 3000b, 3000c, 3000d, and 3000e.

Here, the identification information may be ID (Identification), and for example, the controller 160 may assign different IDs (Identification, for example, , “USER 1”) can be assigned.

On the other hand, the controller 160 may assign different IDs to a plurality of faces included in the image. When the same face is included in different images, the same ID (Identification) is assigned to the same face included in the different images. can be assigned.

For example, in FIG. 7A , the same ID (eg, “ USER 1”) can be assigned.

Meanwhile, when a face is recognized in an image, the controller 160 may extract information about each recognized face. For convenience of explanation, the extracted information is named “face information”.

The controller 160 extracts face information for each recognized face, and the extracted face information 510 is matched with an ID assigned to the recognized face and stored as shown in FIG. 5(b). can The controller 160 may control the face information 510 extracted from the image 400 to be matched with an ID assigned to a face recognized from the image and stored in the storage 120 .

As shown in (b) of FIG. 5, “face information 510” may include various information that can be extracted in relation to a face included in an image. For example, face information 510 includes i) location information (pixel coordinate information, 511) of a face (bounding box) in an image, ii) face size information (size of an area occupied by a face in an image or Corresponds to the size (or width) of the region corresponding to the bounding box, 512), iii) face angle information (information on whether the face is looking in a reference direction (ex: camera reference), iv) facial feature information ( ex: vector information).

Here, the facial feature information is information on the characteristics of each face, and is feature data that becomes a feature for each part where facial features such as eyes, eyebrows, nose, mouth, and chin change, or a feature vector related thereto can mean Each person has different characteristics according to their appearance, and accordingly, users with different appearances may have different facial feature information.

Such face information may be present for each image in which the user's face is recognized.

As shown in (b) of FIG. 5, in the image (or frame) acquired by the camera 111, identification information 520 exists for each image, and the identification information 520 of the image and extracted from the image The face information 510 may be matched with each other. As shown, identification information and face information of an image may exist in a storage unit in the form of a table.

The identification information 520 of the image includes i) a serial number for classifying the image, an ID (eg, FRAME no: #1025), ii) video capture date and time information (or frame capture date and time information, for example, 2021 -11-14-13:00:05), iii) information related to the terminal that captured the video (identification information of the terminal, location information of the terminal (eg, LOBBY A)).

As shown in (b) of FIG. 5, the storage unit 120 includes face information 510 for each face recognized in the image, an ID assigned to the face recognized in the image, and identification information of the image in which the face was recognized ( 520) may be matched and stored.

Meanwhile, the controller 160 stores the image acquired from the camera 111, face information 510 extracted from the image, and identification information of the image only for a preset period (preset information storage period, for example, 10 minutes). By controlling to be stored in the storage unit 120, the storage unit 120 can be used efficiently.

For example, the controller 160 may control the storage unit to delete the information listed above from the storage unit 120 when a predetermined time (preset information storage time) elapses.

On the other hand, the image obtained from the camera not only includes a face corresponding to one user, but also includes a plurality of faces 2000a, 2000b, 2000c, 2000d, 2000e) may be included. In this case, some of the users corresponding to the plurality of faces may be users for whom user authentication is not intended. In addition, when a plurality of faces are included in an image, a specific user who truly intends to authenticate a user may not be able to perform authentication due to the faces of other users.

In addition, when user authentication is performed for at least one face among the plurality of faces 2000a, 2000b, 2000c, 2000d, and 2000e included in the image 400 even when there is no user attempting user authentication in the space may also occur.

Therefore, in order to perform user authentication through face recognition in a space where a plurality of users freely enter and exit, accurate user authentication for a specific user intended for user authentication is essential.

Accordingly, in the present invention, user authentication may be performed on at least one face that satisfies a preset face condition among faces recognized in the image 400 .

More specifically, in the present invention, a process of checking whether a face recognized in an image satisfies a preset face condition may be performed (S330, see FIG. 3).

Here, the preset face condition is whether or not the recognized face meets conditions suitable for performing user authentication, and the preset face condition may be set in various ways. For example, preset face conditions include: i) a position in which a face recognized in an image is included, ii) a size of an area occupied by a face recognized in an image, iii) an angle of a face recognized in an image (ex: a user's face is It may be related to at least one of various conditions, such as whether the user is looking at the front), iv) whether the user's entire face is recognized, and v) whether the user is wearing a mask. The controller 160 may control user authentication for a user corresponding to the corresponding face to be performed using at least one face included in the image that satisfies a preset face condition.

When a plurality of faces are recognized from the image, the controller 160 checks whether or not a preset face condition is satisfied for each of the plurality of recognized faces, and identifies a face that satisfies the preset face condition among the plurality of faces. can do.

For example, as shown in (a) of FIG. 5 , when a plurality of faces are recognized in the image 400, the controller 160 determines the recognized first to fifth faces 2000a, 2000b, 2000c, 2000d, and 2000e. ), it is possible to determine whether a preset face condition is satisfied for each face.

The controller 160 may perform user authentication for a user corresponding to a face included in the image 400 only when the image 400 includes a face satisfying a preset face condition. That is, even if a face is recognized in an image, the controller 400 performs user authentication for a user corresponding to a face included in the image when there is no face that satisfies a preset face condition in the corresponding image. It may or may not send the request to the server.

Meanwhile, as an example of a preset face condition, looking at the size of an area occupied by a recognized face, the controller 160 determines the size occupied by each of a plurality of faces recognized in an image (eg, the face area (or bounding area) It may be determined whether the size or width of the box area) has a predetermined size or more. When a face having a predetermined size or larger exists, the controller 160 may specify a user corresponding to the corresponding face as a face subject to user authentication.

For example, as shown in (a) of FIG. 6 , the controller 160 controls a face region having a preset size or larger among the first face 2000a and the fourth face 2000d recognized in the image 610. The first face 2000a included in may be specified as a target face for user authentication.

Furthermore, the control unit 160. When there are a plurality of face regions having a predetermined size or more in the image, a face included in the largest face region among the plurality of face regions may be specified as a target face for user authentication.

For example, in the image 610 shown in (a) of FIG. 6 , even if both the first face 2000a and the fourth face 2000d have a preset size or larger, the controller 160 displays the largest face. The first face 2000a included in the area may be specified as a target face for user authentication.

As another example of a preset face condition, looking at a location where a recognized face is included in an image, the preset face condition is a location where a recognized face is included in an image, as shown in FIG. It may be related to being located within a predetermined radius from a preset location (eg, center, 611) of the image.

For example, as shown in (a) of FIG. 6 , the controller 160 controls a predetermined position of the image 610 among the first face 2000a and the fourth face 2000d recognized in the image 610. (For example, the first face 2000a located within a predetermined radius from the central position 611 of the image) may be specified as a target face for user authentication.

As another example, the preset face condition may be whether a face recognized in an image corresponds to the entire face. As shown in (b) of FIG. 6 , when the first face 2000a recognized in the image 620 does not correspond to the entire face of the user but to a part of the face, the controller 160 controls the first face 2000a. (2000a) may not be specified as a face to be authenticated as a face subject to user authentication. In this case, if the fourth face 2000d is not a face having a predetermined size, the controller 160 may determine that a face to be user authentication does not exist in the corresponding image 630 .

However, if the frontmost user attempting authentication is too close, only a part of the user's face is recognized and the face of another user behind it can be determined as the face to be authenticated when it is an appropriate size, so it is considered to be the frontmost. If only a part of the face of a user (for example, a user with the largest face) is exposed, it is determined that there is no face subject to user authentication, and a guide is given to the user to step back or move so that the entire face is on the screen. can be printed out.

As another example, the preset face condition may be related to an angle of a face recognized in an image falling within a preset angular range based on a preset direction. This may be related to whether the user is looking in a reference direction (eg, a camera direction), that is, whether the user is looking directly at the camera.

For example, as shown in (c) of FIG. 6 , the angle of the fifth face 2000e recognized in the image 630 (eg, rotated 90 degrees toward -y with respect to the x direction, - 90 ° ) is not within the first angle (eg, -20 °) to the second angle (eg, +20 °) based on the front direction (x, 641) of the camera 111, The controller 160 may not specify the fifth face 2000e as a face subject to user authentication.

That is, the controller 160 may determine that the face satisfies the preset face condition only when the angle of the face recognized in the image satisfies a predetermined condition based on the preset direction.

As such, in the present invention, in order to quickly and accurately recognize a user's face, a face that satisfies a preset face condition may be checked in an image acquired through the camera 111 .

Meanwhile, as described above, when there are a plurality of preset face conditions, the controller 160 selects a face included in the image as a user authentication target only when the face in the image satisfies all of the plurality of face conditions. It can be identified as a face.

Meanwhile, when there are a plurality of faces satisfying preset face conditions in the image, the controller 160 may specify the largest face in the image as the face to be user-certified. Alternatively, if there are a plurality of faces in the image that satisfy a preset face condition, the controller 160 may specify all of the plurality of faces as faces subject to user authentication.

Next, in the present invention, based on a result of checking whether a face recognized in an image satisfies a preset face condition, a process of requesting user authentication may proceed (S340).

When the face recognized in the image satisfies a preset face condition, the controller 160 may request user authentication to the server 200 for user authentication corresponding to the corresponding face. In this case, a user authentication request may be transmitted from the system 100 to the server 200 .

Unlike this, the controller 160 may not transmit a user authentication request to the server 200 when a face recognized in an image does not satisfy a preset face condition. That is, the controller 160 checks whether the face recognized in the image satisfies the preset face condition, and if there is no face that satisfies the preset face condition among the faces, for the server 200, the image may not transmit a user authentication request based on

As such, in the present invention, by controlling the user authentication process to proceed with respect to a face that satisfies a preset face condition, user authentication is prevented from being performed for a face of a user other than the face of the user attempting user authentication. accuracy can be improved. In addition, by not repeating an operation for authentication unnecessarily, it is possible to reduce computational resources and enable fast processing.

Meanwhile, in the present invention, it is described that the user authentication process is performed in the user authentication server 200, but the user authentication process can be performed in the system 100, of course.

When the face recognized in the image satisfies a preset face condition, the controller 160 transmits information related to the recognized face to the server 200 so that the server 200 performs user authentication on the recognized face. can transmit

At this time, along with a user authentication request, information on a user to be authenticated may be transmitted to the server 200 .

The information on the user subject to authentication is information related to a face included in the image, i) an image including the recognized face, ii) identification of a face that satisfies a predetermined face condition among faces recognized in the image information (ex: corresponding face ID), and iii) face information on a face that satisfies a preset face condition among faces recognized in the image.

Here, the face information, as described above, i) location information (pixel coordinate information, 511) of the face (bounding box) in the image, ii) face size information (size of the area occupied by the face in the image) or corresponding to the size (or width) of the region corresponding to the bounding box, 512), iii) angle information about the face (information on whether the face is looking in a reference direction (ex: camera reference), iv) facial feature information may include at least one of them.

Meanwhile, of course, the controller 160 may transmit, to the server 200, only face information for a specific user to be authenticated among faces recognized in the image. For example, the controller 160 may transmit facial feature information corresponding to a specific face to be authenticated to the server 200 instead of transmitting the image itself to the server 200 . In this case, the server 200 may perform user authentication based on facial feature information received from the system 100 .

Meanwhile, in the system 160, when a plurality of faces are recognized in an image, information for specifying a specific face to be authenticated among the plurality of faces may not be transmitted to the server 200. In this case, the server 200 may perform user authentication on at least one face among a plurality of faces by itself. The server 200 may perform user authentication on a face occupying the largest area in an image among a plurality of recognized faces. Even in this case, the server 200 may perform user authentication only on a face that satisfies the previously set face condition among the plurality of faces.

Meanwhile, the server 200 may perform user authentication for a face recognized in an image based on a user authentication request from the system 100 .

As described above, the control unit 230 included in the server 200 determines the similarity between the face information of the user to be authenticated included in the image and the reference facial feature information stored in the user DB 221 of the server 200. can be performed.

As an example of a similarity determination method, the controller 230 may perform user authentication by comparing a face of a user to be authenticated included in an image with a reference face image stored in the user DB 221 .

In this case, comparing the reference face image with the face of the user to be authenticated included in the image may mean comparing feature point information corresponding to the reference face image with feature point information on the face of the user recognized in the image. there is. That is, feature point information in which reference face images of users are defined according to features of eyes, noses, and mouths included in the reference face images may exist in the user DB.

As another example of a similarity determination method, the control unit 230 included in the server 200 compares the similarity between a feature vector of a face of a user to be authenticated extracted from a face image and a feature vector included in reference facial feature information. Through this, user authentication can be performed. In this case, feature vectors for various reference faces may be stored in the user DB. The control unit 230 included in the server 200 searches (or extracts) a feature vector that is most similar to a feature vector of the face of the user to be authenticated from the user DB, and searches (or extracts) a feature vector that is most similar to the feature vector of the face of the user to be authenticated. When the similarity between the vectors is greater than or equal to a predetermined value (threshold), user authentication may be successfully processed.

The controller 230 included in the server 200 may complete user authentication for the face to be authenticated when a person corresponding to the face to be authenticated is found in the user DB 221 . In the present invention, completion of user authentication for the face to be authenticated may be referred to as “user authentication success”.

In contrast, the control unit 230 included in the server 200 determines an accurate person corresponding to the face to be authenticated from the user DB when the similarity between the face information of the user to be authenticated and the reference facial feature information stored in the user DB is low. You may not be able to search. That is, the controller 230 of the server 200 may not be able to complete user authentication for the face to be authenticated. In the present invention, failure to retrieve an accurate person corresponding to the face to be authenticated from the user DB 221 may be referred to as “user authentication failure”.

Furthermore, in the present invention, user authentication can be successfully processed only when the user corresponding to the face to be authenticated has authority for an event or function subject to user authentication. For example, when an event or function subject to user authentication is control of a speed gate to allow access to a specific space, the server 200 allows the user to be authenticated to have access to the specific space. Only in this case, user authentication can be processed as success.

The controller 230 included in the server 200 extracts user information matched to the searched person from the user DB 221 when a person matching a reference face image corresponding to a face to be authenticated is found in the user DB 221. can do.

Information on the authority of the user to be authenticated (or authority information) may be included in user information of the user to be authenticated stored in the user DB 221 .

Here, the authority for an event or function subject to user authentication may be interpreted in various ways. For example, the authority for an event or function subject to user authentication depends on whether i) the user corresponding to the face to be authenticated (hereinafter referred to as the user to be authenticated) has the authority to receive the event or function subject to user authentication , ii) whether the user subject to authentication has the right to access the event or function subject to user authentication, iii) whether the user subject to authentication has the right to access a specific place, iv) whether the user subject to authentication has the right to use a specific electronic device It may be information about whether the user to be authenticated has permission to access any function, device, or location, such as whether or not the user has authority to control.

For example, if the event or function subject to user authentication is a function of controlling an access gate in a viewing state so that a user to be authenticated can enter a specific place, as shown in (a) of FIG. 4, the server ( The controller 230 of 200) may determine whether or not the user to be authenticated has permission to enter a specific place (eg, a building) based on the user information of the user to be authenticated.

The controller 230 included in the server 200 determines whether the user corresponding to the face to be authenticated has authority for an event or function subject to user authentication based on user information extracted from the user DB 221. can do.

The control unit 230 included in the server 200 may complete user authentication and determine that user authentication is successful if the user corresponding to the face to be authenticated has authority for an event or function subject to user authentication. .

In contrast, the control unit 230 included in the server 200 allows the user corresponding to the face to be authenticated to receive an event or function that is subject to user authentication, even if a person matching a reference face image corresponding to the face to be authenticated is found. If it is determined that there is no authority, it may be determined that user authentication has failed.

Meanwhile, the server 200 may transmit the user authentication result for the face to be authenticated to the system 100 . At this time, when user authentication is successful, the server 200 performs at least one of user information (eg, name, work department, etc.) corresponding to the face to be authenticated and a control command for an event or function to be authenticated. may be transmitted to the system 100.

Furthermore, when user authentication is successful, the server 200 transmits user information (eg, name, work department, etc.) corresponding to the face to be authenticated to the system 100, and an event or function subject to user authentication. A control command for may be transmitted to a device (eg, an access control gate, 20) related to an event or function subject to user authentication.

Meanwhile, in the present invention, based on the user authentication result for the face to be authenticated received from the server 200, a process of performing control according to the user authentication result for the face to be authenticated may proceed.

The controller 160 of the system 100 may perform different controls based on the user authentication result.

More specifically, the controller 160 may use one of the display unit 140 and the sound output unit 150 to provide different information about the user to be authenticated based on the success or failure of user authentication for the face to be authenticated. You can control at least one.

When user authentication for the face to be authenticated succeeds, the controller 160 may provide information corresponding to successful user authentication to the user to be authenticated through at least one of the display unit 140 and the sound output unit 150. In this case, the controller 160 may provide customized information of the user to be authenticated using user information corresponding to the face to be authenticated.

As an example, as shown in (a) and (b) of FIG. 8 , when user authentication for the first face 2000a and the second face 2000b is successful, the controller 160 sets the first face ( 2000a) and the user's name corresponding to the second face 2000b and information corresponding to user authentication success (for example, “Welcome, Welcome to User 1”, 810, 820) are displayed on the display unit 140. The display unit 140 may be controlled to visually output images on the screen. Furthermore, the controller 160 may control the audio output unit 150 so that information corresponding to user authentication success is audibly output through the audio output unit 150 .

As another example, as shown in (c) of FIG. 8 , when user authentication for the third face 2000c fails, the controller 160 provides information corresponding to the user authentication failure (eg, , “Oops! Furthermore, the controller 160 may control the audio output unit 150 so that information corresponding to user authentication success is audibly output through the audio output unit 150 .

Furthermore, the controller 160 may perform control related to an event or function so that an event or function subject to user authentication is executed based on success or failure of user authentication for the face to be authenticated.

As described above, as shown in (a) of FIG. 4, when the event or function subject to user authentication is to control the access control gate 20 so that the user to be authenticated enters or does not enter a specific place. , According to the user authentication result received from the server 200, the control unit 160 can control the access control gate 20 so that the access control gate 20 has any one of a reading state and a closed state. .

More specifically, the control unit 160, when user authentication of the face to be authenticated succeeds, allows the access control gate 20 to pass through the access control gate 20 for a predetermined time so that the user corresponding to the face to be authenticated passes through the access control gate 20. It can be controlled to have an open state. Furthermore, the control unit 160 may control the access control gate 20 to be in a closed state so that the access control function is performed after a predetermined time has elapsed.

In contrast, when user authentication for the face to be authenticated fails, the control unit 160 keeps the access control gate 20 in a closed state so that the user corresponding to the face to be authenticated does not pass through the access control gate 20. You can control it.

Meanwhile, an open state or a closed state of the access control gate 20 may be controlled based on a control command received from the server 200 . The server 200 may transmit the result of user authentication to the access control gate 20 as well as the system 100 . An open state of the access control gate 20 may be controlled based on a user authentication result received from the server 200 .

As such, in the present invention, by performing user authentication on a face that satisfies a preset face condition, data processing consumed for user authentication can be efficiently managed.

On the other hand, in the present invention, before user authentication is performed on a face to be authenticated that satisfies a preset face condition, the user authentication intention of whether the user corresponding to the face to be authenticated truly wants to perform user authentication is determined. , user authentication can be performed only when the corresponding user has the intention of user authentication. Hereinafter, a method of determining the user's authentication intention will be described in detail.

In the present invention, when a face recognized in an image satisfies a preset face condition, it may be determined whether a user whose face satisfies the preset face condition has an intention to perform user authentication. And, based on the determination result, user authentication may be selectively performed.

When it is determined that a user corresponding to a face to be authenticated that satisfies a preset face condition has an authentication intent, the controller 160 may request user authentication for the face to be authenticated from the server 200 .

In contrast, in the present invention, if a face recognized in an image does not satisfy a preset face condition or if it is determined that the user corresponding to the face does not have the user authentication intention even if the face meets the preset face condition, the server 200 User authentication for the recognized face may not be requested.

When the controller 160 determines that a face other than the face to be authenticated or a user corresponding to the face to be authenticated does not have the intention of user authentication (in the case of an access gate, for example, a user passing through the gate rather than intending to pass through the gate, or , a user walking out of an entrance gate, etc.), and a user authentication request for a face to be authenticated can be controlled to be limited to the user authentication server 200 .

User authentication of the user corresponding to the face to be authenticated may be determined based on information stored in the storage unit 120 related to the user to be authenticated.

Here, the information related to the user to be authenticated and stored in the storage unit 120 may include at least one of an image including a face of the user to be authenticated and face information of the user to be authenticated included in the corresponding image.

As shown in FIG. 7A , the controller 130 controls an image captured at the current time point t0 (eg, a first image, 700) and a time point prior to the current time point t0 (past time point, tp1 to tp3, etc.) The intention of the user to be authenticated may be determined using face information of the user to be authenticated extracted from at least one image 710 , 720 , and 730 captured in ).

The control unit 160 stores the second through second images 700 obtained at points in time tp1, tp2, and tp3 prior to the point in time t0 at which the first image 700 in which the face to be authenticated 2000a is recognized in the storage unit 120 is obtained. It is possible to check whether face information matched to a specific ID (USER 1) of a user to be authenticated extracted from the fourth images 710, 720, and 730 exists.

More specifically, the controller 160 may check whether face information matched to the same ID as the ID assigned to the face to be authenticated in the first image 700 exists in the storage 120 .

Face information matched with a specific ID may be understood as face information related to the face 2000a to which a specific ID is assigned.

As described above, face information may be various information related to a specific face (or face image) extracted from an image. For example, face information includes i) positional information about a face (face image), ii) size information about a face (size (or width) of a region corresponding to a face image or bounding box), iii) information about a face It may include at least one of angle information and iv) facial feature information.

If there is face information matched to the same ID as the ID assigned to the face to be authenticated 2000a in the first image 700, the controller 160 determines the face information of the face to be authenticated 2000a extracted from the first image 700. Based on the face information and face information of the face to be authenticated (2000a) extracted from the images (710, 720, and 730) taken before the first image (700), the size and location of the face (or bounding box) to be authenticated. change can be calculated.

Here, the amount of change in the size and position of the face to be authenticated is determined by the size of a region corresponding to the face to be authenticated and coordinates corresponding to the face to be authenticated in the plurality of images 700, 710, 720, and 730 in which the face to be authenticated is recognized. It can be understood as meaning the degree of change of .

For example, in FIG. 7A , the size of the face to be authenticated 2000a in the first image 700 obtained at the current time point t0 is the size of the second image image 710 obtained at the first past time point tp1. When the size of the face 2000a is relatively larger than the size of the face 2000a to be authenticated (the width of the region is increased), the size difference (or difference in width) of the face 2000a to be authenticated in each image is the amount of change in size of the face to be authenticated. can be

As another example, in FIG. 7B , the position (pixel coordinates) of the face to be authenticated 2000a in the first image 700 obtained at the current time point t0 and the second image (pixel coordinates) obtained at the first past time point tp1 ( The difference in position (pixel coordinates) of the face to be authenticated 2000a in 710 may be the amount of change in the position of the face to be authenticated. In addition, the speed at which the user approaches or moves to the system can be calculated using the difference between each image capturing time and the change in size.

As such, the controller 160 may calculate at least one of the size and position change of the face to be authenticated based on the face information, and determine the user authentication intention of the user corresponding to the face to be authenticated based on the calculated amount of change. .

When it is determined that the user corresponding to the face to be authenticated has approached the system 100 or the access control gate 20 based on the calculated amount of change, the controller 160 determines that the user corresponding to the face to be authenticated has the user authentication intent. It can be judged that there is

Specifically, when the size of the face to be authenticated increases in order of time, the controller 160 determines that the user corresponding to the face to be authenticated approaches the system 100 or the camera 111 with the intention of user authentication. can

For example, as shown in FIG. 4(a), assume a case where a plurality of users 1000a, 1000b, and 1000c stand in line in front of the system 100 for user authentication. A user having an intention of user authentication may gradually move from a location relatively far from the system 100 or the access control gate 20 to a location close to the system 100 or the camera 111 .

Furthermore, as shown in FIG. 7A , the controller 160 determines that the face to be authenticated (2000a) is a fourth face based on the amount of change in face size calculated from face information on the face to be authenticated (2000a) at the current and past viewpoints. When it is determined that the size of the image 730, the third image 720, the second image 710, and the first image 700 has increased according to the order of time, the user corresponding to the face 2000a to be authenticated is the user. It may be determined that the system 100 or the camera 111 is accessed with an authentication intent.

On the other hand, the controller 160 locates the user's face corresponding to the face to be authenticated in a specific area (eg, an area within a certain radius from the central point, see reference numeral 710 in FIG. 7B) based on the calculated position change amount. When it is determined that the user has moved, it may be determined that the user corresponding to the face to be authenticated has an intention for user authentication.

Specifically, when the location of the face to be authenticated moves to a specific area (eg, the central area of the image) in chronological order, the controller 160 determines that the user corresponding to the face to be authenticated has the user authentication intent and the system (100) or access control gate (20) can be determined to approach.

For example, as shown in FIG. 7B , a user having an intention to authenticate a user has a posture such that his/her face is located in a specific region of an image (eg, within a certain radius from the central point 740 of the image). and the position of the face can be changed.

In this case, as shown in FIG. 7B , the controller 160 determines the face to be authenticated 2000a, based on the amount of change in face position calculated from the face information of the face to be authenticated 2000a at the current and past viewpoints. As in the 4th image 730, the 3rd image 720, the 2nd image 710, and the 1st image 700, if it is determined that it has moved to a specific point 740 according to the order of time, the face to be authenticated ( It may be determined that the user corresponding to 2000a) has placed his or her face on the system 100 or the access control gate 20 with the intention of user authentication.

As such, in the present invention, using face information of a face to be authenticated extracted from at least one other image obtained at a point in time prior to the point in time at which the target image is acquired, it is determined whether the user corresponding to the face to be authenticated has an intention for user authentication. can judge

Through this, in the present invention, even if a face that satisfies a preset face condition is recognized, user authentication is performed only on the face of the user with the intention of user authentication, so that user authentication is performed with the face of the user without the intention of user authentication (誤) Recognition can be solved, and the accuracy and reliability of user authentication means can be improved.

Meanwhile, there may be cases in which user authentication is repeatedly processed for the same user even though user authentication is successful.

For example, as shown in (a) of FIG. 9 , even though a specific user (USER 7) succeeds in user authentication and the access control gate 20 is controlled to an open state, a specific user (USER 7) When standing as it is without passing through the access control gate 20, the camera 111 may re-acquire an image 910 including the face 900 of the authenticated user as shown in (b) of FIG. there is.

In this case, the server 200 may perform user authentication again for the user (USER 7) having a specific ID based on the information received from the system 100, and the user authentication result is sent to the system 100. can be conveyed

In this case, the control unit 160 controls the access control gate 20 according to the user authentication result for the face with the specific ID, and then the user authentication result for the face with the specific ID from the server 200. When is additionally received, control according to the additionally received user authentication result may be limited.

After successful user authentication for a face with a specific ID (USER 7), the controller 160 performs authentication when user authentication for the same user succeeds again within a preset time, as shown in (b) of FIG. 9 . Even if user authentication for the successful face 900 succeeds again, it is possible to control so that a function corresponding to user authentication success is not performed.

For example, as shown in (a) of FIG. 9 , a specific ID (USER 7) is generated based on the image 910 obtained at a second time point t1 within a preset time from the first time point t0. Even if re-authentication of the face 910 is successful, the control unit 160, as shown in (b) of FIG. The display unit 140 may be controlled so that 920) is not output to the display unit 140.

Furthermore, if user authentication for the authentication successful face 910 succeeds again after a predetermined time elapses, the controller 160 may control the target function corresponding to the user authentication success to be re-executed.

For example, assume a case where a user who has successfully authenticated goes out of the access control gate 20 and then needs to pass through the access control gate 20 again. As shown in (a) and (c) of FIG. 9 , a specific ID (USER 7 If re-authentication of the face 910 having ) is successful, the control unit 160 sends information corresponding to successful user authentication (eg, “Welcome, Welcome to User 7! This is the second authentication”, 930). It is output on the display unit 140, and the access control gate 20 can be controlled to operate in an open state again. At this time, as described above, the control of the access control gate 20 can be directly performed by the server 200, of course.

Meanwhile, in the present invention, when a plurality of users sequentially perform user authentication, the target function performed based on the user authentication result for the plurality of users can be controlled to proceed smoothly and smoothly without interruption.

As such, in the present invention, the user authentication for a plurality of users and the term of time caused by the target function that is repeatedly performed can be reduced, thereby solving the user's inconvenience caused by user authentication.

Hereinafter, a case in which user authentication is performed so that the access control gate 20 is operated in an open state in order for a plurality of users to sequentially enter a specific space in the present invention will be described as an example.

For example, as shown in (a) of FIG. 10 , when the distance (a) between the first user 1000a and the access control gate 20 is very short, the first user 1000a moves to the access control gate It may take longer than the time taken to move to the front of (20) until the user authentication for the first user 1000a is performed and the access control gate operates in an open state. In this case, the first user 1000a first stops to attempt user authentication and secondarily has to wait until the access control gate 20 operates in an open state.

In the present invention, when the user completes user authentication, the user and the system ( Based on the reference distance to be secured between the terminal, the camera, or the access control gate 20, a preset face condition may be determined. At this time, the preset face condition is a condition related to the size of the user's face included in the image. can include A condition for the size of a face occupied by a face in an image may be set in consideration of a user authentication processing time in the server 200, a control time of the access control gate 20 according to a user authentication result, and a user's movement time.

When the size of the face included in the image is larger than the preset face size condition, the controller 130 may determine that the size of the face included in the image satisfies the preset face size condition. The preset face size condition at this time is related to the distance between the user and the access control gate 20, and the distance at this time is the user authentication and access control gate 20 before the user reaches the access control gate 20. It may mean a distance at which enough time can be secured for all of the control of the reading state of the .

As such, in the present invention, when a user performs user authentication to enter and exit a specific space, a minimum standard distance required for the user can be set so that the user can enter and exit the specific space without waiting for user authentication. .

Hereinafter, a method for a plurality of users to more conveniently and smoothly pass through the access control gate 20 will be described.

In the present invention, when user authentication for the faces of a plurality of users is successively successful, the access control gate 20 is controlled so that the access control gate 20 remains open until all of the plurality of users pass through. By doing so, time and unnecessary energy for operating the access control gate 20 to repeat the open and closed states can be saved.

As shown in FIG. 11 , the controller 160 recognizes a plurality of faces 2000a, 2000b, and 2000c from the images 1110, 1120, and 1130 acquired by the camera 111, and the server 200 As a result of the performed user authentication, when user authentication for each of the plurality of faces 2000a, 2000b, and 2000c is successful, when all of the plurality of users corresponding to the plurality of faces pass through the access control gate 20 It can be controlled to have an open state from (time t0 to time t2).

Here, the plurality of faces 2000a, 2000b, and 2000c recognized in the image may be understood as faces that satisfy the preset face condition described above.

In this case, the users according to the plurality of faces 2000a, 2000b, and 2000c may be users who have an intention of user authentication, and the controller 130 performs the above-mentioned method, based on the images captured at the current and past time points. Thus, it is possible to determine the intention of each of the plurality of users.

As such, in the present invention, when user authentication for each of the plurality of faces 2000a, 2000b, and 2000c recognized in the image is all successful, the access control gate 20 detects all of the plurality of users corresponding to each of the plurality of faces. It can be controlled to have an open state until it passes (time t0 to time t2).

Therefore, in the present invention, whenever the user passes through the access control gate 20, the access control gate 20 repeats the open and closed states to reduce the time required, so that the user can enter and exit immediately after user authentication is completed. It can be made to pass through the control gate (20).

Furthermore, in the present invention, energy required can be saved by repeatedly operating the access control gate 20 in an open state and a closed state.

Meanwhile, as shown in (a) of FIG. 12 , a plurality of different access control gates 20a and 20b may exist in a specific space 1 . The system 100 of the present invention uses images obtained from different cameras 111a and 111b related to the plurality of access control gates 20a and 20b to provide a user information for each of the plurality of access control gates 20a and 20b. Authentication can be done individually.

For example, the system 100 according to the present invention performs user authentication on the first access control gate 20a using the image 1210 obtained from the first camera 111a, and the second camera ( User authentication for the second access control gate 20b may be performed using the image 1220 obtained in step 111b).

Meanwhile, around the first access control gate 20a and the second access control gate 20b, a plurality of users who try to pass through each access control gate may be located.

In front of the first access control gate 20a or the first camera 111a, first to third users 1000a, 1000b, and 1000c who are trying to pass through the first access control gate 20a line up in order. There may be.

Further, around the second access control gate 20b or the second camera 111b, fourth to fifth users 1000d and 1000e, who are trying to pass through the second access control gate 20b, line up in order. can stand

In the present invention, user authentication for a specific user recognized in both the image 1210 photographed by the first camera 111a and the image 1220 photographed by the second camera 111b can be controlled so as not to be duplicated. there is.

More specifically, the user authentication server 200, after a first user authentication request is made for a face to be authenticated having a specific ID for access to the first access control gate 20a among a plurality of access control gates, Within a set time, if a second user authentication request is made for the face to be authenticated with the specific ID for access to the second access control gate 20b, user authentication according to the second user authentication request is controlled not to be performed. can do.

As shown in (a) of FIG. 12 , in order to pass through the first access gate 20a, the first user 1000a uses the first camera 111a related to the first access gate 20a. You can try authentication.

In this case, as shown in (b) and (c) of FIG. 12 , the first face 2000a of the first user 100a includes an image 1210 captured by the first camera 111a and a second face 2000a. All may be included in the image 1220 captured by the camera 111b.

The controller 160 determines whether the first face 2000a recognized from the image 1210 captured by the first camera 111a and the image 1220 captured by the second camera 1111b satisfy preset face conditions. , User authentication requests to the user authentication server 200 may be repeatedly performed.

That is, the server 200 receives a first user authentication request for the first face 2000a recognized from the image 1210 photographed by the first camera 111a and the second camera 111b from the system 100. A second user authentication request for the first face 2000a recognized from the image 1220 captured in may be received.

As shown in FIG. 13 , the server 200 requests a first user authentication for the same first face 2000a and a second user authentication request for the same first face 2000a within a preset time for access to different access control gates 20a and 20b. If there is a user authentication request, after processing the first user authentication request, user authentication may not be performed on the second user authentication request.

Furthermore, the server 200 transmits, to the system 100, no user authentication result for the second user authentication request or information notifying that user authentication has not been performed (or duplicated with the first user authentication request). can transmit

Based on the second user authentication request related to the second entrance gate 20, the controller 160 determines that the user authentication result has not been received from the server 200 or that user authentication has not been performed (or first user authentication has not been performed). When information notifying that the request is duplicated) is received, it is possible to control not to perform control according to the user authentication result.

For example, as shown in FIG. 13 , when the server 200 performs first user authentication on the first face 2000a recognized from the image 1210 captured by the first camera 111a at time t0 , The system 100 controls the operation of the first access control gate 20a associated with the first camera 111a, and displays the user authentication result or target on the first display unit 140a associated with the first camera 111a. Guidance information related to function execution can be controlled to be output.

As another example, as shown in FIG. 13 , upon receiving the second user authentication request for the first face 2000a recognized from the image 1220 captured by the second camera 111b at time t1, the server 200 ) may not perform user authentication for the second user authentication based on the fact that there is a first user authentication request for the same first face 2000a within a preset time. At this time, user authentication for the first user may be treated as a failure.

As such, in the present invention, even if the same face is recognized in images acquired from different cameras 111a and 111b related to the plurality of access control gates 20a and 20b, user authentication for the duplicately recognized user is not duplicated. As a result, only the operation for the access control gate where the user is located can be accurately processed.

As described above, the face recognition system and its control method according to the present invention acquires an image using a camera provided in a terminal that performs face recognition, and detects a face from the image based on a preset face recognition algorithm. Recognizable. As such, the present invention can accurately and quickly recognize a user's face based on the location of the user around the terminal even without receiving a direct input from the user. Accordingly, the user can more conveniently and intuitively transfer his/her face to the terminal by being located near the terminal.

Furthermore, the face recognition system and method for controlling the face recognition system according to the present invention may determine whether a face recognized in an image is suitable for performing user authentication. That is, in the present invention, by checking whether a face recognized in an image satisfies a preset face condition, when a face unsuitable for user authentication is recognized, user authentication is not performed, thereby improving data processing efficiency. there is.

Furthermore, the face recognition system and method for controlling the same according to the present invention perform user authentication only for users who satisfy a preset face condition, so that even when there are a plurality of users around the terminal, the user's face attempting user authentication is performed. can be accurately recognized to perform user authentication.

Furthermore, the face recognition system and method for controlling the face recognition system according to the present invention may request user authentication for a face recognized by a preset server based on whether a face recognized in an image satisfies a preset face condition. As described above, the face recognition system and method for controlling the face recognition system according to the present invention perform user authentication on the face of a user who truly wants user authentication, so that the user who has not attempted user authentication or the user's surroundings who have attempted user authentication It is possible to improve the accuracy and reliability of user authentication results by reducing errors in which user authentication is performed for other users in .

On the other hand, the computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. there is

Furthermore, the computer-readable medium may be a server or cloud storage that includes a storage and can be accessed by a terminal through communication. In this case, the computer may download the program according to the present invention from a server or cloud storage through wired or wireless communication.

Furthermore, in the present invention, the above-described computer is a processor, that is, a terminal equipped with a CPU (Central Processing Unit), and no special limitation is placed on its type.

On the other hand, the above detailed description should not be construed as limiting in all respects and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (16)

A control method of a face recognition system that performs control based on a face recognition result,
Obtaining an image using a camera provided in a terminal that performs a face recognition function;
Recognizing a face from the image based on a preset face recognition algorithm;
checking whether the recognized face in the image satisfies a preset face condition;
requesting user authentication for the recognized face to a preset server to perform user authentication for the recognized face based on a result of the check; and
and performing a control according to a user authentication result for the recognized face based on a user authentication result for the recognized face received from the server. According to claim 1,
The preset face condition is,
It is related to at least one of a position including the recognized face in the image, a size of an area occupied by the recognized face in the image, and an angle of the recognized face in the image,
In the step of requesting user authentication,
and requesting user authentication for the recognized face based on the fact that the recognized face satisfies the preset face condition.
According to claim 2,
In the step of recognizing the face,
A specific ID (Identification) for specifying the recognized face is assigned to the recognized face,
Extracting face information including at least one of location information, size information, and angle information for the recognized face to which the specific ID is assigned from the image;
The control method of the face recognition system, characterized in that the extracted face information is matched with the specific ID assigned to the recognized face and stored in a storage unit. According to claim 3,
If face information matched to the specific ID extracted from at least one other image obtained at a point in time prior to the point in time at which the image was acquired exists in the storage unit, face information extracted from the image and the other image is stored. and determining user authentication intention of the user corresponding to the recognized face by using the face recognition system. According to claim 4,
In the step of requesting user authentication,
When the recognized face satisfies the preset face condition and it is determined that the user has the user authentication intention, requesting user authentication for the recognized face to the server;
Even if the recognized face satisfies the preset face condition, if it is determined that the user does not have the user authentication intention, the server is not requested for user authentication on the recognized face. How to control the system. According to claim 4,
In the step of determining the user authentication intention,
Based on the face information extracted from the image and the other image, calculating the size and position change of the face corresponding to the specific ID in the image and the other image,
Based on the amount of change, the control method of the face recognition system, characterized in that for determining the user authentication intention of the user.
According to claim 2,
In the checking step,
When a plurality of faces are recognized from the image, checking whether the predetermined face condition is satisfied for each of the plurality of recognized faces;
Specifying a face that satisfies the preset face condition among the plurality of faces,
In the step of requesting user authentication,
A control method of a face recognition system, characterized in that requesting user authentication for the specified face to the server. According to claim 7,
The specified face,
A face included in a face region having a predetermined size or more among face regions occupied by each of the plurality of faces in the image,
When there are a plurality of face regions having a predetermined size or more in the image,
The control method of a face recognition system, characterized in that the user authentication is performed on a face included in the largest face region among a plurality of face regions. According to claim 7,
As a result of the check, when there is no face among the plurality of faces that satisfies the preset face condition,
To the server, a user authentication request based on the image is not made. According to claim 3,
In the step of performing control according to the user authentication result, control is performed on the access control gate having any one of an open state and a closed state,
The access control gate,
When user authentication for the recognized face is successful, the user corresponding to the recognized face is controlled to have an open state for a predetermined time to pass through the access control gate;
A control method of a face recognition system, characterized in that having a closed state so that the access control function is performed after the predetermined time has elapsed. According to claim 10,
A plurality of faces are recognized from the image,
As a result of user authentication performed by the server, when user authentication for each of the plurality of faces is successful,
The access control gate,
The control method of the face recognition system, characterized in that controlled to have an open state until a plurality of users corresponding to each of the plurality of faces pass through.
According to claim 10,
In order to control access to a specific space, in a state where a plurality of different access control gates exist in the specific space,
After a first user authentication request for the recognized face having the specific ID is made to the server for access to a first access control gate among the plurality of access control gates,
Within a predetermined time, when a second user authentication request is made for the recognized face having the specific ID for access to a second access control gate among the plurality of access control gates,
In the server, user authentication according to the second user authentication request is not performed. According to claim 10,
In the step of performing the control,
After the access control gate is controlled according to the user authentication result for the face having the specific ID,
When a user authentication result for the face with the specific ID is additionally received from the server, the control of the access control gate according to the user authentication result additionally received is limited. . a camera unit that acquires an image;
a storage unit for storing the obtained image; and
Based on a preset face recognition algorithm, a controller for recognizing a face from the image and checking whether the recognized face in the image satisfies a preset face condition,
The control unit,
Based on the result of the check, in order to perform user authentication for the recognized face, request user authentication for the recognized face to a preset server;
and performing control according to a user authentication result for the recognized face based on a user authentication result for the recognized face received from the server. a camera unit that acquires an image;
a storage unit for storing the obtained image; and
a controller that recognizes a face from the image based on a preset face recognition algorithm and checks whether the recognized face in the image satisfies a preset face condition;
When the recognized face satisfies the preset face condition, a server performing user authentication on the recognized face;
The control unit,
The face recognition system, characterized in that performing control according to the user authentication result for the recognized face based on the user authentication result for the recognized face performed in the server.
A program that is executed by one or more processes in an electronic device and stored in a computer-readable recording medium,
said program,
Obtaining an image using a camera provided in a terminal performing a face recognition function;
Recognizing a face from the image based on a preset face recognition algorithm;
checking whether the recognized face in the image satisfies a preset face condition;
requesting user authentication for the recognized face to a preset server to perform user authentication for the recognized face based on a result of the check; and
Based on the user authentication result for the recognized face received from the server, performing a control according to the user authentication result for the recognized face. A program stored on a recordable medium.

Images