JP7429885B2 - Facial recognition system and method - Google Patents

Description

The present invention relates to a face authentication system and a face authentication method that perform face authentication processing based on image data obtained by photographing a subject.

Conventionally, as a technology related to face matching, at least a feature extracting means, a registrant feature storing means, and a matching means are configured in a first device, and the registrant face image storing means has a security level higher than that of the first device. The system consists of a second device that is stored in a high place, and when the first device needs to use the facial image of the registrant, it establishes a data communication path with the second device. , the facial image of the registrant is acquired from the second device through the data communication path, and the facial image data of the registrant is used on the work memory without being stored in a non-volatile recording medium, and the face verification system A device is known that makes it possible to protect a registrant's facial image as much as possible even if it is stolen or attacked (see Patent Document 1).

Patent No. 5353147

Now, in the conventional technology, a face authentication machine that has an imaging means and facial feature data of all registrants and executes face matching processing is installed in a physical space, and the facial image data of the registrants is sent to an image server, etc. It is managed in cyberspace. Therefore, no consideration should be given to a configuration in which an authentication server is provided, that is, a system in which both feature data and facial image data are placed on a cloud server, etc., so that the service provider can stably and efficiently operate the facial recognition function. It has not been.

Therefore, the main object of the present invention is to provide a face authentication system and a face authentication method that can stably and efficiently operate a face authentication function for each face authentication service provider.

The face recognition system of the present invention is a face recognition system that executes face recognition processing based on image data of a person to be face recognized. and a server device connected thereto, and the face authentication device has a camera that photographs the target person who appears one after another , and collates the face image data of the target person acquired by the camera and the face recognition device. a face matching request unit that transmits a face matching request including a group and matching conditions to the server device , and an authentication result receiving unit that receives an authentication result of the subject from the server device, the server device a request receiving unit that receives the face matching request of the target person transmitted from a face authentication device; and a facial feature generation unit that executes a process of generating facial feature data of the target person from facial image data of the target person. a facial feature matching unit that performs matching processing between the target person's facial feature data and the registrant's facial feature data for each matching group ; an authentication result transmitting unit that transmits a face matching response including an authentication result of a person to the face authentication device , the face authentication device instructing a threshold value regarding a matching score as the matching condition included in the face matching request. The server device is configured to transmit the face matching response including the matching score that is equal to or higher than the threshold value to the face authentication device when the authentication result of the target person is successful.

Further, the face authentication method of the present invention is a face authentication method in which a face authentication process is executed by a face authentication machine and a server device based on image data obtained by photographing a person to be subjected to face authentication, and the face authentication method is a face authentication method in which a face authentication process is executed by a face authentication machine and a server device, and the face authentication process is performed by a face authentication machine and a server device. The processors of the plurality of face recognition machines connected to each other are configured to take pictures of the target person who appears one after another with a camera, and to process the face image data of the target person obtained by taking pictures with the camera, the matching group of the face recognition machine, and the matching. transmitting a face matching request including conditions to the server device , receiving an authentication result of the target person from the server device, and a processor of the server device receiving the face matching request of the target person from the face authentication device; Then, a process of generating facial feature data of the target person from the facial image data of the target person is performed, and a process of matching the facial feature data of the target person and facial feature data of the registrant is performed for each matching group. and transmits a face matching response including the obtained authentication result of the subject to the face authentication device, and further, the processor of the face authentication device executes the following steps as the matching conditions included in the face matching request: A configuration in which the processor of the server device instructs a threshold value regarding the matching score, and when the authentication result of the subject person is successful, transmits the face matching response including the matching score that is equal to or higher than the threshold value to the face authentication machine. shall be.

According to the present invention, at the time of face authentication , a face matching request including facial image data of a target person obtained by photographing with a camera, a matching group of a face authentication device, and matching conditions is sent to the server device, and In addition to executing the face matching process, it is possible to specify a threshold value regarding the matching score as a matching condition, and if the authentication result is successful, a face matching response including the matching score that is equal to or higher than the threshold value can be sent to the face recognition machine. . This allows each facial recognition service provider to stably and efficiently operate the facial recognition function.

Overall configuration diagram of a face recognition system according to this embodiment Explanatory diagram showing an overview of matching groups Explanatory diagram showing an overview of face verification inquiries performed by face recognition device 1 Explanatory diagram showing an overview of the face matching unit Block diagram showing the schematic configuration of face recognition device 1 Block diagram showing a schematic configuration of the management terminal 2 Block diagram showing a schematic configuration of the face management server 5 Block diagram showing a schematic configuration of the face matching server 6 Explanatory diagram showing an overview of face image extraction processing performed by face authentication device 1 Explanatory diagram showing an overview of face extraction processing performed by face authentication device 1 Explanatory diagram showing an overview of face extraction processing performed by face authentication device 1 Explanatory diagram showing an overview of face image extraction processing Explanatory diagram showing an overview of face recognition inquiries using matching group information Explanatory diagram showing the login screen displayed on the management terminal 2 Explanatory diagram showing the screen displayed on the management terminal 2 when registering, referencing, updating, and deleting a verification group Explanatory diagram showing the screen displayed on the management terminal 2 when registering, referencing, updating, and deleting verification groups Sequence diagram showing the steps for registering, referencing, and updating processing for collation groups Explanatory diagram showing an overview of registration, reference, and update processing related to matching groups Sequence diagram showing the procedure for deletion processing related to matching groups Explanatory diagram showing an overview of deletion processing related to matching groups Sequence diagram showing the steps for batch registration processing regarding verification groups An explanatory diagram showing a screen displayed on the management terminal 2 when registering, referencing, updating, and deleting the association between the face authentication device 1 and the matching group An explanatory diagram showing a screen displayed on the management terminal 2 when registering, referencing, updating, and deleting the association between the face authentication device 1 and the matching group Sequence diagram showing the procedure of registration processing regarding association between face recognition device 1 and matching group Explanatory diagram showing an overview of registration processing regarding association between face authentication device 1 and matching group Sequence diagram showing the procedure of reference processing regarding association between face recognition device 1 and matching group Sequence diagram showing the procedure of update processing regarding association between face recognition device 1 and matching group Sequence diagram showing the procedure of deletion processing regarding association between face recognition device 1 and matching group Explanatory diagram showing a screen displayed on the management terminal 2 when registering, referencing, and deleting an association between the face matching server 6 and a matching group Explanatory diagram showing a screen displayed on the management terminal 2 when registering, referencing, and deleting an association between the face matching server 6 and a matching group Sequence diagram showing the procedure of registration processing regarding association between face matching server 6 and matching group Sequence diagram showing the procedure of reference processing regarding association between face matching server 6 and matching group Sequence diagram showing the procedure of deletion processing regarding association between face matching server 6 and matching group Explanatory diagram showing an overview of deletion processing regarding association between face matching server 6 and matching group Explanatory diagram showing the screen displayed on the management terminal 2 when registering, referencing, updating, and deleting users Explanatory diagram showing the screen displayed on the management terminal 2 when registering, referencing, updating, and deleting users Sequence diagram showing the steps of user registration process Explanatory diagram showing an overview of user registration processing Sequence diagram showing the procedure for bulk registration processing for users Sequence diagram showing the procedure for reference processing regarding users Sequence diagram showing the steps of update processing regarding users Explanatory diagram showing an overview of update processing regarding users Sequence diagram showing the procedure for deletion processing regarding users Explanatory diagram showing an overview of user-related deletion processing Explanation showing the outline of face image extraction processing Explanatory diagram showing an overview of image quality check processing Explanatory diagram showing an overview of facial feature amount calculation processing Explanatory diagram showing an overview of facial feature registration processing (feature specification) Explanatory diagram showing an overview of facial feature registration processing (image specification) Explanatory diagram showing an overview of facial feature deletion processing Explanatory diagram showing an overview of face registration Explanatory diagram showing an overview of face information synchronization processing Explanatory diagram showing an overview of facial feature data copy processing Explanatory diagram showing an overview of processing during face recognition Explanatory diagram showing an overview of reference processing related to authentication logs Explanatory diagram showing the authentication log reference screen displayed on the management terminal 2 Explanatory diagram showing an overview of external application cooperation of face recognition device 1 Explanatory diagram showing an overview of the operating status monitoring function in this system Explanatory diagram showing a backup screen and a restore screen displayed on the management terminal 2 An explanatory diagram showing an overview of face image extraction processing according to a modification of the present embodiment

A first invention made in order to solve the above problem is a face recognition system that executes face recognition processing based on image data of a person to be face recognized. It has an authentication device and a server device connected via a network, and the face authentication device includes a camera that photographs the target person who appears one after another , and facial image data of the target person acquired through photography by this camera. , comprising a face matching request unit that transmits a face matching request including a matching group and matching conditions of the face authentication device to the server device, and an authentication result receiving unit that receives the authentication result of the subject from the server device. , the server device includes a request receiving unit that receives the face verification request of the target person transmitted from the face authentication device, and a process of generating facial feature amount data of the target person from facial image data of the target person. a facial feature quantity generation unit that executes a process, a facial feature quantity matching unit that executes a matching process between the target person's facial feature quantity data and the registrant's facial feature quantity data for each of the matching groups , and this facial feature quantity matching unit. an authentication result transmitting unit that transmits a face matching response including an authentication result of the subject person by the matching process to the face authentication machine, and the face authentication machine is configured to perform the matching condition as the matching condition included in the face matching request. , a threshold value regarding the matching score is specified, and the server device transmits the face matching response including the matching score that is equal to or higher than the threshold value to the face authentication device when the authentication result of the subject person is successful. do.

According to this, when performing face recognition, a face matching request including facial image data of the subject obtained through camera photography, matching groups of the face recognition device, and matching conditions is sent to the server device, and the face is identified in matching groups. While executing the matching process, it is possible to specify a threshold regarding the matching score as a matching condition, and if the authentication result is successful, a face matching response including the matching score equal to or higher than the threshold can be sent to the face authentication machine . This allows each facial recognition service provider to stably and efficiently operate the facial recognition function.

In a second aspect of the present invention, the server device includes an authentication history management section that manages history information of face authentication including the authentication results and matching scores obtained in the matching process .

According to this, it is possible to manage the authentication log regarding the authentication results for the target person's face image acquired from the face authentication device.

Further, in a third aspect of the present invention, the server device has a plurality of facial feature value matching units for each of the matching groups corresponding to the face authentication devices.

According to this, it is possible to specify the conditions for the matching process to the facial feature amount matching unit with matching matching groups.

Further, a fourth invention is a face authentication method in which a face authentication machine and a server device perform face authentication processing based on image data of a photographed person of a person to be facial authenticated, the method being connected to the server device via a network. The processors of the plurality of face recognition devices photograph the target person who appears one after another with a camera, and process the face image data of the target person obtained by the camera photography, the matching group of the face recognition device, and the matching conditions. transmitting a face matching request including a face matching request to the server device , receiving an authentication result of the subject from the server device, a processor of the server device receiving the face matching request of the subject from the face authentication machine, Performing a process of generating facial feature data of the target person from facial image data of the target person, and performing a process of matching the facial feature data of the target person and facial feature data of the registrant for each of the matching groups. Then, a face matching response including the acquired authentication result of the target person is transmitted to the face authentication machine, and further, the processor of the face authentication machine includes a matching score as the matching condition included in the face matching request. and the processor of the server device is configured to transmit the face matching response including the matching score that is equal to or higher than the threshold value to the face authentication machine if the authentication result of the subject is successful.

According to this, similarly to the first invention, the face authentication function can be operated stably and efficiently for each provider of face authentication service.

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is an overall configuration diagram of a face authentication system according to this embodiment.

This face authentication system includes a face authentication device 1, a management terminal 2 (terminal device, face authentication management device), and a face authentication server 3 (cloud server). The face authentication server 3 includes a face management server 5 (face image management section) and a face verification server 6 (face image verification section).

The face authentication device 1, the management terminal 2, the face management server 5, and the face verification server 6 are connected via a network such as the Internet. A facial recognition system is constructed for each facial recognition service provider (operator). A plurality of facial recognition devices 1 are installed at each location where facial recognition is required, such as at the entrance of a building or a room. A required number of face verification servers 6 are provided depending on the number of face authentication machines 1 and the like.

The face authentication device 1 includes a camera 11, and the camera 11 acquires a captured image of the user. The face authentication device 1 also has a display 12, which displays the face authentication results obtained from the face verification server 6 and notifies the user of the face authentication results.

The management terminal 2 is operated by an administrator, is composed of a PC, and has a management application for managing the operations of the face authentication device 1, the face management server 5, and the face matching server 6 installed therein. This management application allows administrators to perform various management tasks. Note that the management application is implemented as a web application.

The face management server 5 centrally manages user information. Specifically, it stores and manages information such as registered users' names and facial image data.

At the time of face authentication, the face matching server 6 acquires the facial image data of the subject of face authentication from the face authentication machine 1, generates facial feature amount data of the subject from the facial image data, and A face that performs face matching by comparing the facial feature data and the facial feature data of the registrant (registered user) stored in the device to determine whether or not the target person is the registrant. Perform authentication.

Furthermore, prior to face authentication, the face matching server 6 acquires the user's facial image data from the management terminal 2 at the time of user registration, and generates the user's facial feature amount data from the facial image data. , stored in its own device. Note that it is also possible to obtain a photographed image of the user from the management terminal 2 and to obtain facial image data from the photographed image at the face matching server 6.

Note that in this embodiment, the face management server 5 (face image management unit) and the face matching server 6 (face image matching unit) are assumed to exist in physically different information processing devices, but they are The information processing device may exist within the information processing device.

Further, in this embodiment, the management terminal 2 and the face management server 5 are provided, but the management terminal 2 and the face management server 5 can be configured by a single information processing device. For example, by installing a management application on the face management server 5, the face management server 5 can also serve as the management terminal 2.

Further, in this embodiment, facial feature amount matching is performed, but face matching is not limited to facial feature amount matching, and a matching method applying machine learning or the like may be adopted. Furthermore, this embodiment can be applied to biometric authentication other than face authentication.

Next, the verification group will be explained. FIG. 2 is an explanatory diagram showing an outline of a matching group.

In this embodiment, users are divided into groups according to the installation location of the face authentication device 1 through which the user undergoes facial authentication, and a matching group is set for each user. Further, similarly to users, the face authentication machines 1 are divided into groups according to the installation location of the face authentication machines 1, and a matching group is set for each face authentication machine 1. Furthermore, the face verification servers 6 corresponding to the face recognition device 1, that is, the face verification servers 6 that receive face verification inquiries (requests) from the face recognition device 1, are divided into groups in the same way as the face recognition device 1, and each A verification group is set in the verification server 6.

For example, in a facial recognition machine 1 installed at the entrance of a multi-tenant building in which multiple companies reside, users belonging to multiple companies undergo facial recognition. On the other hand, information about users belonging to different organizations cannot be managed in the same database. Therefore, in such a case, a plurality of matching groups for each company is formed and user registration information is stored for each matching group, and the face authentication device 1 uses the face matching server 6 of the matching group for each company. A face matching query is made to the

The user can be authenticated only with the face authentication device 1 that corresponds to the verification group to which the user belongs. Therefore, even a registered user cannot be authenticated using a face authentication machine 1 that does not belong to his or her verification group, for example, a face authentication machine 1 that is located in a building that the user is not permitted to enter.
For example, if a user who works at one office goes to another office, he or she cannot be authenticated using the facial recognition device 1 at that office.

Next, a face verification inquiry from the face authentication device 1 to the face verification server 6 will be explained. FIG. 3 is an explanatory diagram showing an outline of a face verification inquiry performed by the face authentication device 1.

The face recognition device 1 makes a face verification inquiry to the face verification server 6 belonging to the same verification group as the device itself, and sends a face verification request to the face verification server 6 serving as the inquiry destination (request destination). In addition, the face recognition device 1 holds the network address (IP address) of the face verification server 6 as verification group information, and based on this network address, the face verification server 6 corresponding to its own device performs face verification. Make inquiries.

This face authentication inquiry method includes a simultaneous inquiry shown in FIG. 3(A), a random inquiry shown in FIG. 3(B), and a sequential inquiry (inquiry destination switching) shown in FIG. 3(C).

As shown in FIG. 3(A), in the case of a simultaneous inquiry, a face matching inquiry is made to the face matching servers 6 corresponding to each of a plurality of matching groups at the same time. For example, if the face recognition machine 1 is installed at the entrance of a multi-tenant building and users in multiple verification groups undergo face recognition, it sends a request for face verification to the face verification server 6 of each group at the same time. .

In addition, in order to reduce the load on one face matching server 6, it is also possible to divide and store the facial feature data of users belonging to one matching group in multiple face matching servers 6. Also, a face verification inquiry is made to all the face verification servers 6 whose registered contents are different from each other within the same verification group as the face authentication device 1.

As shown in FIG. 3(B), in the case of a random inquiry, a face matching server 6 to be contacted for face matching is randomly selected from among multiple face matching servers 6 in the same matching group as the face authentication device 1. do. The registered contents of the plurality of face matching servers 6 are the same. In other words, the same user is used as a matching target, and facial feature amount data of the same user is stored. In such an inquiry method, since inquiries from the face authentication device 1 are distributed to each face matching server 6, the load on each face matching server 6 can be reduced, and load distribution can be achieved.

As shown in FIG. 3C, in the case of a sequential inquiry, a face verification inquiry destination is sequentially selected from among a plurality of face verification servers 6 in the same verification group as the face authentication device 1. Specifically, an order (priority) is given to the face matching servers 6, and the face matching server 6 is selected according to that order, and if there is no response from a higher ranked face matching server 6, the destination is selected. Switch to face matching server 6. That is, the first-ranked face matching server 6 in the matching group is selected and a face authentication request is made to that face matching server 6. If there is no response from the face matching server 6, another face matching server 6 of the next rank within the same matching group is selected and a face authentication is requested to that face matching server 6. Such control enables redundancy (backup).

Next, the face matching unit will be explained. FIG. 4 is an explanatory diagram showing an overview of the face matching unit.

The face matching server 6 can be provided with a plurality of face matching units that perform face matching. This face matching unit includes a face matching process that executes a face matching program, and a face matching database in which information (facial feature amount data) of users to be compared in face matching is registered.

This face verification unit is provided for each verification group, and performs face verification in response to a face verification request from the face authentication device 1 belonging to the verification group. Therefore, one face matching server 6 can support multiple matching groups.

Here, the face authentication device 1 holds the network address (IP address) of the face matching server 6 as matching group information, and based on this network address, the face matching server 6 corresponding to the device itself is A verification inquiry can be made, but in this embodiment, a network address (IP address) is also held for each face verification unit, and based on this network address, the face verification unit corresponding to the device itself is contacted. Face verification queries can be made.

Furthermore, since the face matching database is provided for each matching group, the user's facial feature amount data can be stored for each matching group. The face matching process executes matching processing between facial feature data stored in groups and facial feature data generated from face image data acquired from the face authentication devices 1 of the same group.

Here, there are various aspects of the correspondence between the face authentication device 1 and the face verification unit.

In the example shown in FIG. 4A, the face verification server 6 is provided with one face verification unit of the verification group to which the face recognition device 1 belongs, and the face verification device 1 and the face verification server 6 are connected on a one-to-one basis. It is supported by

In the example shown in FIG. 4(B), the face authentication device 1 and the face matching server 6 have a one-to-one correspondence, but the face matching server 6 has two face matching units of the matching group to which the face authentication device 1 belongs. It is provided.
In this case, in the case of the face recognition machine 1 installed at the entrance of a multi-tenant building, information on users belonging to different organizations (for example, user A and user B) may be managed in different matching databases. I can do it.

In the example shown in FIG. 4(C), the face authentication device 1 and the face matching server 6 have a one-to-two correspondence, and the face authentication device 1 makes a face matching inquiry (user Verification request for X) is performed. In this case, by providing the same face matching unit, that is, two or more face matching units that perform face matching using facial feature data of the same user, in different face matching servers 6, the load on the face matching server 6 can be distributed. It is possible to take life-or-death measures.

Next, a schematic configuration of the face authentication device 1 will be explained. FIG. 5 is a block diagram showing a schematic configuration of the face authentication device 1. As shown in FIG.

The face authentication device 1 includes a camera 11 (imaging section), a display 12 (output section), a communication section 13, a storage section 14, and a control section 15.

The camera 11 constantly photographs a predetermined photographing area, and when a person enters the photographing area, the person is photographed, and a photographed image of the person to be subjected to face authentication can be obtained. Note that in order to save power, a human sensor (not shown) may be provided to detect the arrival of a person and activate the camera 11.

The display 12 displays the face authentication result, so that the person undergoing face authentication can check the face authentication result. Note that a speaker may be provided as an output unit for outputting the authentication result of the target person, and the face authentication result may be notified by voice.

The communication unit 13 (face image transmitting unit, authentication result receiving unit) communicates with the face matching server 6 via the network. In this embodiment, face image data is sent to the face matching server 6. It also receives the user's authentication result from the face matching server 6. Furthermore, the communication unit 13 communicates with the face management server 5 via the network. In this embodiment, matching group information and the like are received from the face management server 5.

The storage unit 14 stores verification group information, a control program executed by a processor forming the control unit 15, and the like.

The control unit 15 includes a startup processing unit 21, a face image extraction unit 22, a face matching request unit 23, an authentication result notification unit 24, an authentication result adjustment unit 25, and an operating status monitoring unit 26. . This control section 15 is composed of a processor, and each section of the control section 15 is realized by executing a program stored in the storage section 14 with the processor.

The activation processing unit 21 acquires matching group information from the face management server 5 and stores it in the storage unit 14 when the face authentication device 1 is activated. Note that the acquisition of the verification group information is not limited to this, and the verification group information may be distributed from the face management server 5 to the target face authentication device 1 when the verification group information is updated.

The face image extraction unit 22 acquires a photographed image of the person to be subjected to face authentication from the camera 11 (camera image capture), detects the face of the person from the photographed image (face detection), and determines the size of the detected face. It is determined whether the image is appropriate (face size check), a face area is cut out from the photographed image (face cutout), and face image data of the subject is obtained. Note that the face image data may be data of only the image of the face area, but may also be a combination of photographed image data and position information (face frame information) of the face area on the photographed image data.

The face matching request unit 23 sends a face matching request to the face matching server 6 whose matching group matches that of the own device.

The authentication result notification unit 24 displays the face authentication result obtained from the face matching server 6 on the display 12 and notifies the user of the face authentication result.

The authentication result adjustment unit 25 performs control for linking the user's face matching result obtained from the face matching server 6 with the management server 8 (external device) shown in FIG. 57, which will be described later, as an external system. The authentication result adjustment unit 25 is configured with a connection application, and notifies the management server 8 of the external system of the user's face verification results.

The operating status monitoring unit 26 monitors the operating status of its own device and notifies the face management server 5 of the operating status of its own device.

Next, the management terminal 2 will be explained. FIG. 6 is a block diagram showing a schematic configuration of the management terminal 2. As shown in FIG.

The management terminal 2 includes a communication section 31, a display 32 (display section), an input device 33 (operation section), a storage section 34, and a control section 35.

The communication unit 31 communicates with the face management server 5 via the network. In the present embodiment, screen information and the like are received from the face management server 5, and corresponding administrator operation information and the like are transmitted to the face management server 5.

The display 32 displays various screens. The input device 33 is a mouse, a keyboard, or the like, and operates the screen displayed on the display 32.

The storage unit 34 stores programs (management applications) and the like that are executed by the processor that constitutes the control unit 35 .

The control section 35 includes a GUI control section 41. The control section 35 is composed of a processor, and each section of the control section 35 is realized by executing a program (management application) stored in the storage section 34 with the processor.

The GUI control unit 41 displays various operation screens distributed from the face management server 5 on the display 32. Further, in response to the administrator's input operation using the input device 33, input information is acquired and screen control is performed. In this embodiment, display input control regarding a screen related to login, specifically, a login screen is performed. It also controls display input on screens related to user management, specifically, screens related to user registration (individual registration, bulk registration), reference, update, and deletion. It also performs display input control for screens related to verification group management, specifically, screens related to registration (individual registration, bulk registration), reference, update, and deletion regarding verification groups. It also performs display input control for screens related to authentication device management, specifically, screens related to registration, reference, update, and deletion of associations between authentication devices and verification groups. It also controls display input regarding the authentication log reference screen.

Next, a schematic configuration of the face management server 5 will be explained. FIG. 7 is a block diagram showing a schematic configuration of the face management server 5. As shown in FIG.

The face management server 5 includes a communication section 51, a storage section 52, and a control section 53.

The communication unit 51 communicates with the management terminal 2 via the network. Furthermore, the communication unit 51 communicates with the face authentication device 1 via the network. Further, the communication unit 51 communicates with the face matching server 6.

The storage unit 52 stores a face information database, a database regarding association information between the face authentication device 1 and the verification group, a database regarding association information between the face verification server 6 and the verification group, an administrator access log, and the control unit 53. It stores control programs executed by the constituent processors.

In the face information database, the user's name, face image, matching group, etc. are registered as information regarding the registered user. Note that the user's face image may be stored in an encrypted state to protect privacy.

The control unit 53 includes an administrator access management unit 61, a user management unit 62, a verification group management unit 63, a face authentication device management unit 64, a face verification server management unit 65, a face information management unit 66, It includes a database management section 67, an operation status monitoring section 68, and an authentication log presentation section 69. This control section 53 is composed of a processor, and each section of the control section 53 is realized by executing a program stored in the storage section 52 with the processor. Each part of the control unit 53 is configured as a Web API (Web Application Programming Interface).

The administrator access management unit 61 allows or denies access (login) of the administrator depending on the access permission group of the administrator who accesses the own device from the management terminal 2.

The administrator access management unit 61 also manages access from the management terminal 2 to the own device, and when the administrator operates the management terminal 2 to access the own device, the administrator access management unit 61 records the information at that time in an administrator access log (history). information) in the storage unit 52. Further, in response to a reference request from the management terminal 2, the administrator access log is presented to the management terminal 2.

The user management unit 62 manages information regarding users and performs necessary processing regarding the users in response to requests from the management terminal 2. In this embodiment, requests for registration, reference, update, and deletion regarding users are made from the management terminal 2, and necessary processing is performed in response to the requests.

The verification group management unit 63 manages information regarding verification groups, and performs necessary processing regarding verification groups in response to requests from the management terminal 2. In this embodiment, the management terminal 2 makes a request for registration, reference, update, and deletion regarding a verification group, and necessary processing is performed in response to the request. In addition, matching group information, that is, information necessary for the face authentication device 1 to request face authentication from the face matching server 6 that matches its own matching group, is generated for each face authentication device 1, and the matching group information is generated for each face authentication device 1. The information is provided to the face authentication device 1.

The face recognition device management unit 64 manages information regarding the face recognition device 1 and performs necessary processing regarding the face recognition device 1 in response to requests from the management terminal 2. In this embodiment, a request is made from the management terminal 2 for registration, reference, update, and deletion regarding the association between the face authentication device 1 and the matching group, and necessary processing is performed in response to the request.

The face matching server management unit 65 manages information regarding the face matching server 6 and performs necessary processing regarding the face matching server 6 in response to requests from the management terminal 2. In this embodiment, the management terminal 2 makes a request for registration, reference, and deletion regarding the association between the face matching server 6 and the matching group, and necessary processing is performed in response to the request.

The face information management unit 66 manages the user's face information stored in the own device and the face matching server 6, respectively. In this embodiment, the user's facial information (user's first and last name, facial image, etc.) stored in the own device and the user's facial information (user's facial feature amount data) stored in the face matching server 6 are used. Facial information is synchronized so that it is maintained in a consistent state. Further, the face information management unit 66 copies face information (user's facial feature amount data).

The database management unit 67 manages a database provided in its own device, and performs backup and restoration of the database.

The operating status monitoring unit 68 monitors the operating status of its own device, receives notifications of the operating status from the face recognition device 1 and the face matching server 6, and detects the face according to the operation of the administrator on the management terminal 2. The operating status of the authentication device 1, its own device (face management server 5), and face matching server 6 is displayed on the screen on the management terminal 2.

The authentication log presentation unit 69 acquires the authentication log from the face verification server 6 in response to a reference request from the management terminal 2, and presents the authentication log to the management terminal 2.

Next, a schematic configuration of the face matching server 6 will be explained. FIG. 8 is a block diagram showing a schematic configuration of the face matching server 6. As shown in FIG.

The face verification server 6 includes a communication section 71, a storage section 72, and a control section 73.

The communication unit 71 (face image receiving unit, authentication result transmitting unit) communicates with the face authentication device 1 via the network. In this embodiment, facial image data and the like are received from the face authentication device 1. It also transmits the user's authentication results and the like to the face authentication device 1. Furthermore, the communication unit 71 communicates with the face management server 5 via the network. In this embodiment, requests for various types of processing are received from the face management server 5, and responses and the like corresponding thereto are transmitted to the face management server 5.

The storage unit 72 stores a face matching database, association information between a user and a feature amount, information regarding a matching group of the own device, an authentication log, a control program executed by a processor constituting the control unit 73, and the like.

In the face matching database, user's facial feature data and the like are registered as information regarding registered users. Further, a face matching database is provided for each matching group, and stores facial feature amount data of users on a group-by-group basis. Here, the registered user's facial feature data may be backed up and saved in a non-volatile memory such as an HDD or SSD provided in the face matching server 6 in case it disappears unexpectedly from the face matching database. good.

The control unit 73 includes a matching group management unit 81 , an image quality check unit 82 , a face image extraction unit 83 , a face feature generation unit 84 , a face feature management unit 85 , a face feature matching unit 86 , It includes an authentication log management section 87, a database management section 88, and an operation status monitoring section 89. This control section 73 is composed of a processor, and each section of the control section 73 is realized by executing a program stored in the storage section 72 with the processor. Each part of the control unit 73 is configured as a Web API.

The verification group management unit 81 manages the verification group to which the device itself belongs, and performs registration and deletion processing regarding the verification group in response to a request from the face management server 5.

The image quality check unit 82 determines whether the image of the face area within the captured image satisfies a predetermined quality. Specifically, from the target image, it is detected whether a mask is being worn or not and whether sunglasses are being worn, and the facial recognition suitability (evaluation value based on facial orientation and expression) is calculated.

The face image extraction unit 83 extracts a face image from the captured image of the user obtained from the management terminal 2 at the time of user registration. At this time, a person's face is detected from the captured image (face detection), it is determined whether the size of the detected face is appropriate (face size check), a facial area is cut out from the captured image (face cropping), and the person's face is detected (face detection). Obtain a face image of. Note that during face authentication, a face image may be extracted from a photographed image of the subject acquired from the face authentication device 1.

The facial feature generation unit 84 detects facial feature points from face image data and generates facial feature data during user registration and face authentication.

At the time of user registration, the facial feature amount management section 85 registers the user's facial feature amount data generated by the facial feature amount generation section 84 in the face matching database corresponding to the user's matching group. Furthermore, when updating or deleting, the facial feature amount data registered in the face matching database is deleted in response to a request from the face management server 5. In addition, when the version of the program related to facial feature generation and face matching algorithms is upgraded, the facial feature data registered in the face matching database is updated in response to a request from the face management server 5. Update to match the program.

At the time of face authentication, the facial feature matching unit 86 compares the facial feature data of the target person generated from the facial image data acquired from the face authentication device 1 and the registrant (registered user) stored in its own device. It is determined whether the target person is a registrant by comparing the face feature amount data. This facial feature amount matching unit 86 corresponds to the matching group associated with the own device, and when multiple matching groups are associated with the own device, the facial feature matching section 86 performs multiple facial feature matching for each matching group. It has a section 86. This facial feature matching section 86 (face matching process) constitutes a face matching unit in combination with a face matching database.

Note that the facial feature matching unit 86 calculates the degree of similarity (matching score) between the target person and the registrant. By comparing this degree of similarity with a predetermined threshold, it is possible to determine the success or failure of face authentication. In addition to notifying the face authentication device 1 of this success/failure determination result as a comparison result, the face authentication device 1 may be notified of the person ID and degree of similarity related to registrants with a high degree of similarity as a comparison result.

When the face authentication device 1 makes a request for face authentication to its own device, the authentication log management unit 87 collects the information at that time, such as the device ID of the face authentication device 1 that made the request for face authentication, the face matching result, etc. The information is stored in the storage unit 72 as an authentication log. Further, in response to a request from the face management server 5, the authentication log is provided to the face management server 5.

The database management unit 88 manages a database provided in its own device, and performs backup and restoration of the database.

The operating status monitoring unit 89 monitors the operating status of its own device and notifies the face management server 5 of the operating status of its own device.

In this embodiment, the main functions of the face matching server 6 include facial image extraction, facial feature amount generation, and facial feature amount matching, but these functions are performed by separate information processing independent of each other. It can be configured with a device. For example, the facial image extraction function may be configured in a separate information processing device that is independent from other functions such as facial feature amount generation and facial feature amount matching.

Next, a description will be given of the processing performed by the face image extraction unit 22 of the face authentication device 1 during face authentication. FIG. 9 is an explanatory diagram showing an overview of the processing performed by the face image extraction unit 22 of the face authentication device 1.

The face authentication device 1 can be provided with a plurality of cameras 11. In this case, a camera image capture process for acquiring a captured image from the camera 11 is provided for each camera 11. This camera image capture process is started when the face authentication device 1 is activated, and passes the photographed image acquired from the camera 11 to the control process. The control process performs face detection, face size check, and face cutting on images taken by each camera 11.

In the face authentication device 1, the control process reads the configuration file and executes the camera image capture process based on the configuration information for each camera 11 written in the configuration file. At this time, during execution, the camera image capture process receives information such as camera type, camera connection information, image acquisition rate, and acquisition image size from the control process as parameters.

In face detection, a face area included in an image captured by the camera 11 is detected. Based on the detection results, a face frame (a rectangular frame surrounding the face area) is set.

Here, the image taken by the camera 11 may include a plurality of face areas. In this case, a face area to be subjected to face authentication is selected from among the plurality of detected face areas (detected face selection process). At this time, only one face area to be subjected to face authentication may be selected, but a plurality of face areas to be subjected to face authentication may be selected. In this case, the administrator may be able to specify the number of face areas to select.

In this detected face selection process, a face area to be subjected to face authentication may be selected based on the area of the face frame. That is, the area of the face frame is calculated from the width and height of the face frame, the areas of each face frame are compared, and a predetermined number of face areas are selected in descending order of area. Note that when selecting only one face area, the face area with the largest area may be selected. Furthermore, if the areas of the face frames are the same, the face areas may be selected in the order in which they are detected.

Further, the face area to be subjected to face authentication may be selected based on the width of the face frame. That is, the widths of each face frame are compared and a predetermined number of face areas are selected in descending order of width. Note that when selecting only one face area, the face area with the largest width may be selected. Furthermore, if the widths of the face frames are the same, the face areas may be selected in the order in which they are detected.

Note that, for example, grayscale conversion is performed as preprocessing for face detection.

In the face size check process, it is determined whether the size of the face area (face frame) detected from the photographed image satisfies a predetermined standard (appropriate range). Specifically, when the width and height of the face frame obtained through face detection are within a predetermined range, it is determined that the size of the face area is appropriate. Specifically, thresholds (minimum and maximum values) are set for each of the width and height of the face frame, and the width and height of the face frame are compared with the respective minimum and maximum values to determine the width and height of the face frame. If the width of the face frame is greater than or equal to the minimum value and less than the maximum value, and at the same time, the height of the face frame is greater than or equal to the minimum value and less than the maximum value, it is determined that the size of the face area is appropriate.

10 and 11 are explanatory diagrams showing an overview of the face extraction process performed by the face authentication device 1.

In the face cropping process, a cropping area is set for the image captured by the camera based on the face area (face frame) obtained by face detection, and the image of that cropping area (face cropping frame) is used as a face image. Perform the cutting process. The cutout area is set with the face area as the center and expanded vertically and horizontally.

In the example shown in FIG. 10A, the expansion coefficient k is set to 3, and an area three times larger than the face area detected by face detection is set as the cutout area. In the example shown in FIG. 10(B), the expansion coefficient k is set to 4, and an area four times as large as the face area is set as the cutout area.

Here, as shown in FIG. 10(B), from the coordinates (x, y) of the top left vertex A of the face frame and the width w and height h of the face frame, the coordinates (x' , y') can be obtained as shown below.
(x', y') = ((x+w)/2, (y+h)/2)
Further, from the width w and height h of the face frame and the expansion coefficient k, the width W and height H of the cutout area are determined as shown in the following equation.
W=w×k
H=h×k
Then, from the coordinates (x', y') of the center B of the face frame and the width W and height H of the cropping area, the coordinates (X, Y) of the top left vertex C of the cropping area are calculated as follows: Desired.
(X, Y) = (x'-(W/2), y'-(H/2))
The cutout area is defined by the coordinates (X, Y) of the top left vertex C of the cutout area, the width W, and the height H of the cutout area.

Furthermore, there is no problem if the entire cropped area is included in the captured image, but as shown in FIG. It may protrude from the image.

In this case, as shown in FIG. 11(B), a background image having the same size as the cutout area is generated. This background image fills in the blank space that protrudes from the photographed image, and is filled in with a predetermined color (for example, black). Further, as shown in FIG. 11C, an image of a cutout area is cut out from the photographed image to obtain a cutout image. Then, as shown in FIG. 11(D), the cutout image is superimposed on the background image to generate a face image.

Next, facial image extraction processing will be explained. FIG. 12 is an explanatory diagram showing an overview of face image extraction processing.

As described above, the face image extraction unit 22 of the face authentication device 1 generates face image data from the captured image data of the user through face image extraction processing, that is, face detection, face size check, and face cropping processing. This face image extraction process is similarly performed by the face image extraction unit 83 of the face matching server 6.

At the time of user registration, by sending captured image data of the user from the management terminal 2 to the face matching server 6 via the face management server 5, the face image extraction unit 83 of the face matching server 6 performs face image extraction processing.

On the other hand, during face authentication, only the face authentication device 1 performs facial image extraction processing, and the face matching server 6 does not perform facial image extraction processing. In this case, the face authentication device 1 is provided with a highly accurate face detection function. In addition, the face authentication device 1 constantly photographs the photographing area with the camera 11, and sends face image information (photographed image data and face frame information) to the face matching server 6 at the timing when a face is detected. Thereby, the load of face image extraction processing is distributed to the plurality of face authentication devices 1, and the load on the face matching server 6 can be reduced. Furthermore, since the amount of communication can be reduced, the load on the network can be reduced. Since the face authentication response can be performed at high speed, the face authentication of the subjects who appear one after another can be efficiently performed.

In other words, since the configuration is such that the user's face recognition processing is shared between the face recognition device 1 and the face matching server 6, there is no need to concentrate all the user's face recognition processing on the face recognition device 1, which is expensive as in the past. There is no need to provide many authentication machines. Furthermore, even if facial feature amount data needs to be updated, large-scale maintenance work for the authentication machine is not required, and the work can be done by the face matching server 6. Therefore, according to this embodiment, it is possible to construct a face authentication system with an inexpensive configuration and excellent workability.

Note that the management terminal 2 may have a configuration in which it does not have a facial image extraction function, that is, the management terminal 2 may not be provided with the facial image extraction section 22.

FIG. 13 is an explanatory diagram showing an outline of a face authentication inquiry based on matching group information.

The face management server 5 holds information regarding the verification group to which the user belongs, association information between the face authentication device 1 and the verification group, and association information between the face verification server 6 and the verification group. In addition, the matching group management unit 63 of the face management server 5 uses the face authentication device 1 and the matching group to determine whether the face authentication device Generate matching group information for each one.

This verification group information is information necessary for the face authentication device 1 to request face verification from the face verification server 6 that matches its own verification group. This verification group information includes the identification information (group number) of the verification group to which the face recognition device 1 belongs, and the face verification server 6 that requests face recognition from the face recognition device 1, that is, the verification group with the face recognition device 1. and destination information of the face matching server 6 corresponding to the address information. Here, the destination information is specifically the network address (for example, IP address) of the face verification server 6, and this destination information specifies the face verification server 6 to which the face verification is requested, and 1 and the face matching server 6 that is the destination of the face authentication request. Note that when the face authentication device 1 belongs to a plurality of matching groups, the address (IP address) of each of the corresponding plurality of face matching servers 6 is included in the matching group information.

In the face authentication device 1, at the time of startup, etc., the startup processing unit 21 acquires verification group information from the face management server 5 as operation setting information and stores it in the device itself. When the face recognition device 1 detects a person's face, it sends a face verification request to the face verification server 6 that matches the verification group to which the device itself belongs. This face matching request includes information on the matching group to which the device itself belongs.

In addition, in the face authentication device 1, the timing for acquiring the setting information regarding the matching group information and the like from the face management server 5 may be other than the time of startup, and may also be performed at a predetermined timing or periodically at a predetermined interval. , it is also possible to adopt a configuration in which the matching group information is distributed from the face management server 5.

The face matching server 6 holds association information between the face matching process of its own device and matching groups. Upon receiving the face recognition request from the face recognition device 1, the face verification server 6 performs face recognition based on the association information between the verification group and the face recognition process and the verification group obtained from the face recognition device 1. A face recognition process corresponding to machine 1 is specified, and the face verification process is caused to perform face verification. As a result, face verification processing is performed using a face recognition process that corresponds to the designated verification group.

By the way, in this embodiment, a face matching database is provided for each matching group in the face matching server 6, and facial feature amount data is stored for each matching group. Therefore, at the time of face verification, face verification may be performed using face feature amount data registered in the face verification database corresponding to the verification group of the face authentication device 1. On the other hand, the face matching server 6 may perform matching regardless of the matching group of the face authentication device 1 and then apply a filter to the matching results. That is, after performing face matching on the facial feature data of all users stored in the face matching server 6, only the matching results with users belonging to the matching group of the face authentication device 1 are extracted. You can do it like this. In this case, the face matching database may not be provided separately for each matching group.

Next, administrator access management will be explained.

This system manages access from the management terminal 2 to the face management server 5 and face matching server 6. The face management server 5 includes an administrator access management section 61. This access management unit monitors the status of access (login) from the management terminal 2 to the face management server 5 and the face matching server 6, and when an access from the management terminal 2 is detected, information regarding the access (the accessing administrator (date and time, etc.) as administrator access logs (history information). In addition, the face management server 5 provides the administrator access log to the management terminal 2 in response to a request from the management terminal 2 to view the administrator access log, and the administrator can view the administrator access log. .

In this embodiment, access rights to user information are set to the administrator, and access to user information (registration, reference, update, deletion, etc. of user information) is restricted. Group administrators into multiple administrator groups depending on the access rights granted to them. Here, administrators are grouped into system operation managers, service operation managers, and general administrators.

Furthermore, in this embodiment, users are divided into a plurality of access permission groups depending on the administrator group that can access the user's information. The access permission group is an administrator group that can access the data of the user. Here, users are grouped into system operation managers, service operation managers, and general managers.

When the face management server 5 receives an administrator's login, it obtains the administrator group (administrator type) of the logged-in administrator, and displays the administrator group and the access permission group to which the target user belongs. Based on this, it is determined whether the user's information can be accessed (referenced, updated, and deleted), and access to the user's information is permitted. That is, it is possible to perform reference, update, and deletion processing only on information of users who are permitted by the administrator group to which the administrator belongs.

The system operations administrator is the person who manages the operation (maintenance, etc.) of this system, and has access to the information of all users, that is, users whose access permission group is system user, administrative user, or general user. can do. In addition, the service operation administrator is a person who manages the operation of the facial recognition service by this system (registration/deletion of users, etc.), and who accesses the information of users whose access permission group is administrative users or general users. be able to. Further, the general administrator is a person who has been given authority by the service operations manager, and can only access information of users whose access permission group is general users.

FIG. 14 is an explanatory diagram showing a login screen displayed on the management terminal 2.

On the management terminal 2, when the management application is started and the face management server 5 is accessed, a login screen is displayed. On this login screen, the administrator can enter his or her user ID and password. When the administrator enters the user ID and password on this login screen and operates the login button, user authentication is performed in the face management server 5. If the login is successful, the top screen will be displayed. If login fails, display an error.

Next, registration, reference, update, and deletion regarding verification groups will be explained. FIGS. 15 and 16 are explanatory diagrams showing screens displayed on the management terminal 2 when registering, referencing, updating, and deleting verification groups.

In the management terminal 2, when registering, referencing, updating, and deleting verification groups, the reference screen shown in FIG. 15(A), the registration screen shown in FIG. 15(B), the update screen shown in FIG. 15(C), and A deletion screen shown in FIG. 16 is displayed.

The reference screen shown in FIG. 15(A) displays a list of registered matching groups. By viewing this reference screen, the administrator can confirm the registered contents of the verification group.

This reference screen is provided with a list display section 101 and an update button 102. In the list display section 101, a matching group ID (number) and a description (installation location) regarding the matching group are displayed side by side for each registered matching group. This allows the administrator to confirm registered verification groups. On this reference screen, when the update button 102 is operated, the list display section 101 is updated to the latest registered contents. Further, by selecting a verification group, specifically, by operating the verification group ID, a transition is made to an update screen (see FIG. 15(C)).

Further, on this reference screen, it is possible to specify an item (matching group, description) and execute sorting. Note that the number of items to be displayed on the list display section 101 (the number of face matching servers 6) and the display range of the face matching servers 6 and matching groups may be specified. Alternatively, the search may be performed by specifying search conditions for each item (device ID, face matching server 6, description).

The registration screen (first screen) shown in FIG. 15(B) allows the administrator to register a verification group.

This registration screen is provided with an explanation input section 103 and a registration button 104. In the explanation input section 103, an explanation regarding the matching group can be input. When the registration button 104 is operated after inputting the description, the face management server 5 performs a process of registering matching group information in the database based on the input contents.

The update screen shown in FIG. 15(C) allows the administrator to update the registered contents of the verification group.

This update screen is provided with an explanation input section 105 and an update button 106. In the explanation input section 105, an explanation regarding the matching group can be input. When the user operates the update button 106 after inputting the description, the face management server 5 performs a process of updating the matching group information registered in the database with the input content.

The deletion screen shown in FIG. 16 allows the administrator to delete a verification group.

This deletion screen is provided with a list display section 107 and a delete button 108. The list display section 107 displays a list of registered verification groups (numbers) and explanations regarding the verification groups. In addition, this list display section 107 is provided with a check box for each matching group. Verification groups can be selected using checkboxes. When a verification group to be deleted is selected using the check box and the delete button 108 is operated, the face management server 5 performs a process of deleting the selected verification group.

Here, by selecting reference on the menu screen (not shown), a transition is made to the reference screen shown in FIG. 15(A). Furthermore, by selecting registration on the menu screen, the screen changes to the registration screen shown in FIG. 15(B). Furthermore, by selecting update on the menu screen, the screen changes to the update screen shown in FIG. 15(C). Furthermore, by selecting delete on the menu screen, the screen transitions to the deletion screen shown in FIG. 16.

Next, processing for registering, referencing, updating, and deleting matching groups will be explained. FIG. 17 is a sequence diagram showing the procedure of registration, reference, and update processing regarding verification groups. FIG. 18 is an explanatory diagram showing an overview of registration, reference, and update processing regarding verification groups.

First, registration processing regarding verification groups in this system will be explained.

As shown in FIG. 17(A), on the management terminal 2, the administrator performs an operation to instruct registration regarding a verification group. In this embodiment, the administrator performs an operation to register a verification group on the registration screen regarding verification groups (see FIG. 15(B)).

As shown in FIG. 18, in the face management server 5, when an operation instructing registration of a matching group is performed on the management terminal 2, the matching group management section 63 performs registration processing regarding the matching group. In this registration process, the verification group information input by the administrator is registered in the verification group database.

When the registration process regarding the matching group is completed, the face management server 5 displays a process result on the management terminal 2 indicating whether or not the registration process has been successfully completed. Further, in the face management server 5, an administrator access management section 61 records an administrator access log.

Next, reference processing regarding matching groups will be explained.

As shown in FIG. 17(B), on the management terminal 2, the administrator performs an operation to instruct reference to a collation group. In this embodiment, an operation is performed to open a reference screen (see FIG. 15(A)) regarding a collation group on a menu screen or the like.

In the face management server 5, when an operation instructing reference to a matching group is performed on the management terminal 2, reference processing regarding the matching group is performed. In this reference process, a database related to matching groups is referenced to generate matching group list information for all matching groups, and a reference screen for all matching groups is created based on the list information. Display it on the management terminal 2. In addition, when the administrator selects an item and instructs sorting on the reference screen displayed on the management terminal 2 (see FIG. 15(A)), list information in a sorted state is generated. A reference screen on which the sorting is reflected is displayed on the management terminal 2.

Next, update processing regarding verification groups will be explained.

As shown in FIG. 17(C), on the management terminal 2, the administrator performs an operation to instruct updating of the collation group. In this embodiment, the user inputs the update details and operates the update button 106 on the update screen regarding the matching group (see FIG. 15(C)).

In the face management server 5, when an operation instructing the updating of a matching group is performed on the management terminal 2, an updating process regarding the matching group is performed. In this update process, the database regarding collation groups is updated with the contents specified by the administrator. When the update process is completed, the face management server 5 displays a result on the management terminal 2 indicating whether or not the update process has been successfully completed.

Next, deletion processing regarding matching groups will be explained. FIG. 19 is a sequence diagram showing the procedure of deletion processing regarding a verification group. FIG. 20 is an explanatory diagram illustrating an overview of deletion processing regarding a verification group.

As shown in FIG. 19, on the management terminal 2, the administrator performs an operation to instruct deletion of a matching group. In this embodiment, on the deletion screen regarding verification groups (see FIG. 16), a verification group to be deleted is selected and the delete button 108 is operated.

As shown in FIG. 20, when an operation instructing deletion of a matching group is performed on the management terminal 2, the face management server 5 performs deletion processing regarding the matching group. In this deletion process, the registration information of the collation group specified by the administrator is deleted from the database regarding collation groups. Furthermore, the association information between the target matching group and the face matching server 6 is deleted. At this time, the association information between the target matching group and the face matching server 6 is deleted, and then the registration information of the matching group is deleted. Additionally, information regarding the target verification group is deleted from the database of associations between verification groups and users. Furthermore, the facial information (facial feature amount data) of the user belonging to the target matching group is deleted from the facial information database. Furthermore, the association information between the target matching group and the face authentication device 1 is deleted. Further, in the face management server 5, an administrator access management section 61 records an administrator access log.

Further, the face management server 5 sends a request to delete a matching group to the face matching server 6. This matching group deletion request includes information on the matching group to be deleted.

When the face matching server 6 receives a request to delete a matching group from the face management server 5, it performs deletion processing regarding the matching group. In this deletion process, association information between the specified matching group and the face matching section (face matching process) is deleted. At this time, the face matching unit for the matching group to be deleted is stopped, and the face matching database corresponding to the matching group is deleted.

Then, when the deletion process is completed, the face matching server 6 sends a matching group deletion response to the face management server 5. This response includes a processing result indicating whether or not the deletion processing was successfully completed.

When the face management server 5 receives the matching group deletion response from the face matching server 6 and completes the deletion process on its own device, it displays on the management terminal 2 a processing result indicating whether or not the deletion process was completed normally. do.

Note that by restarting the face authentication device 1, this processing is reflected and the information on the verification group is deleted.

Next, a description will be given of the batch registration process regarding verification groups. FIG. 21 is a sequence diagram showing the procedure of batch registration processing regarding verification groups.

In this system, matching groups can be collectively registered in the face management server 5 using a list information file that stores information regarding target matching groups.

Specifically, first, the management terminal 2 performs an operation to instruct batch registration regarding the verification group. In this embodiment, on a registration screen (not shown), the administrator selects a file in which list information regarding a target matching group is stored and operates a registration button.

At this time, the selected list information file is read and the list information is displayed on the confirmation screen of the management terminal 2. After confirming the registration details on the confirmation screen, the administrator performs an operation to instruct batch registration of matching groups. Furthermore, on the confirmation screen, the registered contents can be corrected if necessary.

In the face management server 5, when an operation instructing batch registration of matching groups is performed on the management terminal 2, the face management server 5 performs batch registration processing regarding matching groups. In this batch registration process, the registration process for each target matching group is repeated based on the matching group list information obtained from the management terminal 2.

When the batch registration process is completed, the face management server 5 displays on the management terminal 2 a registration result indicating whether the batch registration process has been successfully completed. At this time, the verification group whose registration has failed is reported to the administrator, and the administrator is urged to re-register the verification group. Output the registration results to a log file.

Note that although the bulk registration process regarding verification groups has been described here, bulk updating and bulk deletion processing regarding verification groups is also possible.

Next, registration, reference, update, and deletion regarding the association between the face authentication device 1 and the matching group in this system will be explained. 22 and 23 are explanatory diagrams showing screens displayed on the management terminal 2 when registering, referencing, updating, and deleting the association between the face authentication device 1 and the matching group.

The management terminal 2 displays the reference screen shown in FIG. 22(A), the registration screen shown in FIG. 22(B), and A deletion screen shown in FIG. 23 is displayed.

The reference screen shown in FIG. 22(A) displays a list of association settings between registered face authentication devices 1 and matching groups. By viewing this reference screen, the administrator can confirm the registered contents of the verification group.

This reference screen is provided with a list display section 111 and a face recognition device designation section 112. The list display section 111 displays the device ID of each authentication device, the verification group (number) associated with each authentication device, and the explanation regarding the verification group. In the face authentication device designation section 112, the target face authentication device 1 (device ID) can be selected from a pull-down menu. Thereby, the display contents of the list display section 111 can be narrowed down to the specific face authentication device 1, and only the matching groups associated with the specific face authentication device 1 are displayed on the list display section 111. Note that when the face authentication device 1 is not specified in the face authentication device designation section 112, information regarding all the face authentication devices 1 is displayed on the list display section 111.

Further, on this reference screen, it is possible to specify items (device ID, face recognition device 1, description) and perform sorting. Note that the number of items to be displayed on the list display section 111 (the number of face authentication devices 1) and the display range of the face verification servers 6 and verification groups may be specified. Alternatively, the search may be performed by specifying search conditions for each item (device ID, face authentication device 1, description).

Note that in the example shown in FIG. 22(A), the face authentication machines 1 are grouped by the location where the face authentication machines 1 are installed (for example, the location of a business office).

The registration screen (second screen) shown in FIG. 22(B) is for setting an association between the face authentication device 1 and a verification group. Information that associates a verification group with an authentication device can be registered.

This registration screen is provided with an authentication device designation section 113, a verification group designation section 114, and a registration button 115. In the authentication device designation unit 113, the target face authentication device 1 (device ID) can be selected from a pull-down menu. In the verification group designation section 114, the verification group of the target face authentication device 1 can be selected from a pull-down menu. This pull-down menu displays the matching groups registered on the matching group registration screen (see FIG. 15(B)). When the user operates the registration button 115 after inputting the face authentication device 1 and the matching group on this registration screen, the face management server 5 performs a process of registering matching group information based on the input contents.

Note that a plurality of matching groups may be associated with one face authentication device 1, but in this case, the registration operation of associating one matching group with one face authentication device 1 may be repeated on the registration screen. Further, a plurality of verification group designation sections 114 may be provided on the registration screen.

The deletion screen shown in FIG. 23 is for deleting the association settings between the face authentication device 1 and the matching group.

This deletion screen is provided with a list display section 116, a face recognition device designation section 117, and a deletion button 118. The list display section 116 displays the device ID of each authentication device, the verification group (number) associated with each face authentication device 1, and a description regarding the verification group. Further, the list display section 116 is provided with a check box for each association setting. Association settings can be selected using check boxes. When a registered item to be deleted is selected using the check box and the delete button 118 is operated, the face management server 5 performs a process of deleting the selected registered item. In the face recognition device designation section 117, the face recognition device 1 can be selected from a pull-down menu. Thereby, the display contents of the list display section 116 are updated with the list narrowed down to the specific face authentication device 1.

Here, by selecting reference on the menu screen (not shown), a transition is made to the reference screen shown in FIG. 22(A). Moreover, by selecting registration on the menu screen, the screen changes to the registration screen shown in FIG. 22(B). Furthermore, by selecting delete on the menu screen, the screen transitions to the deletion screen shown in FIG. 23. Furthermore, by selecting update on the menu screen, a transition is made to an update screen (not shown). Further, in the reference screen, when the administrator selects an association between the face authentication device 1 and the matching group, the screen changes to an update screen (not shown).

Note that in the example shown in FIG. 23, a list of registered associations is displayed on the deletion screen, but it may be possible to delete them on an individual editing screen. Further, the update screen (not shown) is similar to the registration screen (FIG. 22(B)).

Next, a registration process regarding association between the face authentication device 1 and the matching group will be described. FIG. 24 is a sequence diagram showing the procedure of a registration process regarding association between the face authentication device 1 and a matching group. FIG. 25 is an explanatory diagram showing an overview of the registration process regarding association between the face authentication device 1 and the matching group.

On the management terminal 2, the administrator performs an operation to open a registration screen (see figure). In this embodiment, an operation for selecting registration is performed on a menu screen (not shown).

As shown in FIG. 25, when the face management server 5 performs an operation to open the registration screen on the management terminal 2, the face management server 5 refers to the database related to the registered devices (the face recognition device 1 and the face matching server 6) and 1 list information is generated. Furthermore, the database regarding verification groups is referred to to generate list information of verification groups. Then, screen information of a registration screen is generated based on the list information of the face authentication device 1 and the list information of the matching group, and the registration screen is displayed on the management terminal 2.

Next, on the management terminal 2, the administrator performs an operation on the registration screen to instruct registration of an association between the face authentication device 1 and the verification group. In this embodiment, on the registration screen (see FIG. 22(B)), the combination of the face authentication device 1 and the matching group is input and the registration button is operated.

When an operation instructing registration of an association between the face authentication device 1 and a matching group is performed on the management terminal 2, the face management server 5 performs a registration process regarding the association between the face authentication device 1 and the matching group. . In this registration process, association information is registered in the database based on the combination of the face authentication device 1 and the matching group specified by the administrator. Note that the association between the face authentication device 1 and the matching group is reflected by restarting the face authentication device 1.

When the registration process is completed, the face management server 5 displays the registration result on the management terminal 2, indicating whether the registration process has been successfully completed.

Next, a reference process regarding the association between the face authentication device 1 and the matching group will be explained. FIG. 26 is a sequence diagram showing the procedure of reference processing regarding association between the face authentication device 1 and the matching group.

At the management terminal 2, the administrator performs an operation to instruct reference to the association between the face matching server 6 and the matching group. In this embodiment, an operation is performed to open a reference screen (see FIG. 22(A)) on a menu screen or the like.

In the face management server 5, when an operation is performed on the management terminal 2 to instruct reference to the association between the face authentication device 1 and the matching group, the face management server 5 performs reference processing regarding the matching group. In this reference process, a database regarding the association between the face authentication device 1 and the matching group is referred to, a list of association information for all the face matching servers 6 is generated, and the reference screen is displayed on the management terminal 2. Display. In addition, when the administrator performs an operation to narrow down target face authentication devices 1 on the reference screen, a list of associations targeting the selected face authentication devices 1 is generated, and the reference screen with the narrowing reflected is displayed. Display it on the management terminal 2. Further, when the administrator selects an item on the reference screen and performs an operation to instruct sorting, list information in a sorted state is generated and a reference screen on which the sorting is reflected is displayed on the management terminal 2.

The face management server 5 also records an administrator access log.

Next, an update process regarding the association between the face authentication device 1 and the matching group will be described. FIG. 27 is a sequence diagram illustrating the procedure of update processing regarding the association between the face authentication device 1 and the matching group.

At the management terminal 2, the administrator performs an operation to open an update screen (not shown). In this embodiment, an operation for selecting update is performed on a menu screen (not shown).

In the face management server 5, when an operation to open an update screen is performed on the management terminal 2, list information of the face authentication device 1 is generated by referring to a database related to registered devices. Furthermore, the database regarding matching groups is referred to to generate matching group list information. Then, screen information of a registration screen is generated based on the list information of the face authentication device 1 and the list information of the matching group, and an update screen is displayed on the management terminal 2. In this update screen, currently registered combinations are displayed in the initial state based on the registered information regarding the association between the face authentication device 1 and the matching group.

Next, on the management terminal 2, the administrator performs an operation on the update screen to instruct update of the association between the face authentication device 1 and the matching group. In this embodiment, the user inputs update details and operates an update button on an update screen (not shown). Note that the update content is to change the combination of the authentication device and the verification group.

In the face management server 5, when an operation instructing to update the association between the face authentication device 1 and the matching group is performed on the management terminal 2, the face management server 5 performs an update process regarding the association between the face authentication device 1 and the matching group. . In this update process, the database regarding the association between the face authentication device 1 and the matching group is updated with the content specified by the administrator. Performs processing to update the association information in the database based on the combination of authentication device and verification group input by the administrator. When the update process is completed, the face management server 5 displays a result on the management terminal 2 indicating whether or not the update process has been successfully completed.

Next, a deletion process regarding the association between the face authentication device 1 and the matching group will be explained. FIG. 28 is a sequence diagram illustrating the procedure of deletion processing regarding the association between the face authentication device 1 and the matching group.

At the management terminal 2, the administrator performs an operation to instruct deletion of the association between the authentication device and the verification group. In this embodiment, on the deletion screen (see FIG. 23), a registered item (association) is selected and the deletion button is operated.

When an operation instructing deletion of the association between the face authentication device 1 and the matching group is performed on the management terminal 2, the face management server 5 performs deletion processing regarding the association between the face authentication device 1 and the matching group. . In this deletion process, association information between the face authentication device 1 and the matching group specified by the administrator is deleted.

When the deletion process is completed, the face management server 5 displays a result on the management terminal 2 indicating whether or not the deletion process has been successfully completed. The face management server 5 also records an administrator access log.

Next, registration, reference, and deletion regarding the association between the face matching server 6 and the matching group in this system will be explained. 29 and 30 are explanatory diagrams showing screens displayed on the management terminal 2 when registering, referencing, and deleting associations between the face matching server 6 and matching groups.

The management terminal 2 displays the reference screen shown in FIG. 29(A), the registration screen shown in FIG. 29(B), and A deletion screen shown in FIG. 30 is displayed.

The reference screen shown in FIG. 29(A) displays a list of association settings between registered face matching servers 6 and matching groups. By viewing this reference screen, the administrator can confirm the registered contents of the verification group.

This reference screen is provided with a list display section 121 and a face matching server designation section 122. The list display section 121 displays the device ID of each face verification server 6, the verification group (number) associated with each authentication device, and the description regarding the verification group. In the face matching server designation section 122, a target face matching server 6 (device ID) can be selected from a pull-down menu. Thereby, the display contents of the list display section 121 can be narrowed down to the specific face matching server 6, and only matching groups associated with the specific face matching server 6 are displayed on the list display section 121. Note that if the face matching server designation section 122 does not specify the face matching server 6, the list display section 121 displays information regarding all the face matching servers 6.

Further, on this reference screen, it is possible to specify items (device ID, face matching server 6, description) and execute sorting. Note that the number of items to be displayed on the list display section 121 (the number of face matching servers 6) and the display range of the face matching servers 6 and matching groups may be specified. Alternatively, the search may be performed by specifying search conditions for each item (device ID, face matching server 6, description).

The registration screen (second screen) shown in FIG. 29(B) is for specifying the association between the matching group and the face matching server 6.

This registration screen is provided with a face verification server designation section 123, a verification group designation section 124, and a registration button 125. In the face matching server designation section 122 , the device ID of the face matching server 6 can be selected from a pull-down menu. In the verification group designation section 124, a verification group can be selected from a pull-down menu. The pull-down menu displays the matching groups registered on the matching group registration screen (see FIG. 15(B)).

When the face matching server 6 and matching group are specified and the registration button 125 is operated, association information between the matching group and the face matching server 6 is registered based on the combination of the specified face matching server 6 and matching group.

Note that a plurality of matching groups may be assigned to one face matching server 6, and in this case, the registration operation of associating one matching group with one face matching server 6 on the registration screen may be repeated. Further, a plurality of verification group designation sections 124 may be provided on the registration screen.

The deletion screen shown in FIG. 30 is for deleting the association settings between the face matching server 6 and matching groups.

This deletion screen is provided with a list display section 126, a face matching server designation section 127, and a deletion button 128. The list display section 126 displays the device ID of each face matching server 6, the matching group (number) associated with each face matching server 6, and a description regarding the matching group. Further, this list display section 126 is provided with a check box for each association setting. Association settings can be selected using check boxes. When a registered item to be deleted is selected using the check box and the delete button 128 is operated, the face management server 5 performs a process of deleting the selected registered item. In the face matching server designation section 127, the face matching server 6 can be selected from a pull-down menu. Thereby, the display contents of the list display section 126 are updated with the search narrowed down to a specific face matching server 6.

Here, the reference screen shown in FIG. 29(A) is displayed by selecting reference on the menu screen (not shown). The registration screen shown in FIG. 29(B) is displayed by selecting registration on the menu screen (not shown). The deletion screen shown in FIG. 30 is displayed by selecting delete on the menu screen (not shown). Furthermore, when the face matching server 6 is selected in the list display section 121 of the reference screen shown in FIG. 29(A), the screen changes to the registration screen shown in FIG. Associations can be changed.

In the example shown in FIG. 30, a list of registered associations is displayed on the deletion screen, but deletion may be performed on an individual editing screen (not shown).

Next, registration processing regarding association between the face matching server 6 and matching groups will be described. FIG. 31 is a sequence diagram showing the procedure of registration processing regarding association between the face matching server 6 and matching groups.

On the management terminal 2, the administrator performs an operation to open the registration screen (see FIG. 29(B)). In this embodiment, an operation for selecting registration is performed on a menu screen (not shown).

In the face management server 5, when an operation to open a registration screen is performed on the management terminal 2, list information of the face matching server 6 is generated by referring to a database related to registered devices. Furthermore, the database regarding verification groups is referred to to generate list information of verification groups. Then, screen information of a registration screen is generated based on the list information of the face matching server 6 and the list information of the matching group, and the registration screen is displayed on the management terminal 2.

Next, on the management terminal 2, the administrator performs an operation on the registration screen to instruct registration of an association between the face matching server 6 and the matching group. In this embodiment, on the registration screen (see FIG. 29(B)), the combination of the face verification server 6 and verification group is input and the registration button 125 is operated.

When an operation instructing registration of an association between the face matching server 6 and a matching group is performed on the management terminal 2, the face management server 5 performs a registration process regarding the association between the face matching server 6 and the matching group. . In this registration process, association information is registered in the database based on the combination of the face matching server 6 and the matching group specified by the administrator.

Further, the face management server 5 sends a request for registration of a matching group to the target face matching server 6.

When the face matching server 6 receives a request for matching group registration from the face management server 5, it performs registration processing regarding the matching group. In this registration process, the verification group is registered as setting information of the own device. When the registration process is completed, the face matching server 6 sends a registration response to the face management server 5.

When the registration process is completed, the face management server 5 displays a result on the management terminal 2 indicating whether or not the registration process has been successfully completed. The face management server 5 also records an administrator access log.

Next, reference processing regarding association between the face matching server 6 and matching groups will be explained. FIG. 32 is a sequence diagram showing the procedure of reference processing regarding association between the face matching server 6 and matching groups.

At the management terminal 2, the administrator performs an operation to instruct reference to the association between the face matching server 6 and the matching group. In this embodiment, an operation is performed to open a reference screen (see FIG. 29(A)) on a menu screen or the like.

In the face management server 5, when an operation instructing to refer to the association between the face matching server 6 and the matching group is performed on the management terminal 2, the face management server 5 performs reference processing regarding the association between the face matching server 6 and the matching group. . In this reference process, a database regarding associations between face matching servers 6 and matching groups is referenced, a list of association information for all face matching servers 6 is generated, and a reference screen is displayed on the management terminal 2. Display. Additionally, when the administrator performs an operation to narrow down target face matching servers 6 on the reference screen, a list of associations targeting the selected face matching servers 6 is generated, and the reference screen with the narrowing reflected is displayed. Display it on the management terminal 2. Further, when the administrator selects an item on the reference screen and performs an operation to perform sorting, a sorted list is generated and the reference screen on which the sorting is reflected is displayed on the management terminal 2.

The face management server 5 also records an administrator access log.

Next, deletion processing regarding the association between the face matching server 6 and the matching group will be explained. FIG. 33 is a sequence diagram illustrating the procedure of deletion processing regarding the association between the face matching server 6 and the matching group. FIG. 34 is an explanatory diagram showing an overview of deletion processing regarding association between the face matching server 6 and matching groups.

At the management terminal 2, the administrator performs an operation to instruct deletion of the association between the face matching server 6 and the matching group. In this embodiment, on the deletion screen (see FIG. 30), a target association is selected and the deletion button is operated.

As shown in FIG. 34, in the face management server 5, when an operation instructing deletion of the association between the face matching server 6 and the matching group is performed on the management terminal 2, the Perform deletion processing related to the association. In this deletion process, the association information between the target face matching server 6 and the matching group is deleted. Furthermore, the face registration ID associated with the target face matching server 6 is deleted from the face information database.

Further, the face management server 5 sends a request to delete a matching group to the target face matching server 6.

When the face matching server 6 receives a request to delete a matching group from the face management server 5, it performs deletion processing regarding the matching group. In this deletion process, setting information regarding the verification group of the own device is deleted. When the deletion process is completed, the face matching server 6 sends a response to the face management server 5.

When the deletion process is completed, the face management server 5 displays a result on the management terminal 2 indicating whether or not the deletion process has been successfully completed. The face management server 5 also records an administrator access log.

Next, registration, reference, update, and deletion regarding users will be explained. 35 and 36 are explanatory diagrams showing screens displayed on the management terminal 2 when registering, referencing, updating, and deleting users.

On the management terminal 2, when registering, referencing, updating, and deleting users, the individual editing screen shown in FIG. 35(A), the search screen shown in FIG. 35(B), and the list display screen shown in FIG. 36 are displayed. Ru.

The individual editing screens (second screen, third screen) shown in FIG. 35(A) are used to perform user-related registration, reference, update, and deletion operations.

This registration screen includes a name input section 131, a user code input section 132, a verification group input section 133, an access permission group input section 134, a validation date input section 135, a deactivation date input section 136, and an additional information input section. 137, a face image input section 138, and an execution button 139 are provided.

In the name input section 131, the user's name can be input. In the user code input section 132, a user code and a code number (for example, an employee number) can be input. In the verification group input section 133, a verification group can be input. The matching group input section 133 displays matching groups registered on the matching group registration screen (see FIG. 15(B)). Verification groups can be selected using checkboxes. In the access permission group input section 134, an access permission group can be input. Access permission groups can be selected from the pull-down menu. In the validation date input section 135, it is possible to input the validation date, that is, the date on which the face matching of the person concerned can be validated. In the invalidation date input section 136, it is possible to input the invalidation date, that is, the date on which face verification can be invalidated. By setting the activation date and invalidation date, services for multiple users can be started or terminated all at once. In the additional information input section 137, additional information can be input.

The face image input section 138 is provided with a face image designation section 141, a selection button 142, and a face image display section 143. This facial image designation section 141 allows designation of a file of a photographed image of the user. When a user's photographed image file stored in advance on the management terminal 2 is specified and the selection button 142 is operated, the user's photographed image is sent to the face matching server 6 via the face management server 5, and the user's photographed image is sent to the face matching server 6 via the face management server 5. Processing (face detection, face cutting, etc.) to generate a face image from the photographed image is performed by the face matching server 6, and the user's face image is displayed on the face image display section 143.

When information on each item is input on this registration screen and the execution button 139 is operated, a process for registering the user's information is performed based on the input contents.

In this way, on the registration screen, it is possible to allocate a verification group to a user, and thereby the association information between the user and the verification group is registered in the database. Also, on this registration screen, the activation date and the invalidation date can be registered. The activation date and the invalidation date define the period during which the user's facial information (facial feature data) is stored in the face matching server 6, and on the activation date, the user's facial image The data is registered in the database of the face matching server 6, and when the invalidation date comes, the user's facial image data is deleted from the database of the face matching server 6.

Further, on this registration screen, by specifying a photographed image of the user, the user's facial image data is registered in the database of the face matching server 6.

The search screen shown in FIG. 35(B) is for inputting search conditions regarding registered user information. Users who meet the search conditions can be narrowed down and displayed on the list display screen. This search screen includes a first and last name input field 131, a user code input field 132, a collation group input field 133, an access permission group input field 134, a validation date input field 135, a deactivation date input field 136, and an update date and time input field. 144, an acquisition number input section 145, and a search button 146 are provided.

The name input section 131, user code input section 132, verification group input section 133, access permission group input section 134, activation date input section 135, and invalidation date input section 136 are the same as those on the registration screen. In the update date and time input section 144, an update date and time (period) can be specified. In the acquisition number input section 145, the number of acquisition items can be specified.

On this search screen, when search conditions are entered in necessary items and the search button 146 is operated, a transition is made to a list display screen (see FIG. 36) in which the search results are reflected.

The list display screen shown in FIG. 36 is used to display a list of registered users, to update them all at once, and to delete them all at once.

By viewing this reference screen, the administrator can confirm the user's registered details. This list display screen is provided with a list display section 147 and an execution button 148. The list display section 147 displays user codes, first and last names, collation groups, and the like. In addition, the list display section 147 is provided with a check box for each user. Users can be selected using check boxes. When a user to be deleted is selected using the check box and the execution button 148 (delete button) is operated, the face management server 5 performs a process of deleting the selected user.

In addition, on the list display screen, it is possible to specify items (user code, first and last name, collation group, etc.) and perform sorting. Note that the number of items to be displayed in the list display section 147 (the number of users) and the display range of user codes and verification groups may be specified. Alternatively, the search may be performed by specifying search conditions for each item.

Here, the individual editing screen (see FIG. 35(A)) is used to perform user-related registration, reference, update, and deletion operations. Specifically, when registration is selected on a menu screen (not shown), an uninputted individual editing screen is displayed, and by entering information regarding the user here, the user can be newly registered. In addition, when you select a user on the list display screen (see Figure 36), you will be redirected to the individual editing screen, where you can check information about registered users individually (individual reference). Furthermore, the registered contents of required items can be re-inputted (individual update), and the registered contents of required items can be deleted (individual deletion).

Furthermore, when performing individual updates, if a captured image of the user is separately specified in the face image specification section 141 of the individual editing screen (see FIG. 35(A)), a new facial image of the user is generated from that captured image. The user's face image at the time of registration can be replaced with a new face image.

Further, the list display screen (see FIG. 36) is used to collectively refer to, collectively update, and collectively delete registered users. Specifically, if you select list display on the menu screen (not shown), a list display screen (see Figure 36) will be displayed, where you can check all the information about registered users (all at once). reference). Further, by selecting a check box, desired users can be deleted all at once (batch deletion). Further, a check box may be provided separately to enable selection of information on a page-by-page basis of the list.

Additionally, information for multiple users can be updated at once (batch update). In this case, on the search screen (see FIG. 35(B)), specify conditions such as target users and matching groups and operate the search button 146, and a list display screen (see FIG. 36(B)) is displayed accordingly. ), operate the execution button 148 (batch update button). Note that in the batch update, items other than user-specific information such as the user's name and user code are subject to update.

In addition, if you select search on the menu screen (not shown), a search screen (see FIG. 35(B)) will be displayed, and if you enter search conditions on this search screen and operate the search button 146, the search results will be reflected. The screen changes to a list display screen (see FIG. 36).

Next, registration processing regarding users will be explained. FIG. 37 is a sequence diagram showing the procedure of registration processing regarding a user. FIG. 38 is an explanatory diagram showing an overview of user registration processing.

At the management terminal 2, the administrator performs an operation to instruct registration regarding the user. In this embodiment, on the individual editing screen in the registration mode (see FIG. 35(A)), the administrator inputs the user's information and specifies the file of the user's captured image, and then clicks the registration button. operate.

As shown in FIG. 38, in the face management server 5, when an operation instructing registration of a user is performed on the management terminal 2, a registration process regarding the user is started. In this registration process, first, a request for facial image extraction is sent to the face matching server 6 whose matching group matches the target user. This request includes the photographed image of the user obtained from the management terminal 2. At this time, if there are multiple face matching servers 6 whose matching group matches the target user, one face matching server 6 is selected and a request is made to that one face matching server 6 to extract a face image. send.

Upon receiving a request for facial image extraction from the face management server 5, the face matching server 6 performs facial image extraction processing. In this face image extraction process, processes such as face detection and face cutting are performed on the photographed image of the user obtained from the face management server 5 to extract the user's face image. Then, a response for facial image extraction is sent to the face management server 5. This response includes the user's face image.

When the face management server 5 receives a response for face image extraction from the face matching server 6, it sends a face registration request to the face matching server 6 whose matching group matches the target user. This request can include the user's face image acquired from the specific face matching server 6. At this time, if there are multiple face matching servers 6 whose matching group matches the target user, all face matching servers 6 including the user's face image acquired from a specific face matching server 6 Send a request for face registration to .

Upon receiving the face registration request from the face management server 5, the face matching server 6 performs face registration processing. In this face registration process, facial features are generated from the previously extracted facial image of the user, and the facial features are registered in the database. At this time, a face registration ID is assigned in association with the user's facial feature amount. Then, the face registration response is sent to the face management server 5. This response includes a result indicating whether or not face registration has been successfully completed and a face registration ID. Note that upon completion of registration of facial features in the database, the extracted facial image of the user is deleted. Further, the data of the facial feature amount may be backed up and stored in a nonvolatile memory such as an HDD or SSD that is managed separately from the face matching DB of the face matching server 6.

The face management server 5 receives the face registration response from the face matching server 6, and performs registration processing regarding the user if the processing has been completed normally. In this registration process, user information acquired from the management terminal 2 is registered in the face information database. Furthermore, the face management server 5 registers the user's face image acquired from the face matching server 6 in the face information database. Further, the face management server 5 registers the face registration ID issued by the face matching server 6 in the face information database as user information.

Next, the face management server 5 displays on the management terminal 2 a result indicating whether or not the update regarding the user has been successfully completed. In addition, the face management server 5 records a log in an administrator access log.

Next, the batch registration process regarding users will be explained. FIG. 39 is a sequence diagram showing the procedure of batch registration processing for users.

In this embodiment, user information can be registered all at once using a file in which list information regarding a plurality of target users is stored.

Specifically, first, on the management terminal 2, the administrator performs an operation to instruct batch registration of users. In this embodiment, on a registration screen (not shown), the administrator selects a file in which list information regarding target users is stored and operates a registration button.

At this time, the selected list information file is read and the list information is displayed on the confirmation screen of the management terminal 2. After confirming the registration details on the confirmation screen, the administrator performs an operation to instruct bulk registration of users. Furthermore, on the confirmation screen, the registered contents can be corrected if necessary.

In the face management server 5, when an operation instructing bulk registration of users is performed on the management terminal 2, bulk registration processing for the users is started. In this batch registration process, the registration process is repeated for every target user. The registration process for each user is similar to the registration process (individual registration process) for the user shown in FIG. 37.

Then, the face management server 5 receives face registration responses for all target users from the face matching server 6, and when the batch registration process is completed, a registration result indicating whether the batch registration process has been completed normally or not. is displayed on the management terminal 2. At this time, the user who failed to register is reported to the administrator, and the user is urged to re-register. It also outputs the registration results in a log file.

Next, reference processing regarding users will be explained. FIG. 40 is a sequence diagram showing the procedure of reference processing regarding a user.

At the management terminal 2, the administrator performs an operation to instruct display of a list of users. In this embodiment, an operation is performed to open a list display screen (see FIG. 36) regarding collation groups on a menu screen or the like.

When the face management server 5 performs an operation to instruct display of a list of users on the management terminal 2, it refers to the database, generates list information including all users, and based on the list information, A list display screen (see FIG. 36) is displayed on the management terminal 2.

Furthermore, on the management terminal 2, the administrator performs an operation to instruct a search regarding users. Specifically, the administrator displays a search screen (see FIG. 35(B)), inputs search conditions, and operates a search button on the search screen.

In the face management server 5, when an operation instructing a search regarding users is performed on the management terminal 2, the face management server 5 refers to the database, generates list information for users who meet the search conditions, and stores the list information. Based on this, a list display screen (see FIG. 36) as a search result is displayed on the management terminal 2.

Further, on the management terminal 2, the administrator performs an operation to instruct sorting regarding display of a list of users. Specifically, the administrator specifies an item and instructs sorting on the list display screen.

When an operation instructing sorting is performed on the management terminal 2, the face management server 5 generates user list information sorted by the specified item, and the sorting is reflected based on the user list information. A list display screen is displayed on the management terminal 2.

Note that when the administrator selects a user on the list display screen (see FIG. 36) and performs an operation to instruct individual display, the face management server 5 refers to the database and displays the information of the selected user. is acquired, and an individual editing screen (see FIG. 35(A)) is displayed on the management terminal 2 based on the user's information.

Next, update processing regarding users will be explained. FIG. 41 is a sequence diagram showing the procedure of update processing regarding a user. FIG. 42 is an explanatory diagram showing an overview of update processing regarding users.

First, a case where there is a change in the captured image of the user will be described. In this case, the administrator performs an operation on the management terminal 2 to instruct updates regarding the user. In this embodiment, the administrator re-enters the user's information on the individual editing screen (not shown) in the update mode in which the user to be updated is specified and displayed. After replacing the captured image file, operate the update button.

As shown in FIG. 42, in the face management server 5, when an operation instructing an update regarding the user is performed on the management terminal 2, the face management server 5 starts update processing regarding the user. In this update process, first, a request for facial image extraction is sent to the face matching server 6 whose matching group matches the target user. This request includes the photographed image of the user obtained from the management terminal 2. At this time, if there are multiple face matching servers 6 whose matching group matches the target user, one face matching server 6 is selected and a request is made to that one face matching server 6 to extract a face image. send.

When the face matching server 6 receives a request for face image extraction from the face management server 5, the face image extraction unit 83 performs face image extraction processing. In this face image extraction process, processes such as face detection and face cutting are performed on the photographed image of the user obtained from the face management server 5 to generate a face image of the user. Then, a response for facial image extraction is sent to the face management server 5. This response includes the user's face image.

When the face management server 5 receives a response for face image extraction from the face matching server 6, it sends a face registration request to the face matching server 6 whose matching group matches the target user. This request can include the user's face image acquired from the specific face matching server 6. At this time, if there are multiple face matching servers 6 whose matching group matches the target user, all face matching servers 6 including the user's face image acquired from a specific face matching server 6 Send a request for face registration to .

Upon receiving the face registration request from the face management server 5, the face matching server 6 performs face registration processing. In this face registration process, facial features are generated from the previously extracted facial image of the user, and the facial features are registered in the database. At this time, a face registration ID is assigned in association with the user's facial feature amount. Then, the face registration response is sent to the face management server 5. This response includes a result indicating whether or not face registration has been successfully completed and a face registration ID.

When the face management server 5 receives the face registration response from the face matching server 6, if the face registration is successful, it sends a request to delete the facial feature amount to the same face matching server 6 that made the face registration request. Note that if face registration fails, a request to delete the facial feature amount is not made, and face matching is performed using the old facial feature amount.

When the face matching server 6 receives a request for facial feature deletion from the face management server 5, it performs facial feature deletion processing. In this facial feature deletion process, old facial features of the subject registered in the database are deleted. Then, a response for facial feature amount deletion is sent to the face management server 5. This response includes a result indicating whether or not facial feature quantity deletion has been completed normally.

The face management server 5 receives the response to delete the facial feature amount from the face matching server 6, and if the processing has been completed normally, performs update processing regarding the user. In this update process, the registration contents of the face information database are updated with the user information obtained from the management terminal 2. Further, the user's face image acquired from the face matching server 6 is registered in the face information database. Furthermore, the registered contents of the face information database are updated with the new face registration ID issued by the face matching server 6. That is, the old face registration ID is deleted and a new face registration ID is registered in its place.

Next, the face management server 5 displays on the management terminal 2 a processing result indicating whether or not the update regarding the user has been successfully completed. The face management server 5 also records an administrator access log.

Next, a case where there is no change in the user's photographed image, that is, a case where there is a change in items other than the photographed image will be described. In this case, on the management terminal 2, the administrator performs an operation to instruct an update regarding the user, but at this time, the administrator inputs the user's information on the individual edit screen (not shown) in the update mode. After doing so, operate the update button.

In the face management server 5, when an operation instructing an update regarding a user is performed on the management terminal 2, an update process regarding the user is performed. In this update process, the registered contents of the face information database are updated with the user information input by the administrator. Then, a result indicating whether or not the update regarding the user has been successfully completed is displayed on the screen of the management terminal 2. The face management server 5 also records an administrator access log.

Next, deletion processing regarding users will be explained. FIG. 43 is a sequence diagram showing the procedure of deletion processing regarding a user. FIG. 44 is an explanatory diagram showing an overview of deletion processing regarding a user.

At the management terminal 2, the administrator performs an operation to instruct deletion regarding a user. In this embodiment, the administrator operates a delete button on an individual editing screen (not shown) in a delete mode that specifies and displays a user to be deleted. Further, on the deletion mode list display screen (not shown), the administrator selects target users and operates a batch deletion button.

As shown in FIG. 44, the face management server 5 starts deletion processing regarding the user when an operation instructing deletion regarding the user is performed on the management terminal 2. In this deletion process, first, a face deletion request is sent to the face matching server 6 whose matching group matches the target user. At this time, if there are multiple face matching servers 6 whose matching groups match the user, a face deletion request is sent to all the face matching servers 6.

Upon receiving the face deletion request from the face management server 5, the face matching server 6 performs deletion processing regarding the target user's facial information. In this deletion process, the information (facial feature amount data) of the corresponding user is deleted from the database.

Furthermore, when the deletion process is completed, the face matching server 6 sends a face deletion response to the face management server 5. This response includes a result indicating whether or not the deletion process was successfully completed.

Further, the face management server 5 receives a face deletion response from the face matching server 6, and if the processing has been completed normally, performs deletion processing regarding the user. In this deletion process, the information of the target user is deleted from the face information database. When the deletion process is completed, a processing result indicating whether or not the deletion process has been successfully completed is displayed on the management terminal 2. The face management server 5 also records an administrator access log.

Next, the facial image extraction process performed by the face matching server 6 will be explained. FIG. 45 is an explanatory diagram showing an overview of face image extraction processing.

The face management server 5 sends a request for face image extraction to the face matching server 6. This request includes the user's captured image data and face extraction conditions (face image extraction conditions). Note that the face cutout conditions are parameters used in face image extraction processing (face detection, face size check, face cutout). Specifically, the number of faces to be adopted when multiple faces are detected (the number of faces detected), the face selection method that determines whether the area or height of the face frame is used as a criterion when detecting a face, and the number of faces to be adopted as a face. These include the minimum and maximum values (face size check threshold) regarding the width and height of the face frame, which serve as the standards to be adopted.

When the face matching server 6 receives a face image extraction request from the face management server 5, the face image extraction unit 83 performs face image extraction processing. In this face image extraction process, face detection, face size check, and face cutting are performed. Then, a response for facial image extraction is sent to the face management server 5. This response includes face image data (in some cases, there may be more than one face image) and position information of the face frame (coordinates of the upper left vertex of the face frame, width and height of the face frame).

The processing performed by the face image extraction unit 83 of the face matching server 6 is similar to that of the face image extraction unit 22 of the face authentication device 1. That is, first, a face area included in a captured image acquired from the face authentication device 1 is detected (face detection). Here, if the captured image includes a plurality of face areas, the face areas are selected based on the area and width of the face areas. Next, it is determined whether the size of the face area (face frame) detected from the photographed image satisfies a predetermined standard (appropriate range) (face size check). Next, for the captured image of the camera 11, a cutout area is set based on the face area (face frame) obtained by face detection, and the image of the cutout area is cut out as a face image (face cutout). Obtain a face image.

Next, the image quality check performed by the face matching server 6 will be explained. FIG. 46 is an explanatory diagram showing an overview of image quality check processing.

The image quality check unit 82 of the face matching server 6 determines whether the image of the face area (face frame) detected from the captured image data satisfies a predetermined quality. Specifically, it is detected from the image whether a mask is being worn or not and whether sunglasses are being worn. It also calculates face recognition suitability from the image. The face recognition suitability is an evaluation value according to the orientation and expression of the face.

When performing this image quality check, the face management server 5 first sends an image quality check request to the face matching server 6. This request includes photographed image data and face frame information.

When the face matching server 6 receives an image quality check request from the face management server 5, the image quality checking unit 82 checks whether the image of the face area (face frame) in the captured image satisfies a predetermined quality. is determined and a quality evaluation value is calculated. Then, the image quality check response is sent to the face management server 5. This response includes a quality evaluation value.

Next, facial feature value generation processing performed during face registration will be described. FIG. 47 is an explanatory diagram showing an overview of facial feature amount generation processing.

The facial feature amount generation unit 84 of the face matching server 6 generates facial feature amount data from the previously extracted user's facial image data, and stores the facial feature amount data in the face matching DB.

When generating this facial feature amount, first, the face management server 5 or the face authentication device 1 sends a request for facial feature amount generation to the face matching server 6.

When the face matching server 6 receives a request for facial feature generation from the face management server 5, the facial feature generation unit 84 generates facial feature data from the image of the previously extracted facial area. Then, a response for generating facial feature amounts is transmitted to the face management server 5.

Next, facial feature amount registration processing performed during face registration will be described. FIG. 48 is an explanatory diagram showing an overview of facial feature amount registration processing (feature amount specification). FIG. 49 is an explanatory diagram showing an overview of facial feature amount registration processing (image designation).

The face matching server 6 has a facial image extraction function and a facial feature amount generation function, and performs each process of facial image extraction and facial feature amount generation in response to a request from the face management server 5. On the other hand, if a plurality of face matching servers 6 exist in the same matching group, it is necessary to register the user's facial features redundantly in the face matching database of each face matching server 6.

At this time, one face matching server 6 performs each process of facial image extraction and facial feature amount generation, and the facial feature amounts obtained here are provided to the remaining face matching servers 6, so that all face matching can be performed. Registration of facial features becomes possible on the server 6. In this case, the face management server 5 receives the facial feature data generated by one face matching server 6 in response to the facial feature generation, and registers the facial feature data to the remaining face matching servers 6. Add facial feature data to the request and instruct facial feature registration (feature specification).

Also, one face matching server 6 performs face image extraction processing, and the facial image data obtained here is provided to the remaining face matching servers 6, and the remaining face matching servers 6 generate facial features. By performing this process, facial feature amount data can be registered in all face matching servers 6. In this case, the face management server 5 receives face image data etc. acquired from a certain face matching server 6 in response to face image extraction, and sends a request to the remaining face matching servers 6 for facial feature amount registration. Instructs to register facial features by adding facial image data to the image (image specification).

As shown in FIG. 48, in the case of feature specification, the face management server 5 first sends a request for facial feature registration to the face matching server 6. This request includes the user's matching group, facial feature data, and user code.

When the face matching server 6 receives a request for facial feature registration from the face management server 5 , the facial feature management unit 85 registers the user's facial feature data added to the request in the face matching database. At this time, the user is registered in the face matching database of the face matching unit corresponding to the matching group to which the user belongs. Further, the face verification server 6 gives a registrant code to the user whose face registration has been completed. The registrant code is registered in the face matching database in association with the facial feature amount.

Then, the face matching server 6 sends a response to the face management server 5 when the facial feature amount registration process is completed. This response includes a processing result indicating whether or not the processing has ended normally, and a face registration ID (registrant code).

As shown in FIG. 49, in the case of image specification, the face management server 5 first sends a request for facial feature amount registration to the face matching server 6. This request includes the user's matching group, the user's captured image data, face frame information, and user code.

When the face matching server 6 receives a request for facial feature registration from the face management server 5 , the facial feature generating unit 84 generates a request from the user's facial image data (photographed image data and face frame information) added to the request. Facial feature data is generated, and the facial feature management unit 85 registers the user's facial feature data in the face matching database. At this time, the user is registered in the face matching database of the face authentication process corresponding to the matching group to which the user belongs. Further, the face verification server 6 gives a registrant code to the user whose face registration has been completed. Further, the registrant code is registered in the face matching database in association with the facial feature amount.

Then, the face verification server 6 transmits a response for facial feature amount registration to the face management server 5. This response includes a processing result indicating whether or not the processing has ended normally, and a face registration ID.

Next, a description will be given of facial feature amount deletion processing performed during update and deletion. FIG. 50 is an explanatory diagram showing an overview of facial feature amount deletion processing.

When deleting a user, the facial feature amount data of the target user is deleted from the face matching database of the face matching server 6. Furthermore, when updating a user, the old facial feature data of the target user is deleted from the face matching database of the face matching server 6. Note that when the facial feature amount is deleted, other information about the target user is also deleted, and information regarding the correspondence relationship between the facial feature amount and the user is also deleted.

Specifically, first, the face management server 5 sends a request to delete the facial feature amount to the face matching server 6. This request includes a matching group and a face registration ID.

When the face matching server 6 receives a deletion request from the face management server 5, the facial feature amount management unit 85 identifies the user based on the face registration ID added to the request, and identifies the user based on the face registration ID added to the request. Delete facial features from the face matching database. At this time, if the target user belongs to multiple verification groups, deletion processing is performed for each verification group. That is, the face feature amount of the target user is deleted from the face matching database for each matching group.

Furthermore, when the deletion process is completed, the face matching server 6 sends a response for facial feature amount deletion to the face management server 5. This deletion response includes a processing result indicating whether or not the deletion process was successfully completed.

Next, face registration performed during user registration processing will be described. FIG. 51 is an explanatory diagram showing an outline of face registration.

In the face management server 5, during the registration process regarding the user, first, the face matching server 6 performs the processing of facial image extraction, and after registering the facial image data obtained from the face matching server 6 in the face information database. , provides the face image data to the face matching server 6, and causes the face matching server 6 to perform face registration processing. In this face registration process, facial feature amounts are generated from the face image acquired from the face management server 5, and the facial feature amounts are registered in the face matching database.

Specifically, first, the face management server 5 sends a face registration request to the face matching server 6. This face registration request includes a user code, matching group, user registration ID, and face image data.

When the face matching server 6 receives a request for face registration from the face management server 5, the image quality checking unit 82 performs an image quality check on the user's face image added to the request. Next, the facial feature amount generation section 84 generates facial feature amount data from the user's facial image data, and the facial feature amount management section 85 assigns the user's facial feature amount data to the user's matching group. Register in the corresponding face matching database. At this time, association information between the user registration ID used in the face information database of the face management server 5 and the registration ID at the time of facial feature amount registration used in the face matching database of the face matching server 6 is saved. Furthermore, if the user registration ID has already been registered, the original facial feature amount is deleted and replaced. Additionally, a user code is registered as user information in the face matching database.

The face matching server 6 sends a face registration response to the face management server 5 after the registration process in the face matching database corresponding to all the matching groups to which the user belongs is performed. This response includes a processing result indicating whether or not the registration processing has ended normally.

Next, the face information synchronization process performed by the face management server 5 will be explained. FIG. 52 is an explanatory diagram showing an overview of face information synchronization processing.

The face information management unit 66 of the face management server 5 executes synchronization processing with the face matching server 6. In this synchronization process, it is assumed that the content registered in the face information database in the face management server 5 and the content registered in the face matching database in the face matching server 6 match. Specifically, the user information (face image) registered in the face information database of the face management server 5 and the user's facial feature amount data registered in the face matching database of the face matching server 6 match each other. Make it into something. Further, it is assumed that the registered contents regarding the matching group of the face matching server 6 match between the face management server 5 and the face matching server 6.

Here, regarding the timing of the synchronization process, the synchronization process is executed periodically at a predetermined cycle. Also, when user information is registered, updated, or deleted, synchronization processing is executed. Furthermore, synchronization processing is executed when a matching group is deleted or when the face matching server 6 is started.

Additionally, synchronization is performed for all users who are subject to synchronization processing. Additionally, synchronization is performed for specified users. In addition, synchronization is performed for users who updated during the specified period.

Furthermore, in this embodiment, the activation date and invalidation date for each user are registered as user information. The activation date is the date on which face verification can be enabled for the target user, and during the period in which the activation status is after this activation date, the user's face recognition will be performed and face recognition will not be performed. Can receive. The invalidation date is the date when face verification can be invalidated, and during the period in which the face matching is in the invalidation state after this invalidation date, the user's face authentication will not be performed and the user will not be able to receive facial authentication.

In addition, the face information management unit 66 of the face management server 5 performs validation and invalidation determination for all users as synchronization timing determination. When the user's face verification is enabled and the activation period begins, the face verification server 6 is instructed to perform face registration processing, that is, processing to register the target user's facial feature amount data. . In addition, when the user's face matching is disabled and enters the invalidation period, the face matching server 6 is instructed to perform face deletion processing, that is, processing to delete the facial feature amount data of the target user. Instruct. Therefore, the face information (facial feature amount data) of the user is not stored in the face matching server 6 during the period before the activation date or after the invalidation date.

In addition, the face management server 5 determines whether or not a predetermined process has been successfully completed in the face matching server 6, and if it is determined that the process has failed and needs to be re-executed, the face management server 5 asks the face matching server 6 to re-execute the process. instruct. Here, the target processes are face registration, face deletion, verification group registration, and verification group deletion.

Note that the face management server 5 records a synchronization log (history information of synchronization processing).

Next, the process of copying facial feature amount data will be explained. FIG. 53 is an explanatory diagram illustrating an overview of facial feature amount data copy processing.

In this embodiment, a face matching database is provided for each matching group in the face matching server 6, and facial feature data of users is managed for each matching group. In addition, multiple face matching databases for the same matching group are provided. Therefore, facial feature data of the same user needs to be registered in multiple face matching databases. On the other hand, the process of generating facial feature amount data from the user's face image data requires a large load, and it is not efficient to repeatedly perform this process on multiple face matching servers 6.

Therefore, in this embodiment, if the user's facial feature data has already been generated and registered in the face matching database, the facial feature data is copied between the two face matching databases. That is, facial feature data registered in the original face matching database can be used to register another face matching database.

The face information management unit 66 of the face management server 5 manages the registration status of facial feature data for each user in the face matching database based on the face information database, and if any changes regarding the user's facial image data are made. If there is, an instruction is given to copy the facial feature amount data between the two face matching databases (transfer instruction).

At this time, if there is a change on an individual basis, the face feature amount data on an individual basis is copied as shown in FIG. 53(A) (individual copy). For example, when a user is newly registered or updated, facial feature amount data for each individual is copied. In addition, in order to improve the accuracy of face matching for a specific user, when additional facial feature data is generated using different facial image data and multiple facial feature data are registered for the same user. There is. In this case as well, facial feature amount data on an individual basis is copied only for a specific user.

Furthermore, if there is a change in the matching group, as shown in FIG. 53(B), the face feature amount data is copied in the matching group (group copy). For example, when a new matching group is registered, the facial feature amount data of the first face matching database in which the facial feature amount data is registered is copied all at once to the second and subsequent face matching databases that have the same registered content. Furthermore, even when multiple face matching databases with the same registered content are provided for load distribution or redundancy, the user's facial feature amount data is copied in units of matching groups.

Next, processing during face authentication in this system will be explained. FIG. 54 is an explanatory diagram showing an overview of processing during face authentication.

When the face authentication device 1 detects a person's face from an image taken by the camera 11 and obtains a facial image, it transmits a request for face matching to the face matching server 6. This face verification request includes the device ID of the requesting face recognition device 1, the verification group of the face recognition device 1, photographed image data of the face recognition target, face frame information, and verification conditions. It will be done.

In the face matching server 6 , first, the facial feature amount generation unit 84 generates facial feature amounts of the target person from the facial image of the target person acquired from the face authentication device 1 . Next, the facial feature amount matching unit 86 of the face matching unit corresponding to the matching group of the face authentication device 1 compares the facial feature amount of the target person with the facial feature amount of the user registered in the face authentication database. . At this time, face verification is performed in a face recognition process in which the requesting face recognition device 1 and the verification group match. When the face matching is completed, the face matching response is sent to the requesting authentication device. This face matching response includes the matching result (success, failure), matching score, and user code.

The facial feature amount matching unit 86 calculates a matching score that represents the degree of similarity between the target person for face authentication and the registered user, and when this matching score is equal to or higher than a predetermined reference value, the face authentication The target person is regarded as the registered user himself/herself, and a matching result indicating that face matching was successful is generated. On the other hand, if the matching scores for all users do not exceed the standard value, it is determined that the person subject to face authentication is not a registered user, and a matching result is generated indicating that face matching has failed. .

Note that processing conditions for face matching (face matching parameters) performed by the face matching server 6 may be added to the matching request. This allows the face authentication device 1 to instruct the details of the face matching process performed by the face matching server 6. For example, as a processing condition for face matching, a threshold regarding a matching score is specified, and matching results in which the matching score is equal to or greater than a predetermined threshold are included in the response. Further, as a processing condition for face matching, the number of matching results is specified, and a predetermined number of matching results are included in the response starting from the one with the highest matching score.

Further, in the face matching server 6, when the face matching is completed, the authentication log management unit 87 stores information such as the matching results obtained in the face matching in the database as an authentication log (facial authentication history information). At this time, in addition to storing only the face authentication results (success, failure) as an authentication log, the matching score may also be included in the authentication log. Furthermore, the facial image of the target person acquired from the face authentication device 1 may be included in the authentication log. In this case, it is preferable to encrypt and save the facial image of the subject.

Note that if a large number of valid matching results with matching scores exceeding a threshold are obtained, the matching results may be narrowed down to a predetermined number of matching results starting from the one with the highest matching score and saved as an authentication log. In addition, if face matching does not complete normally or if a valid matching result with a matching score exceeding the threshold cannot be obtained, only the information included in the request from face recognition device 1 is recorded as an authentication log. You may also save it.

FIG. 55 is an explanatory diagram showing an overview of reference processing regarding authentication logs.

In response to a request from the management terminal 2 via the face management server 5, the face matching server 6 sends the authentication log (facial recognition history information) accumulated in its own device to the management terminal 2, and the management terminal 2 performs authentication. The log information can be viewed by the administrator. At this time, the administrator can specify conditions, and the administrator can view authentication logs that correspond to the specified conditions.

Specifically, on the management terminal 2, a screen for inputting search conditions for searching the authentication log is displayed, and on this screen, the administrator enters the face authentication device 1, which is the source of the request for face authentication, as the search conditions. You can specify the user, the number of extractions, the date and time when face matching was performed (extraction period), and the matching results (success, failure).

The authentication log presentation unit 69 of the face management server 5 sends a request to refer to the authentication log to the face matching server 6 when the administrator inputs search conditions and performs an operation to instruct a search on the management terminal 2 . This authentication log reference request includes search conditions specified by the administrator. At this time, the authentication log presentation unit 69 performs the target face authentication based on information regarding the association between the face authentication device 1 and the verification group, information regarding the association between the face verification server 6 and the verification group, and information regarding the user. A face matching server 6 whose matching group matches that of the device 1 or the user is identified, and a request to refer to the authentication log is sent to the face matching server 6. The face management server 5 also records an administrator access log.

When the face matching server 6 receives a request to refer to an authentication log from the face management server 5, it searches for authentication logs according to the search conditions included in the request, and generates a list of authentication logs that meet the search conditions as a search result. The authentication log list is added to the authentication log reference response and sent to the face management server 5. Note that the face image of the target person can be sent to the face management server 5 as an authentication log, but in this case, since the face image of the target person is encrypted, it is decrypted and added to the response.

When the face management server 5 receives the authentication log reference response from the face matching server 6, a list of authentication logs corresponding to the search conditions specified by the administrator is displayed based on the list of authentication logs included in the response. A search result screen is displayed on the management terminal 2. At this time, for search conditions that cannot be narrowed down by the face matching server 6, authentication logs that match the search conditions are extracted from the list of authentication logs to generate a list of narrowed-down authentication logs.

Note that on the management terminal 2, it is possible to instruct sorting by specifying an item, and when this operation is performed, the face management server 5 sorts the list of authentication logs by the specified item. to generate a sorted list of authentication logs. Furthermore, the management terminal 2 can specify the number of authentication logs to be acquired, and when this operation is performed, the face matching server 6 extracts authentication logs corresponding to the number of acquired authentication logs and generates a list of authentication logs. Furthermore, the management terminal 2 can specify a period, and when this operation is performed, the face matching server 6 extracts authentication logs included in the period and generates a list of authentication logs.

FIG. 56 is an explanatory diagram showing an authentication log reference screen displayed on the management terminal 2.

On the management terminal 2, an authentication log reference screen is displayed. This authentication log reference screen is provided with a search condition specifying section 151 and a search result display section 152.

In the search condition specifying section 151, each item of the device ID of the target face authentication device 1, the user code, the number of acquisitions, the date and time of verification, and the verification result can be input as search conditions. Note that in the verification date and time field, a period (starting point and ending point) can be specified. Further, in the verification result item, success, failure, and both success and failure can be selected from the pull-down menu. When the search button 153 is operated after inputting search conditions for each item in the search condition specifying section 151, a search is executed to search for authentication logs that correspond to the search conditions, and the search results are displayed in the search result display section 152. Is displayed.

The search result display section 152 displays a list of data items such as verification date and time, verification results (success, failure), and facial images as search results. In the face image item, if the matching result is successful, the face image of the person to be subjected to face authentication acquired from the face authentication device 1 and the registered user's face image are displayed. If the matching result is unsuccessful, only the facial image of the person to be facially authenticated is displayed. This allows the administrator to check the authentication logs related to the specified face authentication device 1 or user, and also narrows down the authentication logs based on the verification date and time and verification results.

In this search result display section 152, items can be specified and sorted. Furthermore, when the save button 154 is operated, the authentication log information displayed on the search result display section 152 is saved in a predetermined format. This allows the administrator to check authentication logs related to the specified face authentication device 1 or user in chronological order.

Next, the external application cooperation function of the face authentication device 1 will be explained. FIG. 57 is an explanatory diagram showing an overview of external application cooperation of the face authentication device 1.

The face authentication device 1 has a function of linking with an external application, and notifies an external application, that is, a management server 8 (management device) of an external system, of the authentication result obtained from the face matching server 6. The external system provides various services to users using the authentication results of this system. For example, the external system is an entrance/exit control system, and controls the door to be unlocked if the authentication result is successful. Further, the external system is a payment system, and controls the execution of payment if the authentication result is successful.

The face authentication device 1 also includes an authentication result adjustment unit 25 (connection application), and notifies the management server 8 of the external system of the authentication result via the authentication result adjustment unit 25. The face authentication device 1 notifies the authentication result adjustment unit 25 of the authentication result (success, failure) via a named pipe (interprocess communication). The authentication result adjustment unit 25 notifies the management server 8 of the external system of the authentication result via the network using a predetermined protocol (WebSocket). Note that the authentication result adjustment unit 25 may perform necessary processing such as processing or adding information, depending on how the external system uses the authentication result.

The face matching request unit 23 of the face authentication device 1 sends a face matching request to the face matching server 6, and upon receiving a face matching response from the face matching server 6 in response, the face matching request unit 23 sends a face matching request to the face matching server 6. Authentication success, authentication failure) is written to the named pipe and the connected application is notified. At this time, the face authentication device 1 further acquires the identification information of the authenticator and the camera 11 as the authentication result, and adds it to the notification of the authentication result. Specifically, if the authentication result is successful, the user code and the device ID of the face authentication device 1 (camera 11) are written into the named pipe. On the other hand, if the authentication result is a failure, only the authentication result (authentication failure) is written to the named pipe.

In addition, the authentication result adjustment unit 25 of the face authentication device 1 suppresses notification of the authentication result so as not to repeatedly notify the external system of the same authentication result. In this embodiment, when a connection application is notified of an authentication result, a notification suppression time is set, and within the notification suppression time, notifications of authentication results with the same content are temporarily stopped.

When a notification of successful authentication regarding a certain person is given, notification of successful authentication regarding that person is suppressed for a predetermined period of time (notification success notification suppression time) from the notification of successful authentication. Further, when a notification of authentication failure regarding a certain person is given, notification of authentication failure regarding that person is suppressed within a predetermined time (authentication failure notification suppression time) from the notification of authentication failure.

This notification suppression time is set for each authentication result (authentication success, authentication failure) and each person. If the authentication result is different, the notification suppression time is set separately for the same person, and even if the authentication result is the same but the notification is suppressed for different people. The times are set separately. Therefore, within the notification suppression time of authentication failure regarding a certain person, notification of authentication success regarding that person can be given, and within the notification suppression time of authentication success regarding a certain person, notification of authentication failure regarding that person cannot be given. can. Additionally, within the notification suppression time for authentication failure regarding a certain person, notification of authentication failure regarding another person can be sent, and within the notification suppression time for authentication success regarding a certain person, notification of authentication success regarding another person can be sent. be able to.

Next, the operating status monitoring function of this system will be explained. FIG. 58 is an explanatory diagram showing an overview of the operating status monitoring function in this system.

The face authentication device 1 includes an operating status monitoring section 26. The operating status monitoring unit 26 monitors the operating status of its own device, and periodically notifies the face management server 5 (monitoring server) of the operating status (life and death status) of its own device at a predetermined period. The face verification server 6 also includes an operating status monitoring section 89. The operating status monitoring unit 89 monitors the operating status of its own device, and periodically notifies the face management server 5 (monitoring server) of the operating status (death and death status) of its own device at a predetermined period.

The face management server 5 includes an operating status monitoring section 68. The operating status monitoring unit 68 monitors the operating status of its own device, and periodically records the operating status (alive status) of its own device in the own device at a predetermined period. Further, the face management server 5 records the operating status notified from the face authentication device 1 and the face matching server 6 in its own device.

Here, the information regarding the operating status that is notified includes the device ID, acquisition time, life and death status, program version information, and the like.

Further, in the management terminal 2, the administrator can check the operating status by requesting the face management server 5 to refer to the operating status.

Next, updating of the face matching database in accordance with the version upgrade of the program in the face matching server 6 will be explained.

In the face matching server 6, in order to improve the accuracy of face matching, the versions of the facial feature amount generation program and the face matching program are upgraded (updated) as appropriate. On the other hand, facial feature data depends on the algorithm for facial feature generation and face matching, and if the algorithm is changed, the facial feature data also changes, so it is necessary to regenerate the facial feature data.

Therefore, in this embodiment, when a program related to facial feature generation and face matching algorithms is upgraded, the facial feature data registered in the face matching database is updated to correspond to the new program. . That is, the facial feature amount data that has already been stored is deleted, the facial feature amount data is regenerated based on the facial image data at the time of user registration, and the new facial feature amount data is stored.

Specifically, when the face management server 5 detects an upgrade of the face matching program using the operating status notification function, it sends a request to the face matching server 6 to update the facial feature amount data. This request includes an image of the user's face. Upon receiving the request to update the facial feature data, the facial feature management unit 85 in the face matching server 6 causes the facial feature generation unit 84 to perform a process of generating facial feature data from the user's face image. Then, the new facial feature data is registered in the face matching database.

At this time, if the face matching processes and face matching databases are the same in the plurality of face matching servers 6, that is, the facial feature amount data registered in the face matching databases of the plurality of face matching servers 6 are the same. Yes, and in the face matching process, when face matching processing is performed for face authentication devices 1 of the same matching group, one face matching server 6 regenerates the facial feature amount data, When the new facial feature data is registered in the face matching database, the facial feature data registered in the face matching database is copied to another face matching server 6, so that the registered content in the face matching database is the same. do.

At this time, the face management server 5 instructs the destination face matching server 6 to perform replication, and the face feature data stored in the source face matching server 6 is used for face matching of the destination through the face management server 5. It may be transferred to the server 6.

Next, backup and restoration will be explained.

In this embodiment, databases and files stored in the face management server 5 and face matching server 6 can be backed up and restored.

The information stored in the face management server 5 that is to be backed up is the user information (face information) database, face image files, matching group information database, authentication device and matching group association information database, and matching. These include a database of server and collation group association information, and administrator access log files.

The information stored in the face verification server 6 that is to be backed up includes the authentication log database, face image files, and the like.

FIG. 59 is an explanatory diagram showing a backup screen and a restore screen displayed on the management terminal 2. The backup screen and restore screen can be switched using tabs within the screen.

As shown in FIG. 59(A), the backup screen is provided with a backup destination designation section 161, a backup target designation section 162, a progress display section 163, and an execution button 164. In the backup destination specifying section 161, a backup destination can be specified. The backup target designation unit 162 can select the face management server 5 and the face matching server 6 as devices to be backed up. When the execution button 164 is operated, the backup is executed. The progress display section 163 displays a list of data items to be backed up. In this data item, a check mark is displayed when the backup is completed, allowing the administrator to recognize the progress of the backup.

Here, the data to be backed up is a database of user information (face information), a file of registered user face images, a database of verification group information, and association information between the face authentication device 1 and the verification group. database, a database of association information between the face matching server 6 and matching groups, an administrator access log file, an authentication log database, and a face image file of the person to be face authenticated.

As shown in FIG. 59(B), the restore screen is provided with a restore destination designation section 165, a restore target designation section 166, a progress status display section 167, and an execution button 168, similar to the backup screen. In the restore destination designation section 165, a restore destination can be designated. The restoration target specifying unit 166 can select the face management server 5 and the face matching server 6 as devices to be restored. When the execution button 168 is operated, the restoration is executed. The progress display section 167 displays a list of data items to be restored. In this data item, a check mark is displayed when the restoration is completed, allowing the administrator to recognize the progress of the restoration.

Next, a modification of this embodiment will be described. FIG. 60 is an explanatory diagram showing an overview of face image extraction processing according to a modification of the present embodiment.

In the embodiment described above (see FIG. 12), the face image extraction process for generating face image data from the photographed image data of the user is performed by the face authentication device 1 and the face matching server 6, but in this modification, Face image extraction processing is performed by a face authentication device 1 and a face management server 5, and face image extraction processing is not performed by a face matching server 6. Specifically, the face image extraction unit 83 provided in the face matching server 6 in the above embodiment is omitted, and the face image extraction unit 201 is provided in the face management server 5.

At the time of user registration, the management terminal 2 sends photographed image data of the user to the face management server 5, so that the face image extraction unit 201 of the face management server 5 performs face image extraction processing. On the other hand, during face authentication, the face image extraction unit 22 of the face authentication device 1 performs a face image extraction process, which is the same as in the embodiment described above.

As described above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in the present disclosure is not limited to this, and can also be applied to embodiments in which changes, replacements, additions, omissions, etc. are made. Furthermore, it is also possible to create a new embodiment by combining the components described in the above embodiments.

In recent years, in the world of IoT (Internet of Things), CPS (Cyber Physical Systems), a new concept that creates new added value by linking information between physical space and cyber space, has been attracting attention. Under these circumstances, the CPS concept can also be adopted in this embodiment. In other words, as a basic configuration of CPS, for example, an edge device located in physical space and a cloud server located in cyberspace are connected via a network, and the processors installed in the edge device and cloud server It is possible to perform authentication processing in a distributed manner. In this face authentication process, the edge device acquires the user's face image data photographed by the camera of the authentication device. Next, the cloud server executes processing to generate feature data from the face image data received from the edge device via the network, and compares it with pre-registered user feature data to authenticate the user. The process is executed, and the authentication result related to the authentication process is taken over by the facial recognition application software and displayed on a display or the like in an output format defined on the application. Here, each data generated on an edge device or a cloud server is preferably generated using a web application installed on a standardized platform, and by using such a standardized platform, various types of sensor groups can be generated. It is possible to improve efficiency when building systems including IoT application software.

The face recognition system, face recognition server device, face recognition machine, face recognition method, and face recognition program according to the present invention have the advantage that the face recognition function can be operated stably and efficiently for each face recognition service provider. The present invention is useful as a face recognition system, a face recognition server device, a face recognition machine, a face recognition method, a face recognition program, etc. that performs face recognition processing based on image data obtained by photographing a subject.

1 Face recognition device 2 Management terminal (terminal device, face recognition management device)
3 Face recognition server (server device)
5 Face management server (face image management department)
6 Face matching server (face image matching section)

Claims (4)

A face recognition system that executes face recognition processing based on image data taken of a person to be face recognized,
Multiple facial recognition machines,
A server device connected to this face recognition machine via a network,
has
The face recognition machine is
a camera that photographs the subjects that appear one after another ;
a face matching request unit that transmits a face matching request including face image data of the subject obtained through photography by the camera, a matching group of the face authentication device, and matching conditions to the server device ;
an authentication result receiving unit that receives an authentication result of the target person from the server device;
Equipped with
The server device includes:
a request receiving unit that receives the face verification request of the target person transmitted from the face authentication device;
a facial feature generation unit that executes a process of generating facial feature data of the target person from facial image data of the target person;
a facial feature matching unit that performs matching processing between the target person's facial feature data and the registrant's facial feature data for each matching group ;
an authentication result transmitting unit that transmits a face matching response including the authentication result of the subject person by the matching process of the facial feature matching unit to the face authentication machine,
The face authentication device instructs a threshold regarding the matching score as the matching condition included in the face matching request, and the server device specifies the matching score that is equal to or higher than the threshold when the authentication result of the subject is successful. A face authentication system characterized by transmitting the face matching response including the face matching response to the face authentication device . The server device includes:
2. The face authentication system according to claim 1, further comprising an authentication history management unit that manages face authentication history information including authentication results and matching scores obtained in the matching process . The server device includes:
2. The face authentication system according to claim 1, further comprising a plurality of said facial feature matching units for each of said matching groups corresponding to said face authentication device. A face recognition method in which a face recognition machine and a server device perform face recognition processing based on image data of a person to be subjected to face recognition, the method comprising:
The processors of the plurality of face authentication machines connected to the server device via a network,
Photographing the subjects who appear one after another with a camera,
Sending a face matching request including face image data of the target person obtained by photographing with the camera, a matching group of the face authentication device, and matching conditions to the server device ;
receiving the authentication result of the target person from the server device;
The processor of the server device includes:
receiving the face matching request of the subject from the face authentication device;
performing a process of generating facial feature data of the target person from facial image data of the target person;
performing a matching process between the target person's facial feature data and the registrant's facial feature data for each matching group ;
Sending a face matching response including the authentication result of the subject obtained here to the face authentication machine,
Further, the processor of the face authentication device instructs a threshold regarding a matching score as the matching condition included in the face matching request, and the processor of the server device specifies the threshold when the authentication result of the subject is successful. A face authentication method characterized in that the face matching response including the above matching score is transmitted to the face authentication device .

Images