JP2006065447A - Discriminator setting device, degree-of-attention measuring device, discriminator setting method, degree-of-attention measuring method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a degree-of-attention measuring technology which detects only the face of a person to whom attention is paid, as an object, from images photographed by an image pickup device and measures the degree of attention. <P>SOLUTION: A degree-of-attention measuring device 10 comprises an image inputting part 11, a discriminator 14, a discrimination result integrating part 15 and a degree-of-attention computing part 13. The discriminator 14 and the discrimination result integrating part 15 are composed by a discriminator setting device 20 which comprises, an image data accumulating part 21 which accumulates a plurality of images including the images of persons' faces, a degree-of-attention measuring condition inputting part 25 which inputs degree-of-attention measuring conditions, a discriminator usage table generating part 24 which generates a discriminator usage table, a discriminator specification table 23 which stores discriminator specification information, and an integrated discriminator setting part 22 which sets a plurality of discriminators and the discrimination result integrating part 15 based on the discriminator specification information stored in the discriminator specification table and the image data accumulated in the image data accumulating part 21. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an attention level measurement technique for measuring an attention level of an object from an image obtained by photographing a measurement target range.

  It is important to measure the degree of attention of advertisements and products to measure the effectiveness of advertisements and to promote sales. Conventionally, such attention has been made by collecting data on site all day by hand, and measuring the effectiveness of advertisements through street questionnaires. However, the manual method has a large burden of attention level measurement.

On the other hand, a camera is attached to the advertising media side to detect the face of the person in the captured image, measure the direction of the person's face from the outline of the person's face and the distance between both eyes, and consider the positional relationship between the camera and the advertising media. A method for detecting a person who is paying attention to advertisement media has been proposed (see, for example, Patent Document 1).
Similarly, a method has also been proposed in which a person's line of sight is detected from a photographed image to detect a person of interest (see, for example, Patent Document 2).
JP 2000-209578 A JP 2002-269290 A

  However, in the method of measuring the face orientation of a person from the face outline and the distance between both eyes, the camera installation position must be taken from the angle close to the front of the person in question. In addition, in order to detect the line of sight, a high-resolution imaging device is required to capture the eyes of a person in the image. Furthermore, in any of the methods, it is necessary for the system provider to configure a discriminator for the person who is paying attention for each installation position of the camera according to the installation location.

  In other words, by simply inputting the face angle and size that define the person that the user is paying attention to from the image captured by the imaging device installed freely in the vicinity of the presentation, regardless of the type of imaging device equipment, There was no technology to automatically configure a face detector that can detect a person who is paying attention and measure the degree of attention, and measure the degree of attention.

  An object of the present invention is to provide a degree-of-interest measurement technique that can detect only the face of a person who pays attention to an object and measure the degree of attention from images captured by an imaging device freely installed by a user. .

  Therefore, in order to solve the above-mentioned problem, claim 1 includes: an image input unit that inputs an image to be measured; a plurality of classifiers that identify a person's face from the input image; and the plurality of identifications The plurality of identifications of the attention level measuring device, comprising: an identification result integration unit that integrates a detector and detects a person's face facing a predetermined range; and an attention level calculation unit that calculates a degree of attention from the detection result A discriminator setting device for setting a discriminator and an identification result integration unit, an image data storage unit for storing a plurality of images including a human face, and an identification for setting the plurality of discriminators and the discrimination result integration unit Discriminator specification information input means for inputting discriminator specification information, discriminator specification table for storing the inputted discriminator specification information, discriminator specification information and image data stored in the discriminator specification table. It characterized by having a a combined discriminator setting means for setting the plurality of classifiers and the identification result integration unit based on the stored image in the storage unit.

  According to a second aspect of the present invention, an image input unit that inputs an image to be measured, a plurality of classifiers that identify a person's face from the input image, and the plurality of classifiers are integrated into a predetermined range. A plurality of discriminators and an identification result integrating unit of an attention level measuring device including: an identification result integrating unit that detects a face of a person facing in a direction; and an attention level calculating unit that calculates an attention level from the detection result. A classifier setting device for setting, an image data storage unit for storing a plurality of images including a person's face, and subject information indicating a position of the person's face and a captured state in the image, and an attention degree measurement condition Attention level measurement condition input means for input, discriminator specification table creation means for creating a discriminator specification table based on the attention level measurement conditions and subject information stored in the image data storage means, A discriminator specification table for storing the discriminator specification table formed; the discriminator specification information stored in the discriminator specification table; and the plurality of discriminators based on the image stored in the image data storage means; And an integrated discriminator setting unit for setting the identification result integration unit.

  Further, in the third aspect of the present invention, the classifier specification table includes a classifier usage number indicating a usage number of the plurality of classifiers, a classifier type indicating each type of the plurality of classifiers to be used, and the classifier used. Discriminator configuration variable information that is information on variables for configuring a plurality of discriminators, and an identification result integration function that is information on a function that integrates the plurality of discriminators used in the discrimination result integration unit. And storing information.

  According to a fourth aspect of the present invention, the classifier setting device includes classifier setting information storage means for storing information for configuring the classifier based on the classifier type.

  Further, according to claim 5, each of the plurality of classifiers learns learning data based on an image of a person's face, or learning data based on both an image of a person's face and an image other than the person's face. The discriminator configuration variable information is learning data based on a human face image or an image other than a human face corresponding to each of the plurality of discriminators. Information relating to learning data based on one or more images including different images for each classifier, wherein the integrated classifier setting means stores the learning data corresponding to the classifier configuration variable information in the image data storage means And the discriminator is made to learn.

  According to a sixth aspect of the present invention, image input means for inputting an image to be measured, a plurality of classifiers for identifying a person's face from the input image, and identification results of the plurality of classifiers are integrated. And an identification result integration unit that detects a face of a person facing a predetermined direction, and an attention level calculation unit that calculates a degree of attention from the detection result.

  Further, according to claim 7, each of the plurality of classifiers learns learning data based on a person's face image or learning data based on both a person's face image and an image other than the person's face. Learning discriminating type discriminators that discriminate according to the above, each of which is learning data based on an image of a person's face, or one or more images that are images other than a person's face and that are different from each other It is comprised by learning the learning data based on.

  Further, according to an eighth aspect of the present invention, each of the plurality of classifiers learns learning data based on one or more images each including an image of a person's face and an image having a different shooting angle for each classifier. It is characterized by comprising.

  Further, according to a ninth aspect of the present invention, each of the plurality of classifiers is learning data based on an image of a person's face, or an image other than a person's face, and images having different image sizes for each of the classifiers. It is comprised by learning the learning data based on the 1 or more image containing.

  A tenth aspect of the present invention is characterized in that the plurality of classifiers and the identification result integrating unit are set by the classifier setting device according to any one of the first to fifth aspects.

  Further, according to an eleventh aspect of the present invention, the attention level calculating unit extracts a feature of the person from the input image with respect to the face person detected as being focused by the identification result integrating unit, Tracking image storage means for storing the extracted result and the input image in association with each other, person tracking means for extracting a person in the stored input image and tracking the movement of the extracted person, and the person Attention level calculating means for calculating the level of attention based on a tracking result.

    Further, in the twelfth aspect, the step of inputting an image to be measured by the image input means, the step of identifying a person's face from the inputted image by a plurality of classifiers, and the step of integrating the identification results The plurality of identifications in the attention level measuring method, comprising: integrating a classifier; detecting a person's face facing a predetermined range; and calculating a degree of attention based on the detection result. A discriminator setting method for setting a discriminator and the discrimination result integrating means, wherein the image data accumulating means accumulates a plurality of images including a human face, and the discriminator specification information input means includes the plural discriminators and A step of inputting discriminator specification information for setting the identification result integration means; a step of storing the input discriminator specification information in a discriminator specification table; Setting the plurality of discriminators and the discrimination result integrating means based on the discriminator specification information stored in the discriminator specification table and the image stored in the image data storage means. And

  Further, according to a thirteenth aspect of the present invention, the step of inputting an image to be measured by the image input means, the step of identifying a person's face from the inputted images by a plurality of classifiers, The plurality of identifications in the attention level measuring method, comprising: integrating a classifier; detecting a person's face facing a predetermined range; and calculating a degree of attention based on the detection result. And a classifier setting method for setting the classification result integrating means, wherein the image data storage means includes a plurality of images including a person's face, and subject information indicating a position of the person's face in the image and a shooting state. A step of storing, a step of inputting the attention level measurement condition by the attention level measurement condition input unit, and a discriminator specification table creating unit configured to store the attention level measurement condition and the image data storage unit. A step of creating a discriminator specification table based on the object information stored in the discriminator, a step of storing the discriminator specification table by the discriminator specification table, and an integrated discriminator setting means in the discriminator specification table. Setting the plurality of discriminators and the discrimination result integrating unit based on the stored discriminator specification information and the image stored in the image data storage unit.

  Further, in the present invention, the discriminator specification table includes a discriminator usage number indicating the usage count of the plurality of discriminators, a discriminator type indicating each type of the plurality of discriminators to be used, and the using the discriminator specification table. Discriminator configuration variable information that is information on variables for configuring a plurality of discriminators, and an identification result integration function that is information on a function that integrates the plurality of discriminators used in the discrimination result integration unit. And storing information.

  Further, according to claim 15, in the classifier setting method, the classifier setting information storage means stores information for configuring the classifier based on the classifier type, and the information is stored in the integrated classifier. The setting means is provided.

  Further, according to claim 16, each of the plurality of classifiers learns learning data based on an image of a person's face, or learning data based on both an image of a person's face and an image other than the face of a person. The discriminator configuration variable information is learning data based on a human face image or an image other than a human face corresponding to each of the plurality of discriminators. Information relating to learning data based on one or more images including different images for each classifier, wherein the integrated classifier setting means stores the learning data corresponding to the classifier configuration variable information in the image data storage means And the discriminator is made to learn.

  Further, according to a seventeenth aspect of the present invention, the step of inputting an image to be measured by the image input means, the step of identifying a human face from the inputted images by a plurality of classifiers, and the plurality of identification result integrating means The identification results of the classifiers are integrated to detect a person's face facing a predetermined direction, and the attention level calculating means calculates the attention level from the detection results.

  Further, according to claim 18, each of the plurality of classifiers learns learning data based on a person's face image or learning data based on both a person's face image and an image other than the person's face. Learning discriminating type discriminators that discriminate according to the above, each of which is learning data based on an image of a person's face, or one or more images that are images other than a person's face and that are different from one another for each discriminator It is comprised by learning the learning data based on.

  Further, according to claim 19, each of the plurality of classifiers learns learning data based on one or more images each including an image of a person's face and an image having a different shooting angle for each of the classifiers. It is characterized by comprising.

  Further, according to claim 20, each of the plurality of classifiers is learning data based on an image of a person's face, or an image other than a person's face, and images having different image sizes from one another for each classifier. It is comprised by learning the learning data based on the 1 or more image containing.

  A twenty-first aspect is characterized in that the plurality of classifiers and the identification result integrating unit are set by the classifier setting method according to any one of the twelfth to sixteenth aspects.

  Further, according to a twenty-second aspect of the present invention, the attention level calculating unit extracts a feature of the person from the input image with respect to a face person detected as being noticed by the face detection unit by the identification result integration unit. The tracking image storage means stores the extraction result in association with the input image, and the person tracking means extracts the person in the stored input image and tracks the movement of the extracted person. And a step of calculating the degree of attention based on the person tracking result.

  A twenty-third aspect describes the classifier setting device, the attention level measuring device, the classifier setting method, or the attention level measuring method according to any one of the first to second aspects in a computer program. It is characterized by making it executable.

  In the present invention, due to the characteristic configuration as described above, the orientation angle of the face for defining a person of interest from an image photographed by a freely installed imaging device, regardless of the type of imaging device equipment, It is possible to automatically configure a face detector that can detect a person who is paying attention to an object and measure the degree of attention simply by inputting the size.

  In addition, an apparatus that includes a detector configured according to the present method and detects a person's face can perform processing via a network.

  Furthermore, the attention level can be continuously monitored by storing the measurement result in a storage medium.

  As described above, according to the present invention, it is possible to automatically configure a discriminator that detects only the face of a person having a certain range of variation from an input image.

  As a result, only by inputting the angle and size of the face that is the definition of the person who is paying attention to the object, the image was taken by an imaging device freely installed near the object by the user regardless of the type of the imaging device. It is possible to automatically configure a face detector that can detect a person who is paying attention to an object from an image and measure the degree of attention, and measure the degree of attention.

  FIG. 1 is a diagram showing an outline of the configuration of the attention level measurement system. The attention level measurement system includes an attention level measurement device 10 and a discriminator setting device 20.

  The attention level measurement device 10 includes an image input unit 11, an integrated discriminator 12, and an attention level calculation unit 13, and the integrated discriminator 12 includes a plurality of discriminators 14 and an identification result integration unit 15. The plurality of discriminators 14-1 to 14 -N and the discrimination result integration unit 15 in the integrated discriminator 12 are set by the output from the discriminator setting device 20.

  The image input unit 11 inputs an image obtained by photographing the measurement area.

  The integrated discriminator 12 detects the face of a person who is looking at an object such as a posting from the input image, and outputs the number of persons and the position in the image.

  The attention level calculation unit 13 calculates the value of the attention level based on the output of the integrated discriminator 12. For example, when the attention level is defined as “the number of people who are paying attention”, the number of people watching the target is output as it is. In addition, when the degree of attention is defined as “the ratio of the person who is looking at the object to the person in the measurement area”, the number of all persons present in the image is separately measured and the result is used. To calculate the degree of attention. For this measurement, a discriminator capable of detecting all the persons in the measurement area may be set and used, or another means such as a sensor may be used. Further, the person may be tracked based on the detected face information, and the time during which the person has focused on the object may be measured and the value may be defined as the degree of attention.

  Further, if possible depending on the resolution, a part feature that can be used for gender identification, age estimation, and personal recognition may be extracted, and the extracted feature of the person may be reflected in the degree of attention. For example, using “Erina Kajikawa, Kiyoshi Hosoi,“ Automated gender and age estimation using facial images ”, OMRON, Sensing Proceedings 1 Sensing Special Feature on Ubiquitous Network Society, p.37-41, May be calculated.

  Note that the degree of attention is not limited to the above example, and may be selected from the various definitions described above, or another definition of the degree of attention may be freely set by the user.

  Here, a configuration example of the discriminator setting device 20 will be described with reference to FIG.

  The classifier setting device 20 includes an image data storage unit 21, a classifier specification table 23, a classifier setting information storage unit 26, an attention level measurement condition input unit 25, a classifier specification table creation unit 24, and an integrated classifier setting unit 22. Consists of

  The image data storage unit 21 stores an image of a subject to be detected and information about the subject in the image.

  The attention level measurement condition input unit 25 inputs a condition to be determined as a focused person as the attention level measurement condition. Here, the conditions for determining as the person of interest are specifically the angle of the face direction of the person to be detected, the size of the face area, and the like.

  The discriminator specification table creation unit 24 automatically creates the discriminator specification table 23 based on the input attention level measurement condition.

  The integrated discriminator setting unit 22 automatically sets the integrated discriminator 12 for identifying the face of a person having a target orientation and size based on the discriminator specification table 23.

  The discriminator specification table 23 stores information necessary for configuring the integrated discriminator 12 in which a plurality of discriminators 14 that identify a person's face are combined and integrated.

  Here, in order to explain the example of the discriminator specification table 23 in detail, a specific example will be described below.

  In the discriminator specification table 23, the discriminator type indicating the type of individual discriminators 14 to be combined in the integrated discriminator setting unit 22, the discriminator usage number indicating the usage number of the classifiers 14 to be combined, and the discriminator 14 are configured. Discriminator configuration variable information indicating variables and identification result integration function information indicating functions for integrating the discriminators 14 are stored.

  Specifically, the classifier type is a classifier type indicating the type of the classifier 14 such as template matching or learning discrimination method.

  The number of used classifiers is the number of used classifiers 14 to be combined when configuring the integrated classifier 12.

  The discriminator configuration variable information is information on variables constituting the discriminator 14 such as a template matching template and a learning data set of a learning discriminant method.

  The identification result integration function information is information on a function that integrates the classifier 14.

  These values are preset in the discriminator specification table 23 as defaults, and these values are reset according to the attention level measurement condition or variable by the user.

  The image data storage unit 21 stores an image group including a person's face, and a parameter (hereinafter referred to as subject information) indicating a shooting state of the person's face in the image.

  For example, a subject information table in which information such as the position, size, color, shape, orientation, and feature point of a person's face in the image and the face part is tabulated is stored in association with the image in advance. To do.

  Further, it is assumed that the subject information includes parameters representing a state that is an index having an order relationship, specifically, parameters such as a captured angle and a size in the image.

  Assume that the accumulated image group and subject information can cover continuous fluctuations that a person's face can take. For example, it is assumed that images and subject information for each fixed rotation angle are accumulated for rotation fluctuations.

  Further, the classifier use number determination table as shown in FIG. 3 that records the allowable variation range that can be handled for each classifier type and classifier configuration variable, and when the classifier type and classifier configuration variable are determined, The number of classifiers configured in accordance with the attention level measurement condition may be determined.

  The attention level measurement condition input unit 25 inputs an attention level measurement condition indicating a condition of how to detect the face of a target person from the input image. Specifically, it specifies the angle range of the face orientation that defines the person of interest and the size of the face area in the image.

  In addition, as the attention level measurement condition, the classifier type, the number of classifiers used, the classifier configuration variable information, and the identification result integration function information may be specified together.

  Note that these attention level measurement conditions may be set according to the purpose by setting an initial value in advance.

  The discriminator specification table creation unit 24 creates the discriminator specification table 23 using data stored in the image data storage unit 21 in accordance with the input attention level measurement condition. For example, when it is desired to detect a face from the front direction to 30 degrees to the right, a feature that can detect a face from the front direction to 30 degrees to the right (hereinafter referred to as a common feature) is stored in the image group of the image data storage unit 21. Of these, the image is created by extracting a face image of the corresponding angle. However, this setting may be performed automatically or may be set by the user.

  The integrated discriminator setting unit 22 sets the discriminator 14 based on the created discriminator specification table 23, integrates it, and sets the discriminator 14 that satisfies the attention degree measurement condition. The discriminator 14 is set from the discriminator configuration variable information of the discriminator specification table 23 according to the discriminator type of the discriminator specification table 23. Further, the classifier 14 is set as a combined integrated classifier 12 according to the number of uses and identification result integrated function information.

  Here, information for configuring the discriminator 14 based on the discriminator type information is stored in the discriminator setting information storage unit 26. For example, when a discriminant function is set by a learning discriminator to determine whether or not the subject is an object, information such as a processing procedure or a program for setting the discriminant function by the learning discriminator is stored in the discriminator setting information storage unit 26. It is assumed that

Here, the flow of processing for attention level calculation of the attention level measurement device 10 will be described with reference to FIG. The specific processing flow is as follows.
Step 41: Input an image to be identified into the image input unit 11,
Step 42: Attention level measurement conditions are input to the attention level measurement condition input unit 25.
Step 43: The discriminator specification table creation unit 24 creates the discriminator specification table 23 based on the inputted attention level measurement condition.
Step 44: The integrated discriminator setting unit 22 sets the discriminator 14 based on the set value of the discriminator specification table 23 and the image data of the image data storage unit 21.
Step 45: The integrated discriminator setting unit 22 sets the discrimination result integration unit 15 based on the set value of the discriminator specification table 23.
Step 46: Each discriminator 14 identifies a human face from the input image.
Step 47: The identification result integration unit 15 integrates the identification results of the classifiers 14,
Step 48: If the readjustment of each discriminator 14 and the identification result integration unit 15 is performed, go to Step 43, otherwise go to Step 49.
Step 49: The attention level calculator 13 calculates the attention level based on the identification result of the integrated classifier 12.

  In the example of FIG. 5 described later, it is assumed that readjustment of the discriminator specification table 23 is performed based on the discrimination result (step 48). However, when readjustment of the discriminator specification table 23 is not performed, step 41 is performed. Steps 42 to 45 may be exchanged, and after setting of the discriminator 14 and the discrimination result integration unit 15 is finished, an image to be discriminated may be input.

  Here, an example of the configuration of the attention level calculation unit 13 of the attention level measurement device 10 will be described with reference to FIG.

  Attention level measurement device 10 first performs face detection from an input image input by the user, using integrated discriminator 12 set based on the attention level measurement condition similarly input by the user.

  Next, the face identification unit 61 extracts the characteristics of the person from the input image with respect to the person whose face is detected as being noticed by the integrated classifier 12.

  The extracted result input is stored in the tracking image storage unit 62 in association with the image, and can be used by the attention level calculation unit 65. For example, as described above, gender information and age information may be extracted, or the distance from the presentation may be extracted as a feature by means of different means such as the size of the detected face area or a sensor. Further, the feature extracted here may be used as information for tracking the person.

  Next, the person counting unit 63 counts the number of persons who are paying attention to the presentation object in the measurement area based on the face area detected as being noticed by the integrated discriminator 12.

  The number of people of interest counted here may be presented to the user as the degree of attention as it is, or in the degree-of-interest calculation unit 65, the measurement counted by a different means such as a classifier 14 or a sensor separately set by the method according to the present invention. Using the number of people in the area as the denominator and the number of people in the numerator as the numerator, calculate how much of the people in the measurement range were paying attention to the object and use the result as the degree of attention. May be presented.

  The attention time of the person who is paying attention is measured by tracking the person image stored in the tracking image storage unit 62 by the person tracking unit 64. As a tracking method of the person tracking unit 64, for example, the person can be tracked even when there is a shielding by a person passing between the presented matter and the person disclosed in JP-A-7-192136. Techniques can be applied.

  Here, an example of the attention level measurement condition will be described with reference to FIG.

  The user of the system sets the orientation and size of a person's face that can be determined to be paying attention to the presentation in the input image as the attention level measurement condition. For example, when it is desired to measure the degree of attention of a poster posted on a station as a presentation, if the imaging device is set on the left side of the poster, the person of interest changes from the front to the right with respect to the camera. The angle at this time is input as the attention level measurement condition. Moreover, even if the person is a front-facing person, it cannot be said that a person away from the poster is looking at the poster. Therefore, the size of the face area to be detected can be specified. However, the attention level measurement condition is not limited to the above.

  Further, it may be set in more detail. For example, as shown in FIG. 8, measurement space parameters (a to f in FIG. 8), tracking time, and the like are input as attention degree measurement conditions. These conditions may be tabulated so that they can be used by the integrated discriminator setting unit 22, or may be used as they are in order to create the discriminator specification table 23 in the discriminator specification table creation unit 24.

  Here, the angle range of the orientation of the face of the person of interest is calculated from the measurement space parameter, and the discriminator specification table creation unit 24 creates the discriminator specification table 23 based on this information. The horizontal angle of the face is, for example, a person standing at the point A (the front left point near the posting) of the measurement region when the measurement region of FIG. 8 is viewed from above is equal to or greater than the angle formed by the line segment AO and the line segment AI. A person standing at a range below the angle formed by the line segment AO and the line segment AJ (for example, a range of 30 degrees to 45 degrees to the right) and a point D (the left-point behind the posting) is the line segment DO and the line segment It is obtained such that the angle is greater than the angle formed by DI and less than the angle formed by line segment DO and line segment DJ. Of the four 8 angles, the largest angle is the upper limit, and the smaller angle is the lower limit. The discriminator specification table creation unit 24 creates a discriminator specification table 23 for configuring the discriminator 14 capable of detecting this direction.

  Here, an example of the discriminator specification table 23 automatically created by the discriminator specification table creation unit 24 of the discriminator setting device 20 will be described with reference to FIG.

  The discriminator specification table 23 stores an object to be discriminated, the number of discriminator used, discriminator configuration variable information, discriminator type, and discriminant result integration function information. These items are set as default and their values are variable.

  The number of classifiers used indicates the number of classifiers 14 to be combined to set the integrated classifier 12 that is the final classifier.

  The discriminator configuration variable information indicates a feature common to the subject to be detected that is used when identifying the subject and that meets the subject identification condition (hereinafter referred to as a common feature). When template matching is used as the discriminator 14, it becomes a subject template, and when a learning discriminating discriminator is used, it becomes the size of subject learning data or a feature amount to be learned.

The classifier type indicates the type of the classifier 14. Specific examples include determination by template matching and learning determination method (hereinafter referred to as SVM) using a support vector machine. (For details on the support vector machine, refer to the document "Mathematical Chemistry NO.444, JUNE 2000 Ako et al.". Refer to this document for details.)
Here, the classifier 14 to be specified may be a different type of combination such as a learning method and a matching method. The discriminator 14 may use any discriminating method as long as it is a discriminating system. Information such as a processing procedure or a program for setting the discriminator 14 is stored in the discriminator setting information storage unit 26. It shall be.

  The identification result integration function information indicates how to combine a plurality of classifiers 14. For example, when the number of classifiers used is two, the combination method of the two classifiers 14 is designated. Specifically, a method of taking the logical product of the detection results of the two discriminators 14 and setting only the subject detected from either discriminator 14 as the final discrimination result is designated. When the number of classifiers used is 3, the detection result of another classifier 14 is subtracted from the logical sum of the detection results of the two classifiers 14, and the result detected from either of the previous two classifiers 14 is The combination of the three discriminators 14 is designated, for example, the subject excluding the result detected by one discriminator 14 is used as the final detection result. Here, the combination may be a logical product, a logical sum, an exclusive logical sum, a logical difference (negation), or a function other than the logical operation of the results from the discriminators 14. Further, the results from each classifier 14 may be used as they are, or the results for each classifier 14 may be weighted and used.

  When the number of classifiers used is N, the classifier configuration variable information and the classifier types are also N, and the classification result integration function is a function that combines N classifiers 14.

  In addition, as an initial setting, a table is prepared in advance that has a feature that can detect the classifier type as SVM, the number of classifiers used as one, and classifier configuration variable information that faces the front of the subject. Thus, each item of the table may be automatically rewritten.

  Here, an example of an image group and subject information stored in the image data storage unit 21 will be described with reference to FIG.

  Assume that the image data storage unit 21 stores human images together with subject information as a group of stored images. Also, when data other than a person's face is required, such as when the classifier type is a learning type discrimination method that learns a person's face and the other two classes, images such as scenery other than the person are stored. May be. Further, it is assumed that the face information of the person to be detected is recorded together with the subject information.

  The subject information of the person recorded here includes the position of the entire face area in the image, position information of parts such as the eyes and nose and mouth, position information of feature points such as end points (hereinafter referred to as part features), etc. To record. The opening / closing state of eyes and mouth, the presence / absence of glasses, the presence / absence of partial masking, the presence / absence / shape of bangs, facial contour information, and the like may also be recorded. These may be the image itself or a feature amount extracted from the image.

  Furthermore, the amount of variation that can be handled by one common feature may be tabulated and accumulated based on the classifier type and classifier configuration variable information.

  Here, an example of classifier configuration variable information in the classifier specification table 23 will be described with reference to FIG.

  The classifier configuration variable information is set in advance, is automatically set by the classifier specification table creation unit 24, or can be set by the user. For example, a feature that does not change or a feature that changes little depending on each subject to be detected may be used. For example, when the subject is a person, site features (shading information, shape, and arrangement) that do not depend on the individual are used.

  However, some subjects have the same shape but have a certain appearance depending on the shooting angle, and others have a different appearance depending on the shooting angle.

  In the former case, the features of the entire region can be used as they are, but in the latter case, features whose appearance changes continuously are used. When expanding the detection angle, these continuously changing features are used step by step.

  When the three-dimensional object rotates, if the rotation axis is parallel to the image plane, there is a region where the amount of change in appearance is small due to the rotation. Therefore, when the image itself is used as a feature, it is possible to use an area of the subject in the image whose luminance value variation is small depending on the shooting angle.

  Specifically, when the detection target is a face image, the shape of the eyes and mouth shape of the left half surface is small in variation due to slight clockwise rotation variation. In addition, the luminance values of the eyes, mouth, and nose when the luminance value of the cheek is used as a reference are little changed. Therefore, the left half surface area of the face normalized by the position of the eyes and mouth in the face image of the image data storage unit 21 is cut out and normalized by the size and luminance distribution, and the created template has the above characteristics. Therefore, for example, when it is desired to detect a face from the front direction to 30 degrees to the right, the left side surface including the subject's eyes, nose, and mouth may be a feature.

  The size of such a face region is a feature point that can be a standard for size normalization, for example, an eye, eyebrow, nose, mouth end point (hereinafter referred to as a feature point whose size can be normalized), etc. Existence may be checked and determined so that only common feature points are included, or a variation amount that can be identified by one common feature is prepared in the image data storage unit 21 in advance as a table, and the table is referred to. You may decide by.

  Here, when there is no common feature that can cope with all the fluctuations in the range indicated in the subject identification condition, the range is divided into a plurality of parts. For example, when the range is divided into two and searching for common features in each range is repeated until a common feature is found, or when the region is -15 degrees to 30 degrees across the front (0 degrees) Is divided at 0 degrees, and is divided from -15 degrees to 0 degrees, 0 degrees to 30 degrees, and so on.

  Here, the feature points using the end points of the facial part have been described as an example. However, other common features such as determining the region based on the spatial frequency of the facial region luminance may be extracted and used.

  Here, the flow of determining classifier configuration variable information in the classifier specification table creation unit 24 will be described with reference to FIG. Note that, in order to configure the discriminator 14 using the discriminator for learning discrimination, SVM, a case where the luminance value of the face area in the image is used as a common feature as learning data will be described as an example.

  First, the flow of configuring the classifier 14 from the classifier specification table 23 will be described.

  As shown in FIG. 13, it is assumed that a value for setting a classifier 14 that can detect a general subject in advance is entered as an initial value in the classifier specification table 23. This initial value is variable and can be changed, for example, as in the initial setting according to the order shown in FIG. At the time of configuring the discriminator, the initial value or, for example, the initial setting according to the order shown in FIG. 14 is read. Here, a procedure for configuration based on the initial setting according to the order shown in FIG. 14 will be described.

  First, from the classifier specification table 23, the face direction is 30 degrees to the right from the front (Y axis is the rotation axis and rotation in the X axis direction is -30 degrees), and the classifier to be used is SVM. A condition that a face area image cut out using a part feature point is used as a feature is read (S121).

  Next, the number of features to be used is determined by paying attention to fluctuations to be handled in the read table (S122). Here, the detection target is a face from 0 degree to -30 degrees. By comparing the 0 degree face image and the −30 degree face image of the image data storage unit, the number of classifiers used, that is, the number of common features may be determined from the number of feature points (for example, in FIG. 11). In the case of the lower left figure, the number of common features is 4.) The variation amount that can be identified by one common feature is prepared in the image data storage unit 21 in advance as a table, and this is used to make the common feature. May be determined.

  Further, by referring to the classifier specification table 23, the number of necessary features is determined by setting how many classifiers 14 are combined (S123). Here, in order to combine two SVM classifiers 14 and detect only those detected from either face as a face (logical AND of the two results), the two features constituting the two classifiers 14 are: Necessary. The two are determined according to the rules in the classifier specification table 23.

  For example, by preparing two learning data sets such as images of learning data sizes 2 and 3 cut out at two different sizes from the image of learning data size 1 in FIG. The size of the region that includes the maximum number of feature points, that is, the learning data size 2, is used. The second is the size of the region that is narrowed so that the number of feature points is reduced by one, that is, learning. It is assumed that data size 3 is used.

  A learning data set is set from the image in the image data storage unit 21 with the size determined as described above. Then, the integrated discriminator setting unit 22 configures a desired discriminator 14 by causing each discriminator to learn this learning data set (S124).

  Further, the image data storage unit 21 has a discriminator use number determination table (described in FIG. 3) in which the allowable variation range that can be handled is recorded for each discriminator type and discriminator configuration variable information. When the device configuration variable is determined, the number of classifiers configured in accordance with the subject identification condition may be determined. For example, in the discriminator usage number determination table of FIG. 3, when the classifier type is SVM and the discriminator configuration variable information is determined as the learning data set size, the number of discriminators is determined, the discriminator 14 and the learning data set According to the aspect ratio, the angle range of the face orientation that can be supported by one type of learning data set is determined with reference to the table, and whether the angle range to be detected can be handled by one type of learning data set, The number of classifiers 14 to be configured is determined.

  Here, a method of setting a combination of a plurality of classifiers in the integrated classifier setting unit 22 will be described with reference to FIG.

  As a subject identification condition, when it is desired to detect a face having an angle in a range that does not include the front direction (0 degrees) (for example, a in the figure), with only one classifier 14 configured with a common feature that can detect e, A range including the front direction (0 degree) is detected. This is because when the common feature is extracted by the method of FIG. 11, the data size 2 in FIG. 15 is used at 10 degrees to the right and the data size 3 is also used at 30 degrees to the right. This is because a face that can be detected by the data size 1 is also detected.

  Therefore, from the detection result of the discriminator (a in the figure) that can detect the face from the front direction to the face that has a large absolute value of the rotation angle (a in the figure) Set a function to subtract the detection result of c) in the figure (to obtain a logical difference or negation).

  At this time, as the discriminator 14 for detecting a, the discriminator 14 may be configured with a common feature from the front direction to 30 degrees to the right, or the discriminator 14 with a common feature from the right 10 degrees to the right 30 degrees. It may be configured.

  Specifically, when the discriminator 14 is configured by common features from the front direction to 30 degrees to the right, the face area image of the face that faces 30 degrees to the right side and the size of the face area are matched to the front surface. What is necessary is just to comprise the discriminator 14 made to learn as a learning data set the image of the face area cut out from the face of the direction (equivalent to the front face cut out to the learning data size 3 in FIG. 15). Also, when the classifier is configured with common features from right 10 degrees to right 30 degrees, the face area image of 30 degrees right sideways and the size of this face area 10 degrees right sideways The classifier 14 may be configured to learn a face area image cut out from the face (corresponding to a 10 ° right-side face cut out to the learning data size 3 in FIG. 15) as a learning data set.

  Further, as a subject identification condition, when it is desired to detect a face having an angle in a range including the front direction (0 degrees) (for example, f in the figure), if a common feature that can cover the entire range is not found, the face direction is determined. The detection result of the discriminator 14 (c in the figure) that can detect a face having a negative angle is added to the detection result of the discriminator 14 (d in the figure) that can detect a face having a positive angle from the front (logic). Set the function to find the sum. (For example, it is not possible to obtain a common feature from the image of the data size 3 in FIG. 15 and the left-right reversed image of the data size 3.) In such a case, the discriminator 14 for detecting c is facing forward. Are detected from two discriminators 14, that is, a discriminator 14 configured by a common feature from 10 degrees to the right and a discriminator 14 detecting d from the front direction and the discriminator 14 configured by a common feature from the left to 15 degrees. What is necessary is just to comprise the integrated discriminator 12 which makes the whole area the face.

  Furthermore, in order to cope with not only horizontal fluctuations but also vertical fluctuations, a plurality of classifiers are similarly combined, and finally, an integrated classifier capable of detecting a face including horizontal and vertical fluctuations. 12 can be configured.

  Here, an example of setting of the classifier 14 in the integrated classifier setting unit 22 of the attention level measurement system according to the present invention will be described with reference to FIG.

  When detecting a face image, for example, a method using a support vector machine as a method for generating an identification plane in order to set the discriminator 14 for performing discrimination by generating an identification plane that separates a face and a non-face by learning. An example will be described below.

  As learning data, a face image having features determined by the discriminator specification table 23 or subject information thereof is used as learning face data, and an identification surface is generated by creating learning non-face data from other areas. A positive value greater than or equal to 0 is defined as the threshold value of the feature amount to be determined, and if the feature amount is greater than or equal to this threshold value, it is determined as a face, and if less than the threshold value, it is determined as a non-face. Increasing this threshold decreases the face detection rate (decrease in reproduction rate), but reduces false detection (improves the matching rate). Conversely, if it is made smaller, the recall is improved and the precision is lowered. Therefore, this threshold value may be a predetermined value or may be adjustable by the user depending on the purpose of use of the apparatus.

  The learning data used here uses an image stored in advance in the image data storage unit 21. This image data may be added later. At this time, face detection is performed from the image data stored in the image data storage unit 21 and the correctness of the detected area is determined, and each area is used as relearning data, and the discriminator 14 with higher detection accuracy is used. The means for configuring may be taken multiple times.

  Further, the image itself may be used for learning, or a feature amount obtained from the image may be used. For example, the learning data may be normalized such that the luminance value of the learning image has an average of 0 and a variance of 1, and then the normalized luminance value is vectorized. After performing principal component analysis on a set of face data and obtaining eigenvalues and eigenvectors, multiple eigenvectors corresponding to the eigenvalues are selected from the larger eigenvalues, and the eigenvectors corresponding to the selected eigenvalues are used as learning data as features. Also good.

  In addition, although the description in the identification using the SVM has been described here, another classifier such as another learning type classifier or another classifier such as template matching may be used as the classifier 14. In the case of template matching, one or more face templates are created instead of the learning data set. Also, templates other than faces are not created.

Here, the flow of learning when the classifier 14 is configured by using the SVM as an example in the integrated classifier setting unit 22 of the classifier setting device 20 will be described with reference to FIG.
Step 181: The discriminator 14 is configured by learning using the face and non-face images stored in the image data storage unit 21 as learning data.
Step 182: If the identification accuracy is improved by re-learning, go to step 183, otherwise end.
Step 183: The face area is detected from the image stored in the image data storage unit 21 by the discriminator 14 configured in Step 181.
Step 184: The area detected in step 183 is correctly reclassified into a face and a non-face based on the subject information stored in the image data storage unit 21, and the classified image is used as learning data.

  Here, the number of times of re-learning may be defined or variable. In addition, images used for relearning may be divided into images for learning and relearning in advance, and images different from those for learning may be used, or a plurality of the same images may be used. It may be used once.

Here, the flow of processing of the integrated discriminator 12 of the attention level measuring apparatus 10 will be described with reference to FIG. Specific steps are as follows.
Step 51: Calculate the value of the discriminant function in the other discriminator 14 in the area detected by the discriminator 14.
Step 52: Obtain the value of the discrimination result integration function based on the contribution rate of the discriminant function value of each discriminator 14 to the final discriminant value.
Step 53: If the value of the identification result integration function is equal to or greater than a predetermined threshold value, go to Step 54, and if less than the threshold value, go to Step 55.
Step 54: It is determined that the detection area is a face, and the detection result is obtained.
Step 55: It is determined that the detection area is a non-face, and a detection result is obtained.

Here, the discrimination result integration function f (x, y) obtained in step 52 is a function for obtaining a final discriminant value from the discrimination function values x and y obtained by the two discriminators 14. For example,
f (x, y) = ax + by
It is expressed as At this time,
f (x, y) ≧ T1 (T1: predetermined threshold)
If so, the face area is determined.

Here, a and b are coefficients that determine the weighting of x and y, and are values that can be adjusted by the user as identification result integration function information. This weighting is assumed to be the contribution rate of the discriminant function value of each discriminator 14 to the final discriminant value. This value may be automatically set as an identification result integration function based on the input attention level measurement condition, or may be set as an initial setting or automatically set as a default and variable according to user settings later. .
As a condition for determining the face area,
f (x, y) ≧ T1 and x ≧ T2, y ≧ T3
(T1, T2, T3: predetermined threshold values)
Thus, conditions for x and y may be added.
Further, the identification result integration function f (x, y) may be a quadratic or higher order function.

  Here, an example of how to combine two discriminators 14 when the discriminators 14 are combined in the integrated discriminator setting unit 22 in the discriminator setting device 20 will be described with reference to FIGS. The identification result integration function will be described in more detail.

  The distribution diagrams of FIGS. 19 to 22 are results of face detection by two classifiers 14 having different learning template sizes.

  In the two classifiers 14 having different learning template sizes, the image data added as the re-learning image data in the re-learning process differs because of the difference in the common feature. Therefore, by preparing data that does not include a face as relearning image data, two classifiers 14 having different non-face learning data sets in relearning can be configured. In the two classifiers 14 composed of such learning data sets, the face area is detected by either classifier 14, but the non-face area erroneously detected as the face area differs for each classifier 14. It becomes. By utilizing such a property, not only can the angle range of the face to be detected be controlled, but also false detection can be reduced as described below.

  In FIG. 19, white ◇ dots indicate the face area to be detected, white ◇ dots and black ■ are detected erroneously detected areas detected by the discriminator 14-1, and white ◇ dots and dark gray ▲ are detected by the discriminator 14-2. It is assumed that this is a false detection area. A detection result integration process is performed so that as many white dots as possible can be used as identification results in the detected area.

  In each discriminator, the minimum value (hereinafter referred to as a threshold) of the discriminant function detected as a face is set in advance. As a setting method, for example, the distribution of a sample of a face and a non-face created using a test image is examined, and an appropriate threshold value is set to a default value for classifying two types by means such as projecting on a KL expanded principal axis. As follows. For example, the x-axis and y-axis in FIG. 19 may be default values. However, the identification result integration function may be variable by the user.

  Here, the classifiers 14 to be used are the classifiers 14-1 and 14-2, and the values from the discriminant functions in the two classifiers are x and y, respectively. At this time, only the area that satisfies a certain condition among the area detected by the classifier 14-1 and the area detected by the classifier 14-2 is output as a detection area. For example, as shown in FIG. 20, a logical product may be used, and only a region having a threshold value y of 1 or more and a threshold value 2 or more of x may be used as the face.

  Further, as shown in FIG. 21, the main axis in which the region detected by the classifier 14-1 is distributed (KL developed main axis: straight line 1) and the main axis in which the region detected by the classifier 14-2 is distributed (KL). When the developed main axis: straight line 2) defines a straight line orthogonal to each threshold value and is defined as a straight line 3 and a straight line 4, an identification boundary (identification result integration function f ( x, y)) may be created (example in FIG. 21: dotted line), or an identification boundary (example in FIG. 21: broken line) may be formed by a curve having these line segments as asymptotic lines.

  Furthermore, an identification boundary (identification result integration function f (x, y)) as shown in FIG. 21 may be created, and an area showing a certain value or more may be set as a detection area.

  With the above processing, it is possible to detect a face area that has been missed when a single discriminator as shown in FIG. 22 is used, or to reduce a falsely detected area.

  Here, with reference to FIG. 23, an example of a result when a human face is detected by a combination of two discriminators 14 in the attention level measuring device 10 will be described.

  An example in which detection is performed by limiting the orientation of the face by a combination of two classifiers 14 will be described using a learning discrimination type classifier 14 based on SVM.

  The two classifiers 14 differ in robustness with respect to shielding and face orientation when the classifier configuration variables are different. When the size as a common feature used for learning data is large, it is strong against shielding, and when it is small, it is weak. Further, when the learning data is large, the tolerance for the face direction is narrow, and when the learning data is small, the tolerance is wide.

Therefore, when it is desired to detect only a face facing front with little occlusion, a classifier 14 (FIG. 23) capable of detecting only a face facing front is combined with a classifier 14 (FIG. 24) capable of detecting the face up to 30 degrees sideways. This can be realized by taking a logical product (FIG. 25).
Similarly, in the case where it is desired to take a screen whether it is shielded or sideways, a logical sum can be obtained by the same discriminator 14 (FIG. 25).

  Further, by combining three or more discriminators 14, it is possible to set more detailed detection conditions such as whether or not to detect a face shielded by bangs or overlapping of persons.

Here, an example of the flow of attention time measurement when the person tracking unit 64 of the attention degree measuring apparatus 10 according to the present invention performs person tracking will be described with reference to FIG. The specific processing flow is as follows.
Step 261: Detect a face of a person who is paying attention to the presentation at time N (Nth frame),
Step 262: If the person detected in the Nth frame is also detected in the N + 1th frame, go to Step 263, otherwise end.
Step 263: If there is no change in the direction of the face of the person who has been paying attention to the presentation in the Nth frame, go to Step 264, if there is a change, go to Step 265.
Step 264: It is determined that attention is paid to the N + 1th frame, and one frame time is added to the attention time of the person.
Step 265: It is determined that attention is not paid in the (N + 1) th frame, and it is not added to the attention time of the person.

  Here, an example of use of the attention level measuring apparatus 10 according to the present invention will be described with reference to FIG. The input image may be an image obtained by photographing a person looking at a station advertisement or a bulletin board, or an image from a camera installed for monitoring at a museum or a store.

  If you want to extract only the person in the front direction, such as when a camera is installed in the middle of the bulletin or on the display, the detection condition is 0 degrees for the face and the person looking at the poster In this case, the approximate angle of the face to be detected is set from the positional relationship between the camera and the poster.

  The classifier specification table 23 may be automatically set by inputting imaging device installation parameters in the attention degree measurement condition, or may be set in advance by the user (screen 1), and automatically set. The user may be able to make adjustments after being done.

  Further, when the measurement space seen in FIG. 8 is wide, the discriminator specification table 23 may be created individually by dividing it into a plurality of regions. For example, when the area a in FIG. 8 is divided into two and divided into left and right toward the poster, the discriminator 14 that detects a face from right 30 to right 60 degrees is set in the left area, and the front is displayed in the right area. You may make it set the discriminator 14 which detects the face of right 30 degree | times from direction. Here, information such as how many classifiers 14 are set depending on the size of the measurement space may be tabulated and recorded in the classifier specification table creation unit 24 or determined by the user. It may be input when the attention level measurement condition is input.

  As described above, the detection result regarding the person satisfying the detection condition is displayed as a thumbnail of the face image detection result as shown in the screen 2 in FIG. 27, or the attention degree calculated from each image is individually displayed together with the image as shown in the screen 3. The degree of attention is displayed as a list or a graph as shown in the screen 4 and presented to the user.

  When automatically measuring the degree of attention, after detecting all the face images, extract only the face facing the front, count the number of people,

The degree of attention may be calculated by defining a definition formula as follows.

  In addition, you can track the person you are watching over time, measure the attention time, add the attention time to the definition of attention degree, extract the information of the detected person, You may add to the attention degree which calculates age ratio etc.

  Further, the detection results of several images may be shown as samples so that details can be adjusted when setting the discriminator, and the user can process all images after confirmation. For example, the classifier specification table 23 is re-created by designating one or more images in advance so that the screen 1 can be set again for adjustment of the classifier and confirming the detection result in the image. All the images may be processed using the discriminator 14 finally determined (screen 5).

  Here, an example of the attention degree calculated by the attention degree measuring apparatus 10 according to the present invention will be described with reference to FIG.

  According to the present invention, it is assumed that the calculated attention level stores the location, date and time, the number of detected people, the number of attention people, the personal information extracted from the attention person, the attention level, and the like together with the image data. The storage may be stored in another storage unit via a network, or may be stored in a storage unit built in the attention level measuring apparatus of the present invention. These attention levels may be presented in a table corresponding to place-time, or may be presented as a graph of time-gender ratio, gender ratio, age ratio, and the like.

  Here, the attention level measurement system for manually setting the contents of the discriminator use table 23 will be described with reference to FIGS.

  FIG. 29 is a diagram showing an outline of the configuration of the attention level measurement system. The attention level measurement device 10 is the same as the attention level measurement system of FIG.

  The classifier setting device 290 includes an image data storage unit 291, a classifier specification information input unit 9294, a classifier specification table 293, and an integrated classifier setting unit 292.

  The image data storage unit 291 stores an image of a subject to be detected. The discriminator specification information input unit 294 inputs discriminator specification information that is information for setting the plurality of discriminators 14 and the discrimination result integration unit 15 in the attention level measurement device 10, and enters the input discriminator specification information. Based on this, the discriminator specification table 293 is rewritten. The integrated discriminator setting unit 292 sets the integrated discriminator 12 that identifies the target subject based on the discriminator specification table 293.

  The discriminator specification table 293 stores information necessary for setting a unified integrated discriminator 12 by combining a plurality of discriminators 14 for identifying subjects.

The image data storage unit 291 stores an image group including a subject to be detected.
The accumulated image group includes a plurality of images that continuously cover a range of variation that can be detected by the subject to be detected with respect to variation defined by a parameter representing a state that is an index having an order relationship. Shall.

  Specifically, this parameter is a captured angle or a size in the image. For example, for rotation fluctuation, an image can be taken at a certain rotation angle of the subject to be detected. It is assumed that the number of sheets that can cover the range of rotational fluctuation is accumulated.

  The discriminator specification information is information indicating how to detect a target subject from the input image. Specifically, the discriminator type, the number of discriminator use, discriminator configuration variable information, Identification result integration function information and the like. Based on the discriminator specification information, the discriminator specification table 293 is created in advance. Such discriminator specification information may be changed according to the purpose by setting an initial value in advance.

  The integrated discriminator setting unit 292 sets discriminators based on the created discriminator specification table 293 and integrates them to set the integrated discriminator 12 that detects a target subject. The discriminator 14 is set from the discriminator configuration variable information of the discriminator specification table 293 according to the discriminator type of the discriminator specification table 293. Further, the classifiers 14 are combined according to the number of uses and the identification result integration function information to form an integrated classifier 12.

Here, the flow of processing for attention level calculation of the attention level measurement apparatus 10 will be described with reference to FIG. The specific processing flow is as follows.
Step 301: Input an image to be identified into the image input unit 11,
Step 302: Create a discriminator specification table 293 based on the discriminator specification information input to the discriminator specification information input unit 294.
Step 303: The integrated discriminator setting unit 292 sets the discriminator 14 based on the set value of the discriminator specification table 293 and the image data of the image data storage unit 21.
Step 304: The integrated discriminator setting unit 292 sets the discrimination result integration unit 15 based on the set value of the discriminator specification table 293.
Step 305: Each discriminator 14 identifies a subject from the input image.
Step 306: The identification result integration unit 15 integrates the identification results of the classifiers 14,
Step 307: If the readjustment of each discriminator 14 and the discrimination result integration unit 15 is performed, go to Step 302, otherwise go to Step 308.
Step 308: The attention level calculation unit 13 calculates the attention level.

  In the example of FIG. 30, it is assumed that readjustment of the discriminator specification table 293 is performed based on the identification result (step 307), but when readjustment of the discriminator specification table 293 is not performed, step 301 and step 302 to 304 may be exchanged, and after setting of the discriminator 14 and the integrated discriminator 12 is finished, an image to be discriminated may be input.

  According to the embodiment of the present invention, by setting the desired discriminator 14 and the integrated discriminator 12 based on the discriminator specification information, the subject to be detected is “front-facing toward the camera”. Detection conditions such as “detect only a person in the area” and “detect only a subject area of a certain size or larger” can be set.

  FIG. 31 shows a schematic configuration of the discriminator setting device 290, and is a configuration explanatory diagram illustrating in more detail the configuration for setting the discriminator 14 based on the discriminator specification table 293 in the discriminator setting device 290 according to FIG.

  The integrated discriminator setting unit 292 sets the integrated discriminator 12 based on the discriminator specification table 293. Here, it is assumed that information for setting the discriminator 14 based on the discriminator type information is stored in the discriminator setting information storage unit 295. For example, when setting a discriminant function with a learning discriminator and determining whether or not it is a subject, information such as a processing procedure or a program for setting the discriminant function with the learning discriminator is the discriminator setting information storage unit 295. It is assumed that it is stored in

  Needless to say, the above-described apparatus and method of the present invention can be configured by a computer program, or the processing steps of the flowchart can be configured by a computer program to cause the computer to execute the program. A recording medium that can be read by the computer, such as a flexible disk, a CD, a DVD, an MO, a ROM, a memory card, a program for realizing the function or a program for causing the computer to execute the processing step It can be recorded on a removable disk and stored or distributed.

  It is also possible to provide the above program through a network such as the Internet or electronic mail. The present invention can be implemented by installing the program from these recording media into a computer or by downloading the program from a network and installing the program into the computer.

The figure which shows the outline of a structure of an attention degree measurement system. The block diagram of a discriminator setting apparatus. Explanatory drawing of a discriminator utilization number determination table. The attention level calculation flowchart of an attention level measuring device. The flowchart of a process of an integrated discriminator. The block diagram of an attention degree calculation part. Explanatory drawing of attention degree measurement conditions. Explanatory drawing of attention degree measurement conditions. Explanatory drawing of a discriminator specification table. Explanatory drawing of the image group and subject information accumulate | stored in an image data storage part. Explanatory drawing of the discriminator structure variable information of a discriminator specification table. The flowchart of a discriminator structure variable information determination in a discriminator specification table preparation part. Explanatory drawing of a discriminator specification table. Explanatory drawing of a discriminator specification table. Explanatory drawing of learning data size. Combination explanatory drawing of a some discriminator in an integrated discriminator setting part. Setting explanatory drawing of a discriminator in an integrated discriminator setting unit. The learning flowchart at the time of the discriminator structure in an integrated discriminator setting part. Explanatory drawing of the result of face detection by two discriminators with different learning template sizes. Explanatory drawing of the result of face detection by two discriminators with different learning template sizes. Explanatory drawing of the result of face detection by two discriminators with different learning template sizes. Explanatory drawing of the result of face detection by two discriminators with different learning template sizes. Explanatory drawing of a result at the time of detecting a person's face by the combination of two discriminators in an attention degree measuring device. Explanatory drawing of a result at the time of detecting a person's face by the combination of two discriminators in an attention degree measuring device. Explanatory drawing of a result at the time of detecting a person's face by the combination of two discriminators in an attention degree measuring device. The flowchart of attention time measurement of person tracking in the person tracking unit. Explanatory drawing of the attention level measuring device. Explanatory drawing of the attention degree calculated with the attention degree measuring device. The block diagram of an attention degree measurement system. The attention level calculation flowchart of an attention level measuring device. The block diagram of a discriminator setting apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Attention degree measuring device 11 ... Image input part 12 ... Integrated discriminator 13 ... Attention degree calculation part 14 ... Discriminator 15 ... Identification result integration part 20 ... Discriminator setting apparatus 21 ... Image data storage part 22 ... Integrated discriminator setting 23: Discriminator specification table 24 ... Discriminator specification table creation unit 25 ... Attention level measurement condition input unit 26 ... Discriminator setting information storage unit 290 ... Discriminator setting device 291 ... Image data storage unit 292 ... Integrated discriminator setting unit 293 ... Discriminator specification table 294 ... Discrimination specification information input unit 295 ... Discriminator setting information storage unit

Claims (23)

An image input means for inputting an image to be measured;
A plurality of classifiers for identifying a person's face from the input image;
An identification result integrating means for integrating the plurality of classifiers and detecting a human face facing a predetermined range;
A classifier setting device for setting the plurality of classifiers and the classification result integrating unit of the level-of-interest measurement apparatus, the level-of-interest calculation unit calculating a level of attention from the detection result,
Image data storage means for storing a plurality of images including a person's face;
Discriminator specification information input means for inputting discriminator specification information for setting the plural discriminators and the discrimination result integration means;
A discriminator specification table for storing the input discriminator specification information;
Integrated discriminator setting means for setting the plurality of discriminators and the identification result integrating means based on the discriminator specification information stored in the discriminator specification table and the image stored in the image data storage means. A discriminator setting device. An image input means for inputting an image to be measured;
A plurality of classifiers for identifying a person's face from the input image;
An identification result integrating means for integrating the plurality of classifiers and detecting a human face facing a predetermined range;
A classifier setting device for setting the plurality of classifiers and the classification result integrating unit of the level-of-interest measurement apparatus, the level-of-interest calculation unit calculating a level of attention from the detection result,
Image data storage means for storing a plurality of images including a person's face, and subject information indicating the position of the person's face in the image and a shooting state;
Attention level measurement condition input means for inputting the attention level measurement condition;
A discriminator specification table creating means for creating a discriminator specification table based on the attention degree measurement condition and the subject information stored in the image data storage means;
A discriminator specification table for storing the created discriminator specification table;
Integrated discriminator setting means for setting the plurality of discriminators and the identification result integrating means based on the discriminator specification information stored in the discriminator specification table and the image stored in the image data storage means. A discriminator setting device. The identifier specification table is:
A discriminator usage number indicating the usage number of the plurality of discriminators;
A classifier type representing each type of the plurality of classifiers to be used; and
Discriminator configuration variable information that is variable information for configuring each of the plurality of discriminators to be used;
3. The discriminator setting according to claim 1, further comprising: discriminating result integration function information that is information of a function that integrates the plurality of discriminators to be used, which is used in the discrimination result integration unit. apparatus. The discriminator setting device is
The subject identifier setting device according to claim 3, further comprising a identifier setting information storage unit that stores information for configuring the identifier based on the identifier type. The plurality of discriminators are:
Each is a learning discriminator that performs discrimination by learning learning data based on a person's face image, or learning data based on both a person's face image and an image other than the person's face,
The identifier configuration variable information is
Learning data based on an image of a person's face corresponding to each of the plurality of classifiers, or learning data based on one or more images that are images other than the face of a person and are different from each other for each classifier Information about
The integrated discriminator setting means includes
The discriminator setting device according to any one of claims 1 to 4, wherein learning data corresponding to the discriminator configuration variable information is read from the image data storage means and is made to learn. An image input means for inputting an image to be measured;
A plurality of classifiers for identifying a person's face from the input image;
Identification result integrating means for integrating the identification results of the plurality of discriminators and detecting a face of a person facing a predetermined direction;
An attention level measurement device comprising: attention level calculation means for calculating a level of attention from the detection result. The plurality of discriminators are:
Each is a learning discriminator that performs discrimination by learning learning data based on a person's face image, or learning data based on both a person's face image and an image other than the person's face,
Each is configured by learning learning data based on an image of a person's face, or learning data based on one or more images that are images other than a person's face and include different images for each of the classifiers. The degree-of-interest measurement device according to claim 6. The plurality of discriminators are:
The image processing apparatus according to claim 1, wherein each is an image of a person's face, and learning data based on one or more images including images having different shooting angles for each classifier is configured. 8. The attention level measuring device according to 7. The plurality of discriminators are:
Learning data based on one or more images, each including learning data based on an image of a person's face, or an image other than a person's face, each image having a different image size for each classifier The degree-of-interest measurement device according to claim 7 or 8, wherein   The degree-of-interest measurement apparatus according to claim 6, wherein the plurality of classifiers and the identification result integration unit are set by the classifier setting apparatus according to any one of claims 1 to 5. The attention level calculating means includes:
A face detection unit that extracts a feature of the person from the input image with respect to a face person detected as being focused by the identification result integration unit;
Tracking image storage means for storing the extraction result and the input image in association with each other;
A person tracking means for extracting a person in the accumulated input image and tracking the movement of the extracted person;
The degree-of-interest measurement apparatus according to claim 6, further comprising attention degree calculation means for calculating the degree of attention based on the person tracking result. An image input means for inputting an image to be measured;
A plurality of classifiers identifying a person's face from the input image;
An identification result integrating unit integrating the plurality of classifiers to detect a face of a person facing a predetermined range;
A classifier setting method for setting the plurality of classifiers and the classification result integrating unit in a method of measuring the degree of attention comprising: a step of calculating a degree of attention from the detection result;
An image data storage means for storing a plurality of images including a person's face;
A step of inputting discriminator specification information by which the discriminator specification information input means sets the plurality of discriminators and the discrimination result integration means;
A discriminator specification table storing the inputted discriminator specification information;
A step of setting the plurality of discriminators and the discrimination result integrating unit based on discriminator specification information stored in the discriminator specification table and an image stored in the image data storage unit; A classifier setting method characterized by comprising: An image input means for inputting an image to be measured;
A plurality of classifiers identifying a person's face from the input image;
An identification result integrating unit integrating the plurality of classifiers to detect a face of a person facing a predetermined range;
A classifier setting method for setting the plurality of classifiers and the classification result integrating unit in a method of measuring the degree of attention comprising: a step of calculating a degree of attention from the detection result;
A step in which image data storage means stores a plurality of images including a person's face, and subject information indicating a position of the person's face and a shooting state in the image;
Attention level measurement condition input means for inputting the attention level measurement condition;
A discriminator specification table creating means creating a discriminator specification table based on the attention level measurement condition and subject information stored in the image data storage means;
A discriminator specification table storing the created discriminator specification table;
A step of setting the plurality of discriminators and the discrimination result integrating unit based on discriminator specification information stored in the discriminator specification table and an image stored in the image data storage unit; A classifier setting method characterized by comprising: The identifier specification table is:
A discriminator usage number indicating the usage number of the plurality of discriminators;
A classifier type representing each type of the plurality of classifiers to be used; and
Discriminator configuration variable information that is variable information for configuring each of the plurality to be used;
14. The discriminator setting according to claim 12, further comprising: discriminating result integration function information that is information of a function that integrates the plurality of discriminators to be used, which is used in the discrimination result integration unit. Method. In the classifier setting method,
15. The discriminator setting information storage means stores information for configuring the discriminator based on the discriminator type, and provides the information to the integrated discriminator setting means. The subject identifier setting method described. The plurality of discriminators are:
Each is a learning discriminator that performs discrimination by learning learning data based on a person's face image, or learning data based on both a person's face image and an image other than the person's face,
The identifier configuration variable information is
Learning data based on an image of a person's face corresponding to each of the plurality of classifiers, or learning data based on one or more images that are images other than the face of a person and are different from each other for each classifier Information about
The integrated discriminator setting means includes
The discriminator setting method according to any one of claims 12 to 15, wherein learning data corresponding to the discriminator configuration variable information is read out from the image data storage means and is learned by the discriminator. An image input means for inputting an image to be measured;
A plurality of discriminators identifying a human face from the input image;
An identification result integrating means for integrating the identification results of the plurality of classifiers to detect a face of a person facing a predetermined direction;
An attention level measuring means comprising: a step of calculating an attention level from the detection result. The plurality of discriminators are:
Each is a learning discriminator that performs discrimination by learning learning data based on a person's face image, or learning data based on both a person's face image and an image other than the person's face,
Each is configured by learning learning data based on an image of a person's face, or learning data based on one or more images that are images other than a person's face and include different images for each of the classifiers. The degree-of-interest measurement method according to claim 17. The plurality of discriminators are:
The image processing apparatus according to claim 1, wherein each is an image of a person's face, and learning data based on one or more images including images having different shooting angles for each classifier is configured. The attention level measuring method according to 18. The plurality of discriminators are:
Learning data based on one or more images, each including learning data based on an image of a person's face, or an image other than a person's face, each image having a different image size for each classifier The degree-of-interest measurement method according to claim 18 or 19, characterized by comprising:   The attention classifying method according to any one of claims 17 to 20, wherein the plurality of classifiers and the identification result integrating unit are set by the classifier setting method according to any one of claims 12 to 16. The attention level calculating means includes:
Extracting a feature of the person from the input image with respect to a person whose face is detected as being noticed by the identification result integrating means by the face detection means;
Tracking image storage means stores the extraction result and the input image in association with each other; and
A person tracking means extracting a person in the accumulated input image and tracking the movement of the extracted person;
The attention level measuring method according to any one of claims 17 to 21, further comprising a step of calculating the attention level based on the person tracking result. The discriminator setting device, the attention level measuring device, the discriminator setting method, or the attention level measuring method according to any one of claims 1 to 22 is described in a computer program and can be executed. A program characterized by

Images