JP5067224B2 - Object detection apparatus, object detection method, object detection program, and printing apparatus - Google Patents

Description

  The present invention relates to an object detection apparatus, an object detection method, an object detection program, and a printing apparatus.

A technique for detecting a target image (object) from an input image is known.
The skin color pixel is determined from the image data, the determined skin color pixel value is set to 255, and the other pixel values are set to 0. The projection distribution of the detected skin color pixel is obtained, and the parabola is searched by the genetic algorithm from the shape of the projection distribution. A face area extraction method is known that extracts face area candidates from the searched parabola positions and determines whether the extracted face area candidates are face areas (see Patent Document 1).
JP 2000-48184 A

When an object such as a face image is to be detected from an input image, it is required to reduce the processing required for the detection and increase the processing speed as well as the accuracy of the detection. However, in the above-mentioned document 1, the calculation of the skin color pixel projection distribution, the search for a parabola based on the shape of the projection distribution, the extraction of a face area candidate from the position of the parabola, the determination of whether the face area candidate is a face area, etc. Therefore, it is necessary to take a lot of processing, and it is quite insufficient in terms of reduction in processing and speeding up.
In addition, after the detection of the object, correction processing or the like may be performed on the input image, but if it is possible to acquire information useful for the correction processing in the process until the detection of the object, The burden of the correction process is greatly reduced, which is very suitable.

  The present invention has been made in view of the above-described problems. When detecting an object from an input image, while ensuring high-accuracy detection, the processing is reduced and speeded up as compared with the conventional method, and the object is detected. An object of the present invention is to provide an object detection apparatus, an object detection method, an object detection program, and a printing apparatus that also contribute to reducing the burden of predetermined processing after being performed.

  In order to achieve the above object, the present invention provides an object detection apparatus for detecting a predetermined object from an input image, wherein an image area other than an image area belonging to a color gamut corresponding to the object among image areas in the input image is provided. A conversion unit that replaces the pixel value with a predetermined representative value, a detection window is set on the input image on which the replacement has been performed, and a variation in the pixel value within the detection window is obtained, and the variation is greater than or equal to a predetermined value In this case, a determination unit that determines the presence or absence of the object for the image in the detection window is used. According to the present invention, before setting the detection window on the input image, the pixel value of the image area other than the image area belonging to the color gamut corresponding to the object is replaced with the representative value. For this reason, when the detection window is set in a region where many pixels having undergone the replacement are gathered, the variation is likely to be a small value (a value smaller than a predetermined value). As a result, the area where many pixels are replaced, that is, the area where the object is estimated not to exist, is excluded from the object presence / absence determination target and processed without degrading the detection accuracy of the object from the input image. The amount and processing time can be reduced.

  The conversion unit replaces the pixel value of each image region having a different hue other than the image region belonging to the color gamut corresponding to the object with each representative value corresponding to the representative color for each image region. It is good. According to this configuration, the variation calculated when the detection window is set in a region where a large number of pixels that have undergone the replacement are gathered tends to be small, while maintaining the effect that each region of the input image Color information can be acquired. Such color information of each region is used as useful information in predetermined processing after object detection, for example, correction processing according to the scene of the input image.

  The conversion unit may analyze the input image and change a color gamut corresponding to the object based on the analysis result. According to this configuration, the image area to be replaced can be narrowed or widened according to the contents of the input image. More specifically, when the conversion unit generates a histogram of pixel values of the input image and determines that the input image is a color cast image or a backlight image based on the analysis result of the histogram, the conversion unit As the color gamut corresponding to, a second color gamut may be selected from the first color gamut and the second color gamut wider than the first color gamut. According to this configuration, when the input image is a so-called color cast image or a backlight image, the image area where the replacement is performed is narrowed. Therefore, even if the input image is a color cast image or a backlight image, the object detection accuracy is ensured.

  The determination unit repeatedly sets the detection window on the input image while changing at least one of the position and size of the detection window in the input image, and the variation obtained by setting the detection window is less than a predetermined value. The detection window may be set afterwards while avoiding a certain area. According to this configuration, the area that is excluded from the object presence / absence determination target once the detection window is set is subsequently excluded from the detection window setting target. Therefore, even when attempting to detect an object precisely for the entire input image, the processing amount and processing time are effectively reduced.

  The determination unit may determine the presence / absence of an object by using a neural network that inputs information relating to an image in the detection window and outputs information indicating the presence / absence of the object. According to this configuration, the presence / absence of an object can be accurately determined by using a neural network.

  The technical idea of the present invention is that, in addition to the above-described invention of the object detection device, the invention of the object detection method including each processing step performed by each unit included in the above-described object detection device, and each unit included in the above-described object detection device It can also be understood as an invention of an object detection program for causing a computer to execute a function corresponding to the above. A printing apparatus for detecting a predetermined object from an input image and executing printing based on the input image, wherein an image area other than an image area belonging to a color gamut corresponding to the object among image areas in the input image A conversion unit that replaces the pixel value with a predetermined representative value, a detection window is set on the input image on which the replacement has been performed, and a variation in the pixel value within the detection window is obtained, and the variation is greater than or equal to a predetermined value The determination unit that determines the presence or absence of the object for the image in the detection window, and the correction information determined based on the image in the detection window that is determined by the determination unit to have an object. And a print control unit that corrects at least part of the input image and performs printing based on the corrected input image. Rukoto is possible.

Embodiments of the present invention will be described in the following order.
1. General printer configuration:
2. Processing by printer:
2-1. Preprocessing:
2-2. Determining whether or not an object exists:
2-3. From object presence determination to printing:
3. Variations:

1. General printer configuration:
FIG. 1 schematically shows a configuration of a printer 10 corresponding to an example of an object detection apparatus and a printing apparatus of the present invention. The printer 10 is a color inkjet printer that supports so-called direct printing, in which an image is printed based on image data acquired from a recording medium (for example, a memory card MC). The printer 10 includes a CPU 11 that controls each unit of the printer 10, an internal memory 12 configured by, for example, a ROM and a RAM, an operation unit 14 configured by buttons and a touch panel, a display unit 15 configured by a liquid crystal display, A printer engine 16, a card interface (card I / F) 17, and an I / F unit 13 for exchanging information with an external device such as a PC, a server, or a digital still camera are provided. Each component of the printer 10 is connected to each other via a bus.

  The printer engine 16 is a printing mechanism that performs printing based on print data. The card I / F 17 is an I / F for exchanging data with the memory card MC inserted into the card slot 172. Image data is stored in the memory card MC, and the printer 10 can acquire the image data stored in the memory card MC via the card I / F 17. In addition to the memory card MC, various media can be used as recording media for providing image data. Of course, in addition to the recording medium, the printer 10 can also input image data from the external device connected via the I / F unit 13. The printer 10 may be a printing device for consumers, or may be a business printing device for DPE (so-called minilab machine). The operation unit 14 and the display unit 15 may be an input operation unit (such as a mouse or a keyboard) or a display separate from the main body of the printer 10. The printer 10 can also input print data from a PC or server connected via the I / F unit 13.

  The internal memory 12 stores an object detection unit 20, an image correction unit 30, a display processing unit 40, and a print processing unit 50. The object detection unit 20 and the image correction unit 30 are computer programs for executing pre-processing, object detection processing, image correction processing, and the like described below under a predetermined operating system. The display processing unit 40 is a display driver that controls the display unit 15 to display a processing menu and a message on the display unit 15. The print processing unit 50 is a computer program for generating print data from image data, controlling the printer engine 16 and printing an image based on the print data. The CPU 11 implements the functions of these units by reading and executing these programs from the internal memory 12.

  The object detection unit 20 includes a conversion unit 21, a detection window setting unit 22, a necessity determination unit 23, and a detection execution unit 24 as program modules. The image correction unit 30 includes a correction information determination unit 31 and a correction execution unit 32 as program modules. The detection window setting unit 22, the necessity determination unit 23, and the detection execution unit 24 correspond to a determination unit referred to in the claims. The image correction unit 30 and the print processing unit 50 correspond to a print control unit in the claims. The functions of these units will be described later. Further, the internal memory 12 stores color gamut definition information 14b, various data such as a neural network NN, and programs. The printer 10 may be a so-called multifunction machine having various functions such as a copy function and a scanner function in addition to the print function.

2. Processing by printer:
2-1. Preprocessing:
FIG. 2 is a flowchart showing processing executed by the printer 10 in this embodiment. In step S (hereinafter, step notation is omitted) 100, the object detection unit 20 obtains image data D representing an image (input image) to be subjected to image processing from a predetermined recording medium such as a memory card MC. get. That is, the object detection unit 20 acquires an input image. Of course, if the printer 10 has a hard disk drive (HDD), the object detection unit 20 can acquire the image data D stored in the HDD, and the I / F unit 13 can be used as described above. The image data D can be acquired from the external device connected via the network. That is, the user operates the operation unit 14 while referring to the user interface (UI) screen displayed on the display unit 15 to arbitrarily select the image data D as the input image and print the selected image data D. When the instruction is given, the object detection unit 20 acquires the image data D relating to the selection from a recording medium or the like.

  The image data D is bitmap data composed of a plurality of pixels, and each pixel is expressed by a combination of gradations of RGB channels (for example, 256 gradations of 0 to 255). The image data D may be compressed when recorded on a recording medium or the like, or the color of each pixel may be expressed in another color space. In these cases, the object detection unit 20 executes the development of the image data D and the conversion of the color space to acquire the image data D as RGB bitmap data.

  In S200, the object detection unit 20 reduces the image data D. When preprocessing and object detection processing described later are performed on the image data D with the original image size as a target, the processing load is large. Therefore, the object detection unit 20 reduces the image size by reducing the number of pixels of the image data D, and acquires the reduced image data. For example, the object detection unit 20 acquires image data DR obtained by reducing the image data D to a QVGA (Quarter Video Graphics Array) size (320 pixels × 240 pixels). In the present embodiment, the image data DR is also referred to as an input image as appropriate.

  In S300, the conversion unit 21 of the object detection unit 20 replaces the pixel value of the image region other than the image region belonging to the color gamut corresponding to the predetermined object in the image region of the image data DR with a predetermined representative value. In the present embodiment, description will be made assuming that the object is a human face image. A color gamut corresponding to a human face image means a skin color gamut. However, objects that can be detected using the configuration of the present invention are not limited to human face images, and various objects such as artifacts, living things, natural objects, and landscapes can be detected as objects. .

FIG. 3 is a flowchart showing details of S300. In S310, the conversion unit 21 selects one pixel among the pixels constituting the image data DR.
In S320, the conversion unit 21 determines whether or not the color of the pixel selected in the latest S310 belongs to the skin color gamut by referring to the color gamut definition information 14b stored in the internal memory 12. The color gamut definition information 14b is information defining a skin color gamut in a predetermined color system. The color system that defines the skin color gamut is the L * a * b * color system specified by the International Commission on Illumination (CIE) (hereinafter “*” is omitted) or the XYZ color system specified by the CIE. Various color systems such as a color system, an RGB color system, and an HSV color system can be adopted.

  FIG. 4 shows an example of the skin color gamut A1 defined by the color gamut definition information 14b. In FIG. 4, a circular skin color gamut A1 is shown in the ab plane of the Lab color system. For example, the color gamut definition information 14b defines the skin color gamut A1 by the position (center position) P1 of the area and the radius r of the area. However, the shape of the skin color gamut A1 does not have to be a circle, and may be a solid. The color gamut definition information 14b may be information that defines a range of a color gamut that seems to be a skin color in a certain color system. In S320, the conversion unit 21 converts the RGB data of the pixel selected in the latest S310 into data (Lab data) of the color system (Lab color system) in which the color gamut definition information 14b defines the skin color gamut A1. If the converted Lab data (a value and b value of the Lab data) belong to the skin color gamut A1, the process skips S330 and proceeds to S340, while the converted Lab data does not belong to the skin color gamut A1. The process proceeds to S330. Conversion from RGB data to Lab data can be performed by using a predetermined color conversion profile that performs conversion from the RGB color system to the Lab color system. Such a profile may be stored in the internal memory 12. In this embodiment, the pixel determined to belong to the skin color gamut in S320 is referred to as a skin color pixel, and the pixel determined to not belong to the skin color in S320 is referred to as a non-skin color pixel.

  In S330, the conversion unit 21 replaces (converts) the pixel value (RGB data) of the pixel determined as the non-skin color pixel in the latest S320 with a predetermined representative value (representative color). As the representative value, for example, a value representing black (R = G = B = 0) is applicable. In other words, the conversion unit 21 fills a region composed of non-skin color pixels with a representative color. The representative value may be one kind as described above, but in the present embodiment, there are a plurality of representative values. In FIG. 4, as an example, in the ab plane, a color gamut other than the skin color gamut A1, which is a color gamut (green gamut) A2 that seems to be green, a color gamut (sky gamut) A3 that seems to be a sky (mainly blue sky) color, and magenta. A special color gamut (magenta color gamut) A4 is shown. The central positions of the green gamut A2, the sky gamut A3, and the magenta gamut A4 are the colors of the representative values P2, P3, and P4 corresponding to the gamuts A2, A3, and A4. In other words, the color gamut definition information 14b includes the ranges of the green color gamut A2, the sky color gamut A3, and the magenta color gamut A4, and the representative values P2, P3, and P4 for each color gamut A2, A3, and A4 (Lab data and RGB data). Etc. are also defined in advance. Of course, the types and number of representative values defined in advance in the present embodiment are not limited to those shown in FIG.

  In S330, when the color of the non-skin color pixel belongs to the green region A2 in the ab plane, the conversion unit 21 converts the pixel value (RGB data) of the non-skin color pixel into the RGB data as the representative value P2 of the green region A2. (For example, R = 0, G = 255, B = 0.) Similarly, when the color of the non-skin color pixel belongs to the sky gamut A3, the conversion unit 21 replaces the pixel value with RGB data as the representative value P3 of the sky color gamut A3, and the color of the non-skin color pixel changes to the magenta gamut. If it belongs to A4, the pixel value is replaced with RGB data as the representative value P4 of the magenta color gamut A4. If the color of the non-skin color pixel does not belong to any of the skin color gamut and each color gamut corresponding to each representative value defined as described above, another predetermined representative value (for example, a value representing black) RGB Replace that pixel value with the data. As described above, in this embodiment, a plurality of representative colors (representative values) having different hues are set in advance, and the conversion unit 21 sets the pixel value of a non-skin color pixel to any representative value that is close to the pixel color. Perform replacement work.

  In S340, the conversion unit 21 determines whether or not all the pixels constituting the image data DR have been selected once in S310. If unselected pixels remain, the conversion unit 21 returns to S310 and newly selects a pixel. If one is selected, but no unselected pixel remains, the process of S300 ends. As a result, in the image data DR, the colors of the image regions having different hues in the image regions other than the image region belonging to the skin color gamut are filled with the corresponding representative colors. When the conversion unit 21 replaces the pixel value of the non-skin color pixel with the representative value, the conversion unit 21 accumulates the number of replaced pixels for each type of representative value (in the above example, representative values P2, P3, P4, etc.). The cumulative number is recorded in a predetermined area of the internal memory 12. The conversion unit 21 also accumulates the number of skin color pixels (the number of times “Yes” is determined in S320), and records the accumulated number in a predetermined area of the internal memory 12.

  In S400 (FIG. 2), the conversion unit 21 converts the image data DR processed in S300 into a gray image. That is, the conversion unit 21 converts the RGB data of each pixel of the image data DR into a luminance value Y (0 to 255), and generates image data DR as a monochrome image having one luminance value Y for each pixel. The luminance value Y can generally be obtained by adding R, G, and B with a predetermined weight. In the present embodiment, S200 to S400 are referred to as preprocessing. However, S200 (reduction of image data D) is not essential in the preprocessing. Therefore, when not executing S200, the conversion unit 21 executes S300 and S400 on the image data D, and further executes S500 and S600 described later. Further, S400 (conversion of image data DR or image data D into a gray image) is performed in advance in consideration of the convenience of object detection processing described later. However, such S400 is not essential in the preprocessing and is skipped. May be.

2-2. Determining whether or not an object exists:
In S500, the object detection unit 20 executes an object detection process. Schematically, the object detection unit 20 sets a detection window SW in the preprocessed image data DR (or preprocessed image data D; the same applies hereinafter), and the detection window SW within the detection window SW is set. The process of obtaining the pixel value variation and determining the presence or absence of an object (face image) for the image in the detection window SW when the variation is equal to or greater than a predetermined value is repeated for each detection window SW.

FIG. 5 is a flowchart showing details of S500.
In S510, the detection window setting unit 22 of the object detection unit 20 sets one detection window SW in the preprocessed image data DR. The setting method of the detection window SW is not particularly limited, but the detection window setting unit 22 sets the detection window SW as follows as an example.
FIG. 6 shows how the detection window SW is set in the image data DR. In the first S510, the detection window setting unit 22 is a rectangular detection window SW (two-dot chain line) having a predetermined size including a plurality of pixels at the head position in the image (for example, the upper left corner of the image). Set. The detection window setting unit 22 moves the detection window SW from the position where the detection window SW has been set up to a predetermined distance (a predetermined number of pixels) in the horizontal direction and / or the vertical direction of the image every time S510 after the second time. And one new detection window SW is set at the position of the movement destination. When the detection window setting unit 22 repeatedly sets the detection window SW while moving the detection window SW to the final position of the image data DR (for example, the lower right corner position of the image) while maintaining the size of the detection window SW. Returning to the head position, the detection window SW is set.

  When the detection window SW is returned to the head position, the detection window setting unit 22 sets the detection window SW having a smaller rectangular size than before. Thereafter, in the same manner as described above, the detection window setting unit 22 sets the detection window SW at each position while moving the detection window SW to the final position of the image data DR while maintaining the size of the detection window SW. The detection window setting unit 22 repeats such movement and setting of the detection window SW while stepwise reducing the size of the detection window SW by a predetermined number of times. However, in the present embodiment, in the process of moving the detection window SW, not all areas of the image data DR are set as detection window SW setting targets, but the detection window SW may be set avoiding some areas. . In this manner, every time one detection window SW is set in S510, the processing after S520 is performed.

ｎは、検出窓ＳＷ内の画素数である。Ｙ_iは、ｎ個の画素中のｉ番目の画素の輝度値Ｙであり、ｍは、Ｙ_iの平均値である。 In S520, the necessity determination unit 23 calculates the dispersion of the pixel values in the detection window SW set for the image data DR after the preprocessing in the latest S510. The pixel value for which the variation is calculated may be RGB of each pixel, but in the present embodiment, since the conversion into a gray image is basically performed in the preprocessing, the variation in the luminance value Y of each pixel is calculated. . The variation calculated by the necessity determination unit 23 corresponds to, for example, the width of the contrast in the detection window SW (the difference between the maximum luminance value and the minimum luminance value in the detection window SW). Alternatively, the necessity determining unit 23 may obtain the variance σ ² of the luminance value Y in the detection window SW and the standard deviation σ of the luminance value Y, and the variance σ ² and the standard deviation σ may be used as variations. The variance σ ² can be obtained by the following equation (1).

n is the number of pixels in the detection window SW. Y _i is the luminance value Y of the i-th pixel among n pixels, and m is the average value of Y _i .

  FIG. 7A shows a partial area of the preprocessed image data DR, which is an area composed of non-skin color pixels replaced with a common representative value. FIG. 7B shows a partial area of the preprocessed image data DR, which is an area composed of skin color pixels. 7A and 7B show the luminance value Y of each pixel. As is clear from FIG. 7, there is no change in the luminance value Y within the area painted with one representative value color, and the variation in the luminance value Y is zero. On the other hand, in the region where the skin color pixels gather, the luminance value Y of each pixel is rich in change, and the variation of the luminance value Y is also somewhat large. Therefore, it can be determined that the larger the variation value in the detection window SW, the more skin color pixels are included in the detection window SW.

Therefore, in S530, the necessity determination unit 23 compares the variation (for example, variance σ ² ) calculated in the most recent S520 with a predetermined threshold Th. If the variation is equal to or greater than the threshold value Th, the process proceeds to S540. That is, if the variation is equal to or greater than the threshold Th, it is necessary to determine the presence or absence of a face image because there are many skin color pixels in the detection window SW (the detection window SW may include a face image). Therefore, the necessity determination unit 23 proceeds to S540. On the other hand, if the variation is less than the threshold value Th, the necessity determination unit 23 skips S540 and S550 and proceeds to S560. That is, if the variation is less than the threshold value Th, it can be said that there is no need to determine the presence or absence of a face image because there is very little skin color pixels in the detection window SW or even if they exist. The necessity determining unit 23 proceeds to S560. In the present embodiment, for example, for each of the plurality of sample images that are smaller than the luminance value variance σ ² calculated for each of the plurality of sample face images and are filled with the representative value (one or more representative values). A value larger than the variance σ ² of the calculated luminance value is obtained in advance by experiment, and the obtained value is recorded in the internal memory 12 or the like as the threshold Th, and the necessity determining unit 23 determines the threshold Th. I am trying to use it. Alternatively, the threshold value Th may be given to the printer 10 by the user operating the operation unit 14.

2-3. From object presence determination to printing:
In S540, the detection execution unit 24 determines the presence / absence of a face image (face determination) for the image in the detection window SW set in the latest S510. If it is determined that a face image exists, the process proceeds to S550. If it is determined that no face image exists, S550 is skipped and the process proceeds to S560. In S540, the detection execution unit 24 can employ any method as long as it can determine whether or not a face image exists. In this embodiment, for example, the detection execution unit 24 performs determination using the neural network NN.

  FIG. 8 is a flowchart showing details of S540 executed by the detection execution unit 24. In S541, the detection execution unit 24 acquires image data (window image data) XD composed of pixels in the detection window SW set in the latest S510, and in S542, calculates a plurality of feature amounts from the window image data XD. To do. For these feature amounts, various filters are applied to the window image data XD, and feature amounts (average value, maximum value, minimum value, and the like) indicating image characteristics such as luminance average, edge amount, and contrast in the filter are applied. (Standard deviation, etc.).

  FIG. 9 shows how the feature amount is calculated from the window image data XD. In the same figure, a number of filters FT having different relative sizes and positions with respect to the image data XD are prepared, and each filter FT is sequentially applied to the window image data XD, and image characteristics in each filter FT are obtained. Based on this, a plurality of feature amounts CA, CA, CA... Are calculated. If the feature amounts CA, CA, CA... Can be calculated, the detection execution unit 24 inputs the feature amounts CA, CA, CA... Into the prepared neural network NN in S543, and a face image exists as an output thereof. A determination result of whether or not is calculated is calculated.

  FIG. 10 shows an example of the structure of the neural network NN. The neural network NN has a basic structure in which the value of the unit U in the subsequent layer is determined by a linear combination of the values of the unit U in the previous layer (the suffix i is the identification number of the unit U in the previous layer). Further, the value obtained by the linear combination may be used as the value of the unit U of the next layer as it is, but the value obtained by the linear combination is converted by a non-linear function such as a hyperbolic tangent function, for example. By determining the value of U, non-linear characteristics may be provided. The neural network NN is composed of an outermost input layer and output layer, and an intermediate layer sandwiched between them. Each feature quantity CA, CA, CA... Can be input to the input layer of the neural network NN, and an output value K (value normalized to 0 to 1) can be output from the output layer. Yes. In S544, for example, if the output value K of the neural network NN is 0.5 or more, the detection execution unit 24 determines that the value indicates that a face image exists in the window image data XD, and the process proceeds to S550. On the other hand, if the output value K is less than 0.5, the detection execution unit 24 determines that the face image data does not exist in the window image data XD, and proceeds to S560.

  FIG. 11 schematically shows how the neural network NN is constructed by learning. In the present embodiment, by learning the neural network NN by the error back propagation method, the number of units U, the size of the weight w at the time of linear combination between the units U, and the bias b are determined. The value is optimized. In learning by the back propagation method, first, the magnitude of the weight w and the value of the bias b at the time of linear combination between the units U are initially set to appropriate values. Then, for learning image data for which it is known whether or not a face image exists, feature amounts CA, CA, CA... Are calculated in the same procedure as S542 and S543, and the feature amounts CA, CA, CA. Input to the set neural network NN and obtain the output value K. In this embodiment, it is desirable that 1 is output as the output value K for learning image data in which a face image exists, and 0 is output as the output value K for learning image data in which no face image exists. It is desirable to be done. However, since the weight w and the value of the bias b at the time of linear combination between the units U are merely set to appropriate values, there is an error between the actual output value K and the ideal value. Will occur. The weight w and the bias b for each unit U that minimizes such an error are calculated using a numerical optimization method such as a gradient method. The error as described above is propagated from the subsequent layer to the previous layer, and the weight w and the bias b are sequentially optimized for the subsequent unit U.

A face image exists in the window image data XD by preparing in advance in the internal memory 12 a neural network NN that has been optimized by performing such learning using a plurality of learning image data. It can be determined based on the feature quantities CA, CA, CA.
In S550 (FIG. 5), the detection execution unit 24 determines the position of the detection window SW (for example, the center position of the detection window SW on the image data DR) determined to have a face image in the latest S540 and the detection window SW. Are recorded in a predetermined area of the internal memory 12. Thus, the act of recording the position and size of the detection window SW corresponds to an example of the face image detection act.

In S560, the detection window setting unit 22 still sets the detection window SW by moving the detection window SW and further reducing its size under the concept of the detection window SW setting method described with reference to FIG. If there is room to do so, the process returns to S510, and one detection window SW is newly set on the image data DR. On the other hand, when the reduction of the detection window SW is repeated by the predetermined number of times and all the possible detection window SW settings are completed, the object detection unit 20 ends the process of S500.
In the present embodiment, one detection window SW is set (S510), and when it is determined that the variation is less than the threshold Th with respect to the detection window SW (“No” in S530), the detection window SW is detected. It is assumed that the area occupied by the window SW in the image data DR is not set as the detection window SW setting target in the subsequent S510. In other words, once the area is determined not to require face determination, the detection window SW is set to avoid the subsequent areas, thereby reducing the number of detection window SW settings as much as possible to speed up the processing. ing.

  In S600 (FIG. 2), the correction information determination unit 31 of the image correction unit 30 determines correction information used for correction of the input image. Examples of correction for an input image include brightness correction, contrast correction, saturation correction, and correction for a specific memory color. In the present embodiment, when a face image is detected in S500, the correction information determination unit 31 determines correction information based on at least the face image. Specifically, when the position and size information of the detection window SW detected as a face image is recorded in the internal memory 12, the correction information determination unit 31 uses the detection window SW from the image data DR. The image data in the range indicated by the position and size information (referred to as face image data) is extracted. The image data DR from which face image data is to be extracted may be pre-processed image data DR, or image data DR immediately after S200 before replacement with a representative value for a non-skin color pixel. . The correction information determination unit 31 calculates correction information (correction parameters) based on the face image data. For example, the correction information determination unit 31 calculates an average value of luminance in the face image data, calculates a difference between the average value and a predetermined target value, and uses the calculated difference as correction information.

  In S700, the correction execution unit 32 corrects at least a part of the image data D before the preprocessing is performed based on the correction information determined in S600. For example, if the correction information is the difference between the average value of the luminance in the face image data and the target value as described above, the luminance corresponding to the difference is set in an area (image corresponding to the face image data on the image data D). The position and the size with respect to the data D are added to each pixel in the region where the relationship between the position and the size of the face image data with respect to the image data DR is equal. As a result, the brightness of the face image on the image data D can be improved. Further, the curve degree of the tone curve may be determined based on the magnitude of the difference, and each pixel value of the image data D may be corrected using the tone curve. Of course, the type of correction information determined in S600 and the type of correction performed in S700 are not limited to those described above.

  Further, in S700, in addition to the above-described correction, or when a face image is not detected from the input image, correction according to the scene of the input image may be performed alone as follows. As described above, the conversion unit 21 records the number of pixels (cumulative number) for each representative value type in the internal memory 12 for the pixel replaced with the representative value, and also records the cumulative number of flesh color pixels in the internal memory 12. is doing. Therefore, the correction execution unit 32 determines the scene of the input image based on the cumulative number, and performs correction according to the determined scene. For example, when the number of pixels replaced with a representative value of a certain color gamut exceeds a certain ratio (for example, 50%) with respect to the total number of pixels of the image data DR, the color tendency of the one color gamut is It is determined that the scene is strong, and correction corresponding to the determined scene is performed. For example, when the number of pixels replaced with the representative value of the sky color gamut exceeds the certain ratio, the correction execution unit 32 regards the input image as an image obtained by capturing a scene including a lot of blue sky and the sea, Color correction is performed on the image data D so as to increase the degree of color development of blue sky and sea blue. The correction execution unit 32 increases or decreases the degree of correction based on the correction information calculated from the face image data according to the ratio between the cumulative number of skin color pixels and the cumulative number for each representative value (skin color pixel). If the cumulative number of the images is small, the degree of correction based on the correction information may be weakened).

In this way, scene determination at the time of image correction can be performed based on the information obtained in the process until the object is detected. This eliminates the need for the image correction unit 30 to separately generate information necessary for scene determination, and speeds up the image correction process.
In S800, the print processing unit 50 controls the printer engine 16 to print the input image. That is, the print processing unit 50 performs necessary processes such as resolution conversion process, color conversion process, and halftone process on the corrected image data D to generate print data. The generated print data is supplied from the print processing unit 50 to the printer engine 16, and the printer engine 16 executes printing based on the print data. Thereby, the printing of the input image is completed.

  As described above, according to the present embodiment, as a pre-process of the process of setting the detection window SW for the input image and determining the presence / absence of an object (face image) for each detection window SW, the skin color gamut for the pixels of the input image In the case of non-skin color pixels, the pixel value is replaced with a representative value so that an image area other than the image area belonging to the skin color gamut is filled with the representative value. Then, the detection window SW is set on the preprocessed input image, the magnitude of the variation of the pixel value in the detection window SW is evaluated, and when the variation is less than the threshold value Th, the detection window SW With respect to SW, the detection window SW is newly set at another position on the input image without performing face determination. That is to say, of the locations on the input image, locations where there is no need to determine the presence or absence of an object are excluded from the subject of the determination. Therefore, it is possible to greatly reduce the overall amount of processing for repeating the setting of the detection window SW and the determination of the presence / absence of an object in the input image without degrading the object detection accuracy. As a result, the object detection processing is very fast. It becomes.

3. Variations:
In the present embodiment, the skin color gamut defined by the color gamut definition information 14b is not limited to one. The color gamut definition information 14b may be information in which a plurality of types of skin color gamuts having different sizes are defined in advance. The conversion unit 21 may change the skin color gamut used for the determination in S320 (FIG. 3) according to the characteristics of the input image. In this case, the conversion unit 21 analyzes the input image at a predetermined timing. For example, the conversion unit 21 generates a frequency distribution (histogram) for each of R, G, and B of the image data DR at a timing between S200 and S300 (FIG. 2). Then, feature values, for example, average values (may be median or maximum distribution value) Rave, Gave, Bave are calculated in the respective R, G, B histograms, and the relative values between these average values Rave, Gave, Bave are calculated. It is determined whether or not the input image is a so-called color cast image based on the specific positional relationship.

  12A, 12B, and 12C show examples of histograms for R, G, and B in the image data DR, respectively. For example, the conversion unit 21 may calculate | Rave-Gave | and | Rave-Bave | out of the differences | Rave-Gave |, | Rave-Bave |, | Bave-Gave | between the average values Rave, Gave, and Bave. If Bave−Gave | is greater than a certain value by a certain difference, and Rave> Gave and Rave> Bave, it can be determined that the input image is in a so-called red or orange fog (a type of color fog). The color conversion unit 21 also determines whether the input image is a so-called backlight image by analyzing the input image. For example, the converter 21 determines whether or not the image is a backlight image by generating a luminance histogram of the image data DR and analyzing the shape of the luminance histogram. For example, in a luminance histogram, there are two distribution peaks divided into a predetermined luminance range on the low luminance side and a predetermined luminance range on the high luminance side, and the number of pixels constituting each of the two peaks is a predetermined reference. When the number is exceeded, it is determined that the image is a backlight image. Alternatively, in the backlight image, the central portion of the image is often dark, so the conversion unit 21 samples pixels from a predetermined central region on the image data DR, and the reference pixel has an average luminance of the sampled pixels. When the value is lower than the value, the input image may be determined to be a backlight image. The method for determining whether or not the input image is a color cast image and the method for determining whether or not the input image is a backlight image are not limited to the methods described above.

  FIG. 13 illustrates two skin color gamuts A1S (first color gamut) and skin color gamut A1L (second color gamut) defined by the color gamut definition information 14b in the ab plane of the Lab color system. The range of the skin color gamut A1S is the same as the skin color gamut A1 shown in FIG. On the other hand, the range of the skin color gamut A1L includes the entire skin color gamut A1S, includes the lightness axis (gray axis) in the Lab color system, and further, the + side (red direction) of the a axis and the b axis of the skin color gamut A1S. It is the range expanded to the + side (yellow direction). If the conversion unit 21 determines that the input image is a color cast image or a backlight image, in S320, the conversion unit 21 selects the skin color gamut A1L from the skin color gamuts A1S and A1L defined by the color gamut definition information 14b, and the skin color gamut A1L. The pixel to which the color belongs is determined to be a skin color pixel. As described above, when the input image is a color cast image or a backlight image, the skin color gamut is automatically changed to a large range, thereby reducing the number of pixels replaced with the representative value in S330 and performing face determination in S500. The number of detection windows SW that are detected increases (the number of detection windows SW that are determined as “Yes” in S530 increases).

  As a result, in the input image, the face is expressed in an orange-like color than the general skin color (the face part is in a red fog (or orange fog) state) or lower than the general skin color. Even when the face is represented by the color of saturation (the face portion is backlit), the face image is accurately detected. Note that the printer 10 may determine which of the skin color gamuts A1S and A1L is used in the process of S320 according to an instruction input by the user operating the operation unit 14. In addition, when the user sets processing processing (high-speed printing mode) giving priority to speed to the printer 10, the printer 10 may select the skin color gamut A1S. Note that the types of skin color gamuts defined by the color gamut definition information 14b are not limited to the two types as described above. For example, when the input image is a color cast image, the skin color gamut to be selected is a backlight image. The skin color gamut to be selected may be defined separately.

A method other than the method using the neural network NN, which is the face determination performed by the detection execution unit 24 in S540, will be described.
FIG. 14 schematically illustrates an example of a face determination technique performed by the detection execution unit 24. In the example shown in FIG. 14, determination means in which a plurality of determination devices J, J. The determination means including the plurality of determination devices J mentioned here may be a substantial device or a program having the following determination functions corresponding to the plurality of determination devices J. Each of the determiners J, J... Inputs one or more feature amounts CA, CA, CA,... Of different types (for example, different filters) from the window image data XD subjected to face determination. Outputs positive or negative judgment. Each of the determiners J, J... Has a determination algorithm such as a feature size comparison between CA, CA, CA... And a threshold determination, and the window image data XD is a face (positive) or not a face ( Execute the original determination of NO). Each of the next stage determiners J, J... Is connected to the positive output of the previous stage determiners J, J..., And the output of the previous stage determiners J, J. Only when this is the case, the next stage decision devices J, J... Face determination is terminated at the point in time when no is output in any stage, and a determination that a face image does not exist is output (“No” in S540). On the other hand, when all of the determination devices J, J... At each stage output a positive value, the face determination is ended and a determination that a face image exists is output (“Yes” in S540).

  FIG. 15 shows the determination characteristics of the determination means. This figure shows a feature space defined by the axes of the feature values CA, CA, CA,... Used in each of the above-described determiners J, J ..., and finally determines that a face image exists. Coordinates in the feature amount space represented by a combination of feature amounts CA, CA, CA... Obtained from the window image data XD. Since the window image data XD determined that the face image exists has a certain feature, it can be considered that a distribution is seen in a certain region in the feature amount space. Each of the determiners J, J... Generates a boundary plane in such a feature amount space, and among the spaces partitioned by the boundary plane, the determination target feature amounts CA, CA, CA. If the coordinate of exists, positive is output. Therefore, by connecting the determination devices J, J... In a cascade, it is possible to gradually narrow down the space in which the positive output is made. According to the plurality of boundary planes, it is possible to accurately determine the distribution having a complicated shape.

  In the above, an example in which the object detection device and the object detection method of the present invention are embodied by the printer 10 has been described. For example, the object detection device and the object detection method may be a computer, a digital still camera, a scanner, or the like. It may be realized in the image equipment. Further, the present invention can be realized not only in a printer that outputs image processing results on a printing sheet but also in an apparatus that outputs image processing results on a display such as a photo viewer. Furthermore, the present invention can also be applied to an ATM (Automated Teller Machine) that performs person authentication. Furthermore, the face determination performed by the detection execution unit 24 can use various determination methods in the feature amount space of the feature amount described above. For example, a support vector machine may be used.

FIG. 2 is a block diagram illustrating a schematic configuration of a printer. 4 is a flowchart illustrating processing executed by a printer. It is a flowchart which shows the detail of the replacement process to a representative value. It is a figure which shows the skin color gamut etc. which color gamut definition information defines. It is a flowchart which shows the detail of an object detection process. It is a figure which shows a mode that a detection window is set. It is a figure which shows the one part area | region on the image data after a pre-processing. It is a flowchart which shows the face determination using a neural network. It is a figure which shows a mode that a feature-value is calculated from window image data. It is a figure which shows an example of the structure of a neural network. It is a figure which shows typically a mode that a neural network is learned. It is a figure which shows the histogram for every RGB. It is a figure which shows the several skin color gamut which color gamut definition information defines in a modification. It is a figure which shows typically the face determination process concerning a modification. It is a figure which shows the determination characteristic of the face determination process concerning a modification.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Printer, 11 ... CPU, 12 ... Internal memory, 14b ... Color gamut definition information, 16 ... Printer engine, 17 ... Card I / F, 20 ... Object detection part, 21 ... Conversion part, 22 ... Detection window setting part, 23 ... Necessity determination unit, 24 ... Detection execution unit, 30 ... Image correction unit, 31 ... Correction information determination unit, 32 ... Correction execution unit, 50 ... Print processing unit, 172 ... Card slot

Claims (9)

An object detection device for detecting a predetermined object from an input image,
A conversion unit that replaces pixel values of an image area other than an image area belonging to a color gamut corresponding to the object in an image area of the input image with a predetermined representative value;
A detection window is set on the input image on which the replacement has been performed, and a variation in the pixel value in the detection window is obtained. When the variation is a predetermined value or more, the image in the detection window is used as a target. An object detection apparatus comprising: a determination unit that determines presence or absence of an object.   The conversion unit replaces the pixel value of each image region having a different hue other than the image region belonging to the color gamut corresponding to the object with each representative value corresponding to the representative color for each image region. The object detection apparatus according to claim 1.   3. The object detection according to claim 1, wherein the conversion unit analyzes the input image and changes a color gamut corresponding to the object based on the analysis result. apparatus.   When the conversion unit generates a histogram of pixel values of the input image and determines that the input image is a color cast image or a backlight image based on the analysis result of the histogram, the color gamut corresponding to the object The object detection apparatus according to claim 3, wherein a second color gamut is selected from the first color gamut and a second color gamut wider than the first color gamut.   The determination unit repeatedly sets the detection window on the input image while changing at least one of the position and size of the detection window in the input image, and the variation obtained by setting the detection window is less than a predetermined value. The object detection apparatus according to claim 1, wherein the detection window is set for the subsequent area while avoiding the area.   The said determination part determines the presence or absence of an object by utilizing the neural network which inputs the information concerning the image in a detection window, and outputs the information which shows the presence or absence of the said object. Item 6. The object detection device according to any one of Items 5 to 6. An object detection method for detecting a predetermined object from an input image,
A conversion step of replacing pixel values of an image area other than an image area belonging to a color gamut corresponding to the object in an image area of the input image with a predetermined representative value;
A detection window is set on the input image on which the replacement has been performed, and a variation in the pixel value in the detection window is obtained. When the variation is a predetermined value or more, the image in the detection window is used as a target. And a determination step of determining the presence or absence of an object. An object detection program for causing a computer to execute processing for detecting a predetermined object from an input image,
A conversion function for replacing pixel values of image areas other than the image area belonging to the color gamut corresponding to the object among the image areas in the input image with predetermined representative values;
A detection window is set on the input image on which the replacement has been performed, and a variation in the pixel value in the detection window is obtained. When the variation is a predetermined value or more, the image in the detection window is used as a target. An object detection program for executing a determination function for determining the presence or absence of an object. A printing apparatus that detects a predetermined object from an input image and executes printing based on the input image,
A conversion unit that replaces pixel values of an image area other than an image area belonging to a color gamut corresponding to the object in an image area of the input image with a predetermined representative value;
A detection window is set on the input image on which the replacement has been performed, and a variation in the pixel value in the detection window is obtained. When the variation is a predetermined value or more, the image in the detection window is used as a target. A determination unit for determining the presence or absence of an object;
Printing in which at least a part of the input image is corrected according to correction information determined based on an image in the detection window determined that the object is present by the determination unit, and printing is performed based on the corrected input image A printing apparatus comprising: a control unit.

Images