JP2007264887A - Human region detection apparatus, human region detection method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique to specify candidate regions for human regions independently of the pixel color of an image. <P>SOLUTION: A human region detection apparatus derives the edge strength of each pixel of a target image (Step S21) and sets each group of adjacent pixels whose edge strength belongs to the same level in the target image as a contour region belonging to the level (Step S23). One contour region whose edge strength belongs to the highest level is set as an initial target region, and contour regions near the target region belonging to levels different from the highest level are combined with the target region in descending order of edge strength level (Step S26). The resultant target region is a human candidate region. The human candidate regions is specified without using the pixel color of target images but based on the edge strength, so that the human candidate regions can be specified independently of the pixel color of target images. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a technique for detecting a person region indicating a person from an image.

  In recent years, with the spread of digital cameras and computers, digital images have been handled in a wide range of fields. Accordingly, there is a demand for a technique for detecting a person area indicating a person from a digital image to be handled, and various techniques have been proposed.

  For example, Patent Document 1 discloses a technique of detecting a skin color pixel region in a target image and detecting a person region based on the shape of the skin color region. Further, Patent Document 2 detects a skin color pixel area in a target image, and determines whether a person has an element (such as an eye or a mouth) that is a facial feature in the skin color area. A technique for detecting a region is disclosed.

JP 2000-48184 A JP 2004-5384 A

  Any of the conventionally proposed human area detection methods is premised on the detection of a skin color pixel area. However, the skin color of a person varies from person to person, and changes depending on the illumination light that is emitted. Therefore, the skin color in the image cannot be uniquely defined. For this reason, it is necessary to ensure a comparatively wide allowable range as the skin color determination condition, and as a result, many areas other than a person are detected.

  In addition, when targeting grayscale images, images with greatly changed hues, etc., even if it is a human area, it will not become flesh-colored, so it is impossible to detect a human area by detecting a flesh-colored area. It is.

  That is, since the conventional technique employs a method that depends on the color of the pixel of the image, the person area or the candidate area cannot always be correctly specified.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a technique that can specify a region that is a candidate for a human region without depending on the color of a pixel of an image.

  In order to solve the above-mentioned problem, the invention of claim 1 is a person area detecting device for detecting a person area indicating a person from a target image, the means for deriving edge strength in each pixel of the target image, and the edge Means for classifying intensities into a plurality of levels, and setting the adjacent pixel groups belonging to the same level in the target image as contour regions belonging to the level, and the level with the maximum edge strength One of the contour regions belonging to the initial region of interest, and the contour region belonging to a level different from the maximum level and close to the region of interest is combined with the region of interest and part of the region of interest And combining means for combining the contour regions to the region of interest in the order of the contour regions belonging to a level having a large edge strength. Said region of interest is a candidate to become a person candidate region of the human region.

  The invention according to claim 2 further includes means for expanding the contour area before combining by the combining means in the human region detecting apparatus according to claim 1.

  Further, the invention of claim 3 is the person area detecting device according to claim 1 or 2, further comprising complementing means for complementing a free area existing inside the person candidate area obtained by the combining means. ing.

  According to a fourth aspect of the present invention, in the human region detection device according to the third aspect, in the human candidate region after complementing the vacant region, each of pixel groups adjacent to each other whose edge strength is equal to or greater than a predetermined threshold value, Eye candidate area extracting means for extracting as eye candidate areas that are candidates for areas indicating human eyes is further provided.

  According to a fifth aspect of the present invention, in the human region detecting device according to the fourth aspect, the eye candidate region extracting means is configured such that the edge strength is equal to or greater than a predetermined threshold value and a size is a predetermined condition among adjacent pixel groups Are extracted as the eye candidate regions.

  Further, the invention of claim 6 is the person area detection device according to claim 4 or 5, wherein means for excluding the person candidate area from which a predetermined number or more of the eye candidate areas are not extracted from the person area candidates, Is further provided.

  The invention according to claim 7 is the person area detection device according to any one of claims 4 to 6, based on a comparison result of the sizes of any two of the eye candidate areas extracted from the person candidate area. And determining means for determining whether the person candidate area is the person area.

  Further, the invention of claim 8 is the person region detection device according to any one of claims 4 to 7, based on a mutual arrangement relationship between any two of the eye candidate regions extracted from the person candidate region. And determining means for determining whether the person candidate area is the person area.

  The invention of claim 9 is a person area detection method for detecting a person area indicating a person from a target image, wherein (a) deriving edge strength at each pixel of the target image; Dividing the edge strength into a plurality of levels, and setting adjacent pixel groups belonging to the same level in the target image as the contour regions belonging to the level, and (c) the edge strength One of the contour regions belonging to the maximum level is set as the initial attention region, and the contour region belonging to a level different from the maximum level and close to the attention region is combined with the attention region to generate the attention. And in the step (c), combining the contour regions to the region of interest is performed in the order of the contour regions belonging to a level having a large edge strength. Said region of interest is a candidate to become a person candidate region of the human region.

  The invention of claim 10 is a program for detecting a person region indicating a person from a target image, wherein the computer includes (a) a step of deriving an edge strength at each pixel of the target image; ) Dividing the edge strength into a plurality of levels, and setting pixel groups belonging to the same level and adjacent to each other in the target image as contour regions belonging to the level, and (c) the One contour region belonging to a level having the maximum edge strength is set as an initial attention region, and the contour region belonging to a level different from the maximum level and close to the attention region is combined with the attention region. And making the part of the region of interest a part of the region of interest, and in the step (c), the edge strength belongs to a level where the edge strength belongs to the region of interest. Performed in order of the high region, the region of interest obtained as a candidate to become a person candidate region of the human region.

  According to the first to tenth aspects, since the person candidate area is specified based on the edge strength of the target image, the person candidate area can be specified without depending on the color of the pixel of the target image.

  In particular, according to the invention of claim 2, it is possible to relax the criteria for proximity determination when combining.

  In particular, according to the invention of claim 3, it is possible to acquire a person candidate area that is more like a person by filling the empty area.

  In particular, according to the invention of claim 4, the eye candidate region can be easily extracted based on the edge strength.

  In particular, according to the invention of claim 5, the eye candidate region can be extracted more accurately.

  Further, according to the invention of claim 6, in order to exclude a person candidate area that does not include a predetermined number or more of eye candidate areas from the candidate for the person area, only the more correct person candidate area can be set as the candidate for the person area. it can.

  In particular, according to the invention of claim 7, the person area can be accurately specified based on the comparison result of the sizes of the eye candidate areas.

  In particular, according to the invention of claim 8, the person area can be accurately specified based on the mutual arrangement relationship of the eye candidate areas.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<1. Device configuration>
FIG. 1 is an external view of a human region detection apparatus 10 according to an embodiment of the present invention. The person area detecting apparatus 10 includes an area (hereinafter referred to as “person area”) representing an image of a person (typically, a human face image) from a target digital image (hereinafter referred to as “target image”). ")"). As shown in the figure, the human region detection device 10 is a general computer as a device configuration, and includes a main body 1 as a computer, a display 2 for displaying various information, and a keyboard 3 for receiving various inputs from a user. And a mouse 4.

  FIG. 2 is a block diagram schematically showing the internal configuration of the main body 1. The main unit 1 includes a CPU 21, a ROM 22, a RAM 23, a hard disk 25 that is a storage device for storing various information, a reading device 26 that reads a recording medium 91, and a communication unit 27 that performs communication via a network 92 such as the Internet. Each of which is connected by a bus line 29. The display 2, the keyboard 3 and the mouse 4 are connected to the bus line 29 through an interface (I / F). With this configuration, each unit of the person area detection device 10 operates under the control of the CPU 21.

  A program 41 is stored in the hard disk 25. The program 41 is stored in the hard disk 25 by reading from the recording medium 91 or downloading from an external server device connected to the network 92. When the CPU 21 executes the program 41 in cooperation with the RAM 23 serving as a memory, various functions as a person area detecting device are realized.

  In the drawing, main functions realized by the program 41 being executed by the CPU 21 are schematically shown as a person candidate extraction unit 31, a region complementation unit 32, an eye candidate extraction unit 33, and a person determination unit 34.

<2. Processing>
Next, processing of the person area detection device 10 will be described. In the following, first, the overall process for detecting a person region from a target image will be schematically described, and then the details of each of the sub-processes that are steps of the overall process will be described.

<2-1. Overview of overall processing>
FIG. 3 is a diagram showing an overall flow of processing for detecting a person area.

  First, a person candidate area extraction process, which is a sub-process, is performed (step S11). In this person candidate area extraction processing, a person candidate area that is a candidate for a person area is extracted from the target image. Usually, in this processing, a plurality of person candidate regions are extracted.

  Next, among the extracted person candidate areas, one person candidate area is determined as a “target person candidate area” to be processed (step S12). Subsequently, for the target person candidate area, three sub-processes, specifically, an empty area complement process (step S13), an eye candidate area extraction process (step S14), and a person determination process (step S15) are performed in this order. Done in

  In the empty area complementing process (step S13), the pixels of the target person candidate area are complemented, and the empty areas existing in the target person candidate area are complemented. Further, in the eye candidate area extraction process (step S14), eye candidate areas that are candidates for areas indicating the eyes of the person are extracted from the target person candidate areas after the empty areas are complemented. Further, in the person determination process (step S15), it is determined whether or not the target person candidate area is a person area based on the extracted eye candidate area. That is, the person area is detected by such three sub-processes.

  When the process is performed on one target person candidate area, it is determined whether an unprocessed person candidate area exists (step S16). If it exists (Yes in step S16), the next target person candidate area is determined (step S12), and the same three sub-processes as described above are repeated for this target person candidate area. Such a process is repeated, and finally three sub-processes are performed for all the person candidate areas, and it is determined whether or not the area is a person area. As a result, all the human regions in the target image are detected.

<2-2. Person candidate area extraction processing>
Next, details of the person candidate region extraction process (FIG. 3: step S11) will be described. FIG. 4 is a diagram illustrating a detailed flow of the person candidate area extraction process. Unless otherwise stated, all the steps of the person candidate area extraction process are performed by the person candidate extraction unit 31 shown in FIG.

  First, the edge strength at each pixel of the target image is derived (step S21). Specifically, a convolution process using the Laplacian filter 5 shown in FIG. 5 is performed on each pixel of the target image, and an edge image indicating the edge strength of each pixel is generated. For example, when the image illustrated in the upper part of FIG. 6 is the target image 61, the edge image 62 shown in the lower part of FIG. 6 is obtained by applying the Laplacian filter 5. The number of pixels of the edge image 62 is the same as that of the target image.

  Each pixel of the edge image 62 indicates the edge intensity at each pixel of the corresponding target image. In the convolution process using the Laplacian filter 5, the larger the difference in luminance between the central pixel and the peripheral pixel (that is, the higher the edge strength), the larger the resulting value. For this reason, the pixel value of the pixel of the edge image 62 becomes larger as the edge intensity of the corresponding pixel of the target image 61 is larger.

  Next, based on the generated edge image 62, mask data which is data indicating the position of a pixel belonging to the same level of edge intensity in the target image 61 is generated (FIG. 4: step S22).

  In the present embodiment, the edge strength is divided into three levels of level 0, level 1 and level 2. The level with the maximum edge strength is level 0, and thereafter the edge strength decreases in the order of level 1 and level 2. Specifically, the edge strength is level 0 when the threshold value is Th1 or more, and is level 1 when the threshold value Th1 is less than the threshold value Th2 (Th2 <Th1) or more, and is level 2 when the threshold value Th2 is less than the threshold value Th2 (0 <Th3 <Th2). Each is divided.

  As shown in FIG. 7, mask data 63a to 63c are generated for each of these levels. These mask data are binary image data indicated by the value of each pixel being “0” or “1”, and the number of pixels is the same as that of the target image 61 and the edge image 62.

  In the edge image 62, the pixel value of the mask data 63a at the level 0 corresponding to the pixel whose pixel value (that is, edge strength) belongs to the level 0 is “1”, and the values of the other pixels of the mask data 63a are “1”. 0 ". Similarly, the pixel value of the level 1 mask data 63b corresponding to the pixel whose pixel value belongs to level 1 in the edge image 62 is “1”, and the values of the other pixels are “0”. Further, the pixel value of the level 2 mask data 63c corresponding to the pixel whose pixel value belongs to level 2 in the edge image 62 is “1”, and the values of the other pixels are “0”.

  Therefore, the mask data 63a of level 0 indicates the position of a pixel whose edge intensity belongs to level 0 (a pixel having a relatively large edge intensity) in the target image 61. Similarly, level 1 mask data 63b indicates the position of a pixel whose edge strength belongs to level 1 in the target image 61, and level 2 mask data 63c indicates a pixel whose edge strength belongs to level 2 in the target image 61 (edge strength is 0). The position of a relatively small pixel) is indicated.

  Next, based on the generated mask data, pixel groups having edge strengths belonging to the same level and adjacent to each other are set as contour regions belonging to the level (FIG. 4: step S23).

  Specifically, as shown in FIG. 7, mask data 63 a to 63 c are used, and images 64 a to 64 c each composed of only the pixel of the target image 61 corresponding to the pixel whose value is “1” in the mask data. Generated for each level. That is, these images 64a to 64c are images (hereinafter referred to as “contour images”) configured only from pixels belonging to the same level of edge intensity in the target image 61.

  In each of the contour images 64a to 64c belonging to each level, a pixel group forming one region adjacent in the horizontal direction or the vertical direction is set as one contour region belonging to the level and stored in the RAM 23. Remembered. The contour region is set as belonging to the level of the contour image in which it is included. For example, a contour area included in the level 0 contour image 64 a is set as a contour area belonging to level 0.

  One or a plurality of contour regions are set from one contour image. For example, in the level 0 contour image 64a illustrated in FIG. 7, the region A1 near the eye, the region A2 near the ear, the region A3 near the nose, the region A4 near the mouth, and the like belong to level 0. Set as high region. Further, in the level 1 contour image 64b illustrated in FIG. 7, the region A5 corresponding to the skin is set as the contour region belonging to the level 1.

  Returning to FIG. 4, next, one contour area is determined as the initial attention area from the contour areas of level 0 (step S24). Next, level 1 is set as the target level to be processed (step S25), and the contour area belonging to this target level (level 1) is changed into the attention area (contour area belonging to level 0) under a predetermined condition. ) Is performed (step S26).

  FIG. 8 is a diagram showing a detailed flow of the contour region combining process (FIG. 4: step S26). First, among the contour regions belonging to the target level, one contour region is determined as the “target contour region” to be processed (step S31).

  Next, it is determined whether or not the target contour area is close to the attention area (step S32). This proximity determination is performed based on whether or not at least some of the pixels are adjacent in the horizontal direction or the vertical direction.

  For example, the region illustrated in FIG. 9 is the attention region TA1, and the region illustrated in FIG. 10 is the target contour area EA1. As shown in FIG. 11, when the pixels of these two areas TA1 and EA1 are adjacent to each other in the target image, it is determined that the target contour area EA1 is close to the attention area TA1. On the other hand, as shown in FIG. 12, when none of the pixels in these two areas TA1 and EA1 are adjacent (separated) in the target image, the target contour area EA1 is close to the attention area TA1. It is determined that it is not. Since the adjacent pixels are determined in the horizontal direction or the vertical direction, the pixels are not adjacent even in the state of FIG.

  As a result of such determination, when the target contour area is close to the attention area (FIG. 8: Yes in step S32), the target contour area is set as a combination target to be combined with the attention area (step S33). On the other hand, when the target contour area is not close to the attention area (No in step S32), the target contour area is excluded without being set as a combination target.

  When the proximity determination is made for one target contour area in this way, it is determined whether there is an unprocessed contour area belonging to the target level (step S34). If it exists (Yes in step S34), the next target contour area is determined (step S31), and the proximity determination similar to the above is repeated for this target contour area. Such a process is repeated, and finally the proximity determination is made for all the contour regions belonging to the target level, and the contour regions that are close are set as the combination target.

  Next, all the contour areas set as the combination target are combined with the attention area to be a part of the attention area. That is, the entire area composed of the original attention area and all the contour areas set as the combination target is set as a new attention area. For example, in the example of FIG. 11, the original attention area TA1 and the contour area EA1 are combined to form a new attention area TA2 (step S35).

  In the proximity determination in step S32, for the pixels forming the periphery of the target contour area, a pixel is added to both the horizontal direction and the vertical direction for a predetermined pixel (for example, 1 to 2 pixels). The proximity determination may be performed after expanding. According to this, since the target contour region is expanded, the criteria for proximity determination can be relaxed according to the expanded pixels, and it can be determined that the regions separated by the influence of noise or the like are close to each other. . For example, even in the state shown in FIG. 12, it is determined that the target contour area EA1 is close to the attention area TA1. In addition, when such an extension is performed, an overlap may occur in some pixels, but it may be determined that the overlap is also close.

  Returning to FIG. When the contour region combination processing (step S26) is completed as described above, it is next determined whether or not there is a lower level whose edge strength is lower than the target level at that time (step S27). ). If a lower level exists, a level (step S28) one level lower than the current target level is set as a new target level. For example, if the target level at that time is level 1, level 2 is set as a new target level. Then, the contour area combination process for combining the contour area belonging to the new target level with the attention area is performed again (step S26).

  As described above, in the present embodiment, the edge strength is divided into three levels of level 0, level 1, and level 2. Therefore, after the above-described processing, after the contour-level region combining processing in which level 1 is set as the target level. Then, the contour region combination processing with level 2 as the target level is performed. Accordingly, the level 1 contour region is first combined with the initial level 0 attention region, and the level 2 contour region is further combined with the resultant attention region.

  Assuming that the edge strength is divided into four or more levels as another embodiment, even in this case, each level is set to the target level in order from the level with the highest edge strength with respect to the levels other than level 0 by the above processing. The contour region combining process is sequentially performed. That is, the contour regions are combined with the attention region in the order of contour regions belonging to a level with a large edge strength.

  When the contour region combination processing is performed for all levels other than level 0, the attention region obtained as a result at that time is set as a human candidate region. That is, the result obtained by combining the contour regions of each level with the initial attention region of level 0 in the descending order of the edge strength is the person candidate region (step S29).

  In general, in an image showing a person's face, the eye region has the highest edge strength. From this, it is highly possible that the level 0 area is the eye area. Therefore, as shown in FIG. 13, if the level 0 area is the attention area TA, and the lower level contour areas EA adjacent to the attention area TA are combined in descending order of edge strength, The resulting area NA is likely to be an area representing a person. In the present embodiment, the person candidate area is extracted based on this principle.

  Returning to FIG. 4, when a candidate human area is extracted with the level 0 one contour area as the attention area as described above, there is an unprocessed contour area that belongs to level 0 and is not regarded as the attention area. It is determined whether or not (step S30). If it exists (Yes in step S30), the next one contour area is determined as the initial attention area (step S24), and the lower level adjacent to the attention area in the same manner as described above. The contour regions of the level are combined, and the person candidate region is extracted. Such processing is repeated, and finally, the candidate area is extracted by combining the lower level contour areas with respect to all the contour areas belonging to level 0.

<2-3. Free space complement processing>
Next, details of the free space complementing process (FIG. 3: step S13) will be described. In the human candidate area extracted by the above-described human candidate area extraction processing, as shown in the lower part of FIG. 13, empty areas (pixels) that are not recognized as a part of the closed area NA in the algorithm. In many cases, there is a missing region (Sa) in a state in which is missing. In the empty area complementing process, such empty areas are complemented with respect to the target person candidate area, and a target person candidate area having no empty area is generated.

  FIG. 14 is a diagram showing a detailed flow of the empty area complementing process. Unless otherwise stated, all the steps of the empty area complementing process are performed by the area complementing unit 32 shown in FIG.

  First, mask data indicating the position where the target person candidate area exists in the target image is generated (step S41). This mask data is image data in which the number of pixels is the same as that of the target image, and the value of each pixel is represented by a binary value of “0” or “1”. In the mask data, the value of the pixel corresponding to the pixel included in the target person candidate area in the target image is “1”, and the values of the other pixels are “0”. For example, mask data MD0 illustrated in the upper part of FIG. 15 is generated from the candidate person area NA illustrated in the lower part of FIG. In FIG. 15, pixels whose value is “1” in the mask data are indicated by hatching.

  In this process, as shown in FIG. 15, an XY coordinate system is set for the mask data, the horizontal direction is the X-axis direction, and the vertical direction is the Y-axis direction. The origin of this XY coordinate system is the upper left end of the mask data data, the right side is the + X side, and the lower side is the + Y side. Each pixel of the mask data data is indicated by P (x, y) using the coordinate position (x, y) of the XY coordinate system. For example, the upper left pixel is indicated by P (1,1), the right pixel thereof is indicated by P (2,1), and the lower pixel thereof is indicated by P (1,2).

  When the mask data is generated, each horizontal pixel column of the mask data is then scanned in the + X direction, and the following equation (1) is calculated for each pixel (step S42).

P (x, y) = P (x-1, y) OR P (x, y) (1)
That is, a process is performed for each pixel by newly setting the logical sum of the value of the pixel and the value of the pixel adjacent to the left side to the value of the pixel. The pixel to be calculated is changed in the order from the left end pixel to the right end pixel. As a result, if there is a pixel having a value “1” in the horizontal pixel row, the values of the pixels on the right side of the pixel are all “1”. This process is performed for all the horizontal pixel columns of the mask data, and as a result, first correction data is generated. For example, from the mask data MD0 illustrated in the upper part of FIG. 15, first correction data MD1 illustrated in the lower part is generated.

  Next, contrary to step S42, each horizontal pixel column of the mask data is scanned in the -X direction, and the following equation (2) is calculated for each pixel (step S43).

P (x, y) = P (x, y) OR P (x + 1, y) (2)
In other words, a process is performed for each pixel by newly setting the logical sum of the pixel value and the value of the pixel adjacent to the right side to the value of the pixel. The pixel to be calculated is changed in the order from the right end pixel to the left end pixel. As a result, if there is a pixel having a value of “1” in the horizontal pixel column, the values of the pixels on the left side of the pixel are all “1”. This process is performed for all the horizontal pixel columns of the mask data, and as a result, second correction data is generated. For example, from the mask data MD0 illustrated in the upper part of FIG. 15, second correction data MD2 illustrated in the lower part is generated.

  Next, each vertical pixel column of this mask data is scanned in the + Y direction, and the following equation (3) is calculated for each pixel (step S44).

P (x, y) = P (x, y-1) OR P (x, y) (3)
In other words, a process is performed for each pixel by newly setting the logical sum of the pixel value and the value of the adjacent pixel above the pixel value. The pixel to be calculated is changed in the order from the upper end pixel to the lower end pixel. As a result, if there is a pixel having a value “1” in the vertical pixel column, the values of the pixels below the pixel are all “1”. This processing is performed for all the vertical pixel columns of the mask data, and as a result, third correction data is generated. For example, from the mask data MD0 illustrated in the upper part of FIG. 15, third correction data MD3 illustrated in the lower part is generated.

  Next, contrary to step S44, each vertical pixel column of this mask data is scanned in the -Y direction, and the following equation (4) is calculated for each pixel (step S45).

P (x, y) = P (x, y) OR P (x, y + 1) (4)
In other words, a process is performed for each pixel by newly setting the logical sum of the pixel value and the value of the adjacent pixel below the pixel value. The calculation target pixel is changed in the order from the lower end pixel toward the upper end pixel. As a result, if there is a pixel having a value “1” in the vertical pixel column, the values of the pixels above the pixel are all “1”. This process is performed for all the vertical pixel columns of the mask data, and as a result, fourth correction data is generated. For example, from the mask data MD0 illustrated in the upper part of FIG. 15, fourth correction data MD4 illustrated in the lower part is generated.

  When the first to fourth correction data are obtained as described above, a logical product (AND) of these four correction data is derived. As a result, data in which pixels corresponding to empty areas are filled in the original mask data is generated as complementary mask data (step S46). For example, complementary mask data MD5 shown in the lower part of FIG. 15 is generated from the first to fourth correction data MD1 to MD4 illustrated in FIG. As can be seen by comparing the mask data MD0 and the complementary mask data MD5, the complementary mask data MD5 is filled with pixels corresponding to empty areas.

  When complementary mask data is obtained, this complementary master data is used, and an area composed of pixels of the target image corresponding to a pixel having a value of “1” in the complementary mask data is acquired. Thereby, the target person candidate area | region where the empty area | region was complemented is acquired (step S47).

<2-4. Eye candidate area extraction processing>
Next, details of the eye candidate area extraction process (FIG. 3: step S14) will be described. In this eye candidate area extraction process, eye candidate areas that are candidates for eye areas are extracted from the target person candidate areas in which the empty areas are complemented as described above. As described above, in an image showing a person's face, the eye region has the highest edge strength, and therefore the edge strength is used even when the eye candidate region is extracted.

  FIG. 16 is a diagram showing a detailed flow of the eye candidate area extraction process. Unless otherwise stated, all the steps of the eye candidate region extraction process are performed by the eye candidate extraction unit 33 shown in FIG.

  First, the edge strength of each pixel in the target person candidate area is derived (step S51). The derivation of the edge strength is performed by a convolution process using the Laplacian filter 5 as in step S21 of FIG.

  Next, the derived edge strength is equal to or greater than a predetermined threshold Th0, and adjacent pixel groups are set as high edge regions (step S52). Thereby, one or a plurality of high edge regions are set. The method for setting the high edge region is substantially the same as the method for setting the level 0 contour region described above unless the threshold value is taken into consideration. Therefore, if the threshold value Th0 and the threshold value Th1 may be the same, the level 0 contour area included in the target person candidate area may be set as the high edge area as it is.

  Next, among the set high edge regions, one high edge region is determined as the “target high edge region” to be processed (step S53). Then, it is determined whether or not the size of the target high edge region satisfies a predetermined condition (step S54). The size of the target high edge region is expressed by three size variables, and it is determined whether or not each size variable is equal to or larger than a predetermined threshold value.

  One of the size variables is “number of pixels” (Pnum) included in the target high edge region. Then, it is determined whether or not the number of pixels (Pnum) is equal to or greater than a predetermined threshold value (PnumTh). Specifically, it is determined whether or not the following expression (5) is satisfied.

Pnum> PnumTh (5)
Further, as shown in FIG. 17, the other one of the size variables is the number of pixels included in the target high edge region EN with respect to the number of pixels (RaPn) of the circumscribed rectangle Re that is the smallest rectangle circumscribing the target high edge region EN. It is a “rectangular area ratio” (Rar) which is a ratio of (Pnum). The rectangular area ratio (Rar) is derived by the following equation (6).

Rar = Pnum / RaPn (6)
Then, it is determined whether or not the rectangular area ratio (Rar) is equal to or greater than a predetermined threshold (RarTh). Specifically, it is determined whether or not the following expression (7) is satisfied.

Rar> RarTh (7)
Furthermore, another size variable is an “image area ratio” (Aar) which is a ratio of the number of pixels (Pnum) included in the target high edge region to the total number of pixels (AaPn) of the target image. The image area ratio (Aar) is derived by the following equation (8).

Aar = Pnum / AaPn (8)
Then, it is determined whether or not the image area ratio (Aar) is equal to or greater than a predetermined threshold (AarTh). Specifically, it is determined whether or not the following expression (9) is satisfied.

Aar> AarTh (9)
When all of the above three size variables satisfy the condition (that is, when all of the expressions (5), (7), and (9) are satisfied), it is determined that the size of the target high edge region satisfies the predetermined condition. (FIG. 16: Yes in step S54). In this case, the target high edge region is set as the eye candidate region (step S55). On the other hand, when the size of the target high edge region does not satisfy the predetermined condition (when the size is relatively small), the target high edge region is not set as the eye candidate region.

  As described above, when the size of one target high edge region is determined, it is determined whether there is an unprocessed high edge region (step S56). If it exists (Yes in step S56), the next target high edge region is determined (step S53), and the same size determination as described above is performed for this target high edge region. Such processing is repeated, and finally the size is determined for all the high edge regions, and only the high edge regions that satisfy the conditions are extracted as eye candidate regions. As a result, a relatively small high-edge region that is unlikely to be an eye candidate is excluded, so that the eye candidate region can be extracted more accurately.

  Once the eye candidate areas have been extracted in this way, it is next determined whether or not two or more eye candidate areas have been extracted (step S57). Since there are two eyes on the face of a person, there must be at least two or more eye candidate areas in the person area. For this reason, when two or more eye candidate areas are not extracted from the target person candidate area (No in step S57), the target person candidate area is excluded from the candidates for the person area (step S58). The subsequent person determination process (FIG. 3: step S15) is not performed, and the process proceeds to step S16 in FIG. 3 as it is.

<2-5. Person determination processing>
Next, details of the person determination process (FIG. 3: step S15) will be described. In this person determination process, it is determined whether or not the target person candidate area is a person area based on any two of the eye candidate areas extracted from the target person candidate area as described above. . If the two eye candidate regions are actually human eye regions, they will be approximately the same size, and they should be within a certain range of each other. For this reason, in this person determination process, points are assigned so that the possibility of a person area increases with respect to the comparison result of the sizes of the two eye candidate areas and the mutual arrangement relationship. If the score is equal to or greater than a predetermined value, it is determined that the target person candidate area is a person area.

  FIG. 18 is a diagram showing a detailed flow of the person determination process. Unless otherwise stated, all the steps of the person determination process are performed by the person determination unit 34 shown in FIG.

  First, two eye candidate areas are selected as target eye candidate areas to be processed from among a plurality of eye candidate areas extracted from the target person candidate areas (step S61).

  Next, the sizes of the two target eye candidate regions are compared, and a first determination score used for person determination is obtained as a comparison result (step S62). Specifically, the circumscribed rectangles of the two target eye candidate areas are assumed. Then, the first determination score P1 is derived so as to increase as the size of the two circumscribed rectangles is the same (that is, as the possibility of the person area increases). Here, assuming that the vertical width of one circumscribed rectangle is Ht1, the horizontal width is Wt1, the vertical width of the other circumscribed rectangle is Ht2, and the horizontal width is Wt2, the first determination point P1 is expressed by the following formula ( 10).

P1 = {1-2 * ABS [(Ht1-Ht2) / (Ht1 + Ht2)]} * {1-2 * ABS [(Wt1-Wt2) / (Wt1 + Wt2)]}
(However, “ABS” is the absolute value of [].) (10)
When the first determination score P1 is derived, next, the second determination score used for person determination is obtained according to the mutual arrangement relationship between the two target eye candidate regions (step S63).

  Specifically, as shown in FIG. 19, the average value of the length of the diagonal line DL1 of the circumscribed rectangle Re1 of one target eye candidate area and the length of the diagonal line DL2 of the circumscribed rectangle Re2 of the other target eye candidate area is calculated as DLAv. And Further, a minimum rectangle (hereinafter referred to as “included rectangle”) Re3 including two circumscribed rectangles R1 and R2 is assumed, and the length of the diagonal line DL3 is assumed to be DLL. And the value K shown by following Formula (11) is derived | led-out.

K = DLL / DLAv (11)
Then, α is a predetermined constant, and the second determination score P2 is derived by the following formula (12).

When K ≧ α
P2 = α / K;
When K <α
P2 = K / α (12)
That is, the determination score P2 is derived so that the value K becomes higher as it approaches the predetermined constant α. The constant α is set to “3”, for example. In general, in the face of a person, the length occupied by two eyes is about three times the length of one eye. Therefore, in this equation, the length of the diagonal line DL3 of the inclusion rectangle Re3 (DLL) is the length occupied by the two eyes, and the average value (DLAV) of the diagonal lines of the two circumscribed rectangles is regarded as the length of one eye, respectively. The closer the value K is to the constant α (= “3”) (that is, the higher the possibility of the person area), the higher the determination point P2 is given.

  When the two determination points P1 and P2 are derived, they are added to derive a total determination point P3 (FIG. 18: Step S64).

As described above, when the total determination score P3 is derived for one combination of the two eye candidate regions, it is determined whether another unprocessed combination exists (step S65). If it exists (Yes in step S65), the combination of the next two eye candidate regions is determined (step S61), and the total determination score P3 is derived for this combination in the same manner as described above. The Such processing is repeated, and finally the total judgment score P3 is derived for all combinations. When the number of the plurality of eye candidate areas extracted from the target person candidate area is _n , the number of combinations is _n C ₂ , and _n C ₂ total determination points P3 are derived.

  Next, the maximum value of the derived total determination score P3 is acquired, and it is determined whether or not this maximum value is equal to or greater than a predetermined threshold (step S66). If the maximum value of the total determination points P3 is equal to or greater than the threshold value, the target person candidate region is determined as a person region (step S67). If the maximum value of the total determination points P3 is less than the threshold value, the target person candidate region is determined. It is determined that the area is not a person area (step S68).

<3. Summary>
As described above, in the human region detection device 10 according to the present embodiment, the edge strength at each pixel of the target image is derived, and the pixel groups that belong to the same level and are adjacent to each other in the target image are at that level. It is set as the contour area to which it belongs. One contour region belonging to the level with the highest edge strength is set as the initial attention region, and contour regions close to the attention region belonging to a level different from the maximum level are noticed in order from the level with the highest edge strength. Bound to the region. The attention area obtained as a result of the combination is set as the person candidate area.

  Therefore, since the person area detection device 10 specifies the person candidate area based on the edge strength without using the pixel color of the target image, the person candidate area can be specified without depending on the pixel color of the target image. For this reason, even when a gray scale image, an image with a greatly changed hue, or the like is used as the target image, the person candidate area can be correctly specified.

  Moreover, since the empty area is complemented by the empty area complementing process for the candidate person area that has an empty area inside, a person candidate area that is more likely to be a person can be acquired. Further, the eye candidate area is extracted from the person candidate area, and the person area is accurately specified in order to determine whether or not it is a person area based on the comparison result of the sizes of the two eye candidate areas and the mutual arrangement relationship. It is possible.

<4. Other embodiments>
Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications are possible. Hereinafter, such other embodiments will be described. Of course, the following embodiments may be appropriately combined.

  In the above embodiment, according to the algorithm, the result of performing the empty area complementing process (FIG. 3: step S13) for one person candidate area is the result of performing the empty area complementing process for the other processed person candidate areas. May be nearly identical. In this case, even if the subsequent sub-process (steps S14 and S15) is performed, only the same person region is detected, and the subsequent sub-process may be omitted.

  In the eye candidate region extraction process of the above embodiment, when all of the expressions (5), (7), and (9) are satisfied, it is determined that the size of the target high edge region satisfies the predetermined condition. When only one or two are satisfied, it may be determined that the predetermined condition is satisfied. Further, in Expressions (5), (7), and (9), only the lower limit of each size variable is specified as a threshold value, but an upper limit may be further specified.

  Further, in the eye candidate area extraction process of the above embodiment, the target person candidate area for which the eye candidate area of “2” or more was not extracted is excluded from the subsequent processes. As well as “2”, it may be “1” or “3” or more.

  In the person determination process of the above embodiment, the person area is determined based on both the comparison result of the sizes of the two target eye candidate areas and the mutual arrangement relationship. The person area may be determined.

  In the above embodiment, the case where a still image is the target image has been described as an example, but a moving image may be the target image. In this case, the same processing as described above may be performed on each frame included in the moving image.

  In the above-described embodiment, the person area detection device is described as being configured by a general computer. However, the function described in the above-described embodiment can be applied to an imaging device such as a digital still camera or a video camera, a printer, or the like. It may be included as a part of the function of another type of device that handles images, such as a printing apparatus, and the device may function as a person area detection device.

  Further, in the above-described embodiment, it has been described that various functions are realized in software by the arithmetic processing of the CPU according to the program. However, some of these functions are realized by an electrical hardware circuit. Also good.

It is an external view of a person area detection apparatus. It is a block diagram which shows typically the internal structure of a main-body part. It is a figure which shows the flow of the whole process which detects a person area. It is a figure which shows the detailed flow of a person candidate area | region extraction process. It is a figure which shows the example of a Laplacian filter. It is a figure explaining the method of calculating | requiring an edge image from a target image. It is a figure explaining the method of setting a contour area. It is a figure which shows the detailed flow of a contour area joint process. It is a figure which shows an example of an attention area. It is a figure which shows an example of an object contour area | region. It is a figure which shows an example of arrangement | positioning of an attention area | region and a target level area. It is a figure which shows an example of arrangement | positioning of an attention area | region and a target level area. It is a figure explaining the principle which extracts a person candidate area | region. It is a figure which shows the detailed flow of an empty area complementation process. It is a figure explaining the method of complementing an empty area. It is a figure which shows the detailed flow of an eye candidate area | region extraction process. It is a figure which shows object high edge area | region and its circumscribed rectangle. It is a figure which shows the detailed flow of a person determination process. It is a figure which shows two circumscribed rectangles and those inclusion rectangles.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Person area detection apparatus 31 Person candidate extraction part 32 Area complement part 33 Eye candidate extraction part 34 Person determination part 41 Program 61 Target image 62 Edge image 63a-63c Mask data 64a-64c Contour image A1-A5 Contour area

Claims (10)

A person area detection device for detecting a person area indicating a person from a target image,
Means for deriving edge strength at each pixel of the target image;
Means for classifying the edge intensity into a plurality of levels, and setting pixel groups adjacent to each other belonging to the same level in the target image as contour regions belonging to the level;
One of the contour regions belonging to the level having the maximum edge strength is set as an initial attention region, and the contour region belonging to a level different from the maximum level and close to the attention region is coupled to the attention region. Coupling means to be part of the region of interest;
With
The combining means performs the combination of the contour areas to the attention area in the order of the contour areas belonging to a level having a large edge strength, and the obtained candidate area is a person candidate area that is a candidate for the person area. A person area detecting device characterized by the above. In the person area detection device according to claim 1,
Means for expanding the contour region before joining by the joining means;
The person area detecting device further comprising: In the person area detection device according to claim 1 or 2,
Complementing means for complementing a free area existing inside the person candidate area obtained by the combining means;
The person area detecting device further comprising: In the person area detection device according to claim 3,
Eye candidate region extraction means for extracting pixel groups adjacent to each other whose edge strength is equal to or greater than a predetermined threshold in the candidate human region after complementing the vacant region as candidate eye regions that are candidates for regions indicating human eyes. ,
The person area detecting device further comprising: In the person area detection device according to claim 4,
The eye candidate area extracting means extracts, as the eye candidate area, each of the pixel groups whose edge intensity is equal to or greater than a predetermined threshold and whose size satisfies a predetermined condition among adjacent pixel groups. Detection device. In the person area detection device according to claim 4 or 5,
Means for excluding the human candidate area from which the predetermined number or more of the candidate eye areas are not extracted from the candidate human area;
The person area detecting device further comprising: In the person area detection device according to any one of claims 4 to 6,
Determination means for determining whether or not the person candidate area is the person area based on a comparison result of the sizes of any two of the eye candidate areas extracted from the person candidate area;
The person area detecting device further comprising: In the person area detection device according to any one of claims 4 to 7,
Determination means for determining whether or not the person candidate area is the person area based on a mutual arrangement relationship between any two of the eye candidate areas extracted from the person candidate area;
The person area detecting device further comprising: A person area detection method for detecting a person area indicating a person from a target image, comprising:
(A) deriving edge strength at each pixel of the target image;
(B) dividing the edge intensity into a plurality of levels, and setting pixel groups belonging to the same level and adjacent to each other in the target image as contour regions belonging to the level;
(C) One of the contour regions belonging to the level having the maximum edge strength is set as an initial attention region, and the contour region belonging to a level different from the maximum level and close to the attention region is defined as the attention region. To be a part of the region of interest;
With
In the step (c), the contour areas are combined with the attention area in the order of the contour areas belonging to a level having a large edge strength, and the obtained attention area is a person who is a candidate for the person area. A person area detection method characterized in that the area is a candidate area. A program for detecting a person area indicating a person from a target image,
(A) deriving edge strength at each pixel of the target image;
(B) dividing the edge intensity into a plurality of levels, and setting pixel groups belonging to the same level and adjacent to each other in the target image as contour regions belonging to the level;
(C) One of the contour regions belonging to the level having the maximum edge strength is set as an initial attention region, and the contour region belonging to a level different from the maximum level and close to the attention region is defined as the attention region. To be a part of the region of interest;
And execute
In the step (c), the contour areas are combined with the attention area in the order of the contour areas belonging to a level having a large edge strength, and the obtained attention area is a person who is a candidate for the person area. A program characterized as a candidate area.