KR101357581B1 - A Method of Detecting Human Skin Region Utilizing Depth Information - Google Patents

Abstract

A method of detecting a depth-based human skin region by labeling pixels having similar depth characteristics from a stereo three-dimensional image and detecting regions having a distribution of human skin color among the labeled regions as actual skin color regions. a) extracting depth information from the stereo three-dimensional image; (b) labeling the image of the depth information as an object; (c) detecting a skin color region from the 3D image; (d) detecting a candidate group of skin regions by AND combining the labeling region and the skin color region; And (e) evaluating texture complexity in the candidate group of skin regions to detect the final skin region.
By using the skin region detection method as described above, by effectively combining the color feature and the depth feature, a large number of errors of the skin color region which are incorrectly detected in the existing background region can be eliminated, thereby extracting the skin region more accurately. can do.

Description

Depth Information-based Human Skin Region Detection Method {A Method of Detecting Human Skin Region Utilizing Depth Information}

The present invention relates to a depth information-based human skin region detection method for labeling pixels having similar depth features from a stereo three-dimensional image and detecting regions having a distribution of human skin color among the labeled regions as actual skin color regions. will be.

Recently, due to the rapid spread of low-cost digital cameras and smartphones or tablet computers with built-in cameras, it has become very easy to shoot still images and videos, and the amount of video data is increasing exponentially. In addition, there is a growing interest in various image analysis techniques to effectively manage these materials.

One of the major research topics in this field of image analysis is to find areas with specific colors in a given image. Among these subjects, the technique of accurately extracting the area representing human skin color is particularly important because it provides meaningful clues for detecting a person from an input still image or video. This extraction of skin color is very useful in many applications such as detection and tracking of moving objects, gesture recognition using hand area detection, face recognition, content-based image retrieval, harmful content detection and filtering. It is used [Document 1,2].

In the related literature, several existing methods for extracting two-dimensional skin color regions can be identified. Lee segmented skin color using skin color models that can withstand color bias due to special lighting effects in the YCbCr space [3]. The various features were then used to determine the authenticity of the segmented skin area. Cho proposed a method of finding the facial region by adaptively shifting the threshold value according to the color and brightness of the entire image based on a predetermined threshold value using the HSV color space [Ref. 4]. This method attempts to detect the skin color area of only one race (yellow race), and therefore, a problem occurs when the skin color of another race (white or black) is present. Hsu proposed a new method for face detection using EyeMap and MouthMap [Ref. 5]. In this method, facial components were detected using a feature map derived from the skin color model, and their relationships were defined based on the geometric features of the detected facial components. However, this method uses only simple geometric relationships, which limits its flexibility. Fang proposed a new color histogram-based method for face detection [Ref. 6]. In this method, the color histograms for the different areas of the face are connected to form a vector that sets up the spatial relationship between these areas, and the algorithm is developed to detect the face effectively using the vector. In addition to the methods mentioned above, other methods continue to be introduced in the literature [7].

These existing algorithms still have many problems with the skin color gamut extraction algorithm. That is, the skin color of the human skin is not basically the same in the photographed image due to differences between individuals or races. In addition, the skin color in the input image is slightly different due to various environmental conditions such as color makeup, makeup, camera used, and lighting changes. In particular, there are limitations in extracting skin color using only two-dimensional features such as color, texture, shape, and geometric relationship.

The skin color region extraction method as described above may be applied to a 3D image to detect a human face region. However, even in this case, there still exists a problem in the skin color region extraction method of the 2D image.

Therefore, there is a need for a method of more accurately detecting a skin color region by combining color features most frequently used in a two-dimensional skin color extraction method with three-dimensional depth information, which is a characteristic of a three-dimensional image.

[1] A. Drosou, D. Ioannidis, K. Moustakas, and D. Tzovaras, "Spatiotemporal Analysis of Human Activities for Biometric Authentication," Computer Vision and Image Understanding, Vol. 116, No. 3, pp. 411-421, 2012. [Document 2] S.-W. Jang, Y.-J. Park, G.-Y. Kim, and S.-Y. Lee, "Skin Region Extraction Combining 3D Depth and Color Features," In Proc. of the Winter Conference on the Korea Society of Computer and Information, Vol. 20, No. 1, pp. 201-204, 2012. [Reference 3] J.-S. Lee, Y.-M. Kuo, P.-C. Chung, and E.-L. Chen, "Naked Image Detection based on Adaptive and Extensible Skin Color Model," Pattern Recognition, Vol. 40, No. 8, pp. 2261-2270, 2007. Document 4 K.-M. Cho, J.-H. Jang, and K.-S. Hong, "Adaptive Skin-Color Filter," Pattern Recognition, Vol. 34, No. 5, pp. 1067-1073, 2001. Document 5 R.-L. Hsu, M. Abdel-Mottaleb, and A. K. Jain, "Face Detection in Color Images," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, No. 5, pp. 696-706, 2002. J. Fang and G. Qiu, "A color histogram-based approach to human face detection," In Proc. of the International Conference on Visual Information Engineering, pp. 133-136, 2003. [7] K. M. Lee, "Component-based Face Detection and Verification," Pattern Recognition Letters, Vol. 29, No. 3, pp. 200-214, 2008. [8] N. Baha and S. Larabi, "Accurate Real-Time Neural Disparity MAP Estimation with FPGA," Pattern Recognition, Vol. 45, No. 3, pp. 1195-1204, 2012. Document 9 Y.-M. Paik, H.-J. Choi, Y.-H. Seo, and D.-W. Kim, "A Study on the Outlier Improvement Method Using Cost Function," In Proc. of the Fall Conf. of the Korean Society of Broadcasting Engineers, pp. 269-272, 2009. Document 10 G.-J. Liu, X.-L. Tang, H.-D. Cheng, J.-H. Huang, and J.-F. Liu, "A Novel Approach for Tracking High Speed Skaters in Sports Using a Panning Camera," Pattern Recognition, Vol. 42, No. 11, pp. 2922-2935, 2009. Document 11 H.-H. Do, S. Melnik, and E. Rahm, "Comparison of Schema Matching Evaluations," Lecture Notes in Computer Science, Vol. 2593, pp. 221-237, 2003. 12. N. Otsu, "A Threshold Selection Method from Gray-Level Histogram," IEEE Transactions on Systems, Man and Cybernetics, Vol. 9, No. 1, pp. 62-66, 1979.

SUMMARY OF THE INVENTION An object of the present invention is to solve the above-mentioned problems, and to label pixels having similar depth characteristics from a stereo three-dimensional image, and to have a real skin color region among the labeled regions with human skin color distribution. It is to provide a depth information based human skin area detection.

In order to achieve the above object, the present invention relates to a depth information based human skin area detection method for detecting a skin area of a human from a stereo three-dimensional image composed of left and right images, (a) depth information from the stereo three-dimensional image; Extracting; (b) labeling the image of the depth information as an object; (c) detecting a skin color region from the 3D image; (d) detecting a candidate group of skin regions by AND combining the labeling region and the skin color region; And (e) evaluating texture complexity in the candidate group of skin regions to detect the final skin region.

In addition, the present invention is a depth information-based human skin region detection method, in the step (a), characterized in that the depth information is extracted using a stereo matching technique based on graph cuts (graph cuts).

In addition, the present invention provides a depth information-based human skin region detection method, in the step (a), converts the left and right color image of the stereo three-dimensional image to the left and right gray image, and applies stereo matching between the left and right gray image. And extracting depth information.

In addition, the present invention is a depth information-based human skin area detection method, in the step (c), by detecting the eye area in the three-dimensional image, to generate a skin map from the detected eye area, and from the skin map The skin color model may be generated to extract a skin color region.

In addition, in the depth information based human skin region detection method, in step (c), a sample located in an extended region of the minimum inclusion rectangle (MER) corresponding to the extracted eye region is selected and selected. It is characterized by generating a skin map such as [Equation 1] from the sample.

[Equation 1]

과

는 각각 일반적인 피부 색상의 YC_bC_r 공간의 화소값 임.However, C _r and C _b are pixel values of the YC _b C _r space located in the extended area of the minimum containing rectangle (MER) corresponding to the eye area.

and

Are the pixel values of the YC _b C _r space of the normal skin color, respectively.

In addition, in the depth information based human skin region detection method, in step (e), edges are applied to a region (hereinafter, candidate region) belonging to the candidate group of the skin region by applying a Sobel edge operator. It is characterized by calculating the texture complexity by extracting the degree of.

In another aspect, the present invention provides a depth information-based human skin region detection method, wherein the texture complexity T (R _i ) for the i-th candidate skin region belonging to the candidate group of the skin region is calculated by [Equation 2]. do.

[Equation 2]

Where I _gray (x, y) is the contrast at the x and y positions,

w _h (x, y) and w _v (x, y) are the horizontal and vertical masks of the Sobel edge operator,

E (x, y) is the edge at x and y positions,

N (R _i ) is the number of pixels belonging to the region R _i .

As described above, according to the depth information-based human skin region detection method according to the present invention, by effectively combining the color feature and the depth feature, it is possible to remove a large number of errors in the skin color region that are incorrectly detected in the existing background region. As a result, the overall skin area can be extracted more accurately.

1 is a diagram showing a configuration of an overall system for carrying out the present invention.
2 is a flowchart illustrating a method for detecting a skin region of a human based on depth information according to an embodiment of the present invention.
3 is a view for explaining the basic concept of stereo matching.
4 illustrates an example of extracting depth information from a stereo image according to the present invention.
5 is a flowchart illustrating a step of extracting a skin color region according to an embodiment of the present invention.
6 illustrates an example of a skin map generation and binarization process according to the present invention.
7 is an example of a Sobel operator mask in accordance with the present invention.
8 is an example of skin area detection according to an experiment of the present invention.
9 is a table for comparing the performance with the prior art according to the experiment of the present invention.
10 is a graph of a performance comparison with the prior art according to the experiment of the present invention.
FIG. 11 is a block diagram illustrating a configuration of a system for detecting a skin area of a human based on depth information according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the drawings.

In the description of the present invention, the same parts are denoted by the same reference numerals, and repetitive description thereof will be omitted.

First, examples of the configuration of the entire system for carrying out the present invention will be described with reference to Fig.

As shown in FIG. 1, a depth information based human skin region detection method according to the present invention receives a stereo image (or image) 10 and detects a human skin region from the image (or image). 20) on a program system. That is, the skin region detection method may be configured as a program and installed and executed in the computer terminal 20. The program installed in the computer terminal 20 may operate like one program system 30.

Meanwhile, as another embodiment, the stereo matching method may be implemented as a program and operate in a general-purpose computer, and may be implemented as one electronic circuit such as an ASIC (custom semiconductor). Alternatively, the present invention may be developed as a dedicated computer terminal 20 for exclusively processing skin area detection of a digital image (or image). This is called a skin area detection device. Other possible forms may also be practiced.

Next, a method for detecting a skin region of a human based on depth information according to an embodiment of the present invention will be described in more detail with reference to FIG. 2.

As shown in Figure 2, the depth information-based human skin region detection method according to the present invention comprises the steps of (a) extracting depth information from the three-dimensional image (S10); (b) dividing an area by labeling pixels in depth information (S20); (c) extracting the skin color region (S30); (d) detecting a candidate group of skin regions by AND combining the labeling region and the skin color region (S40); And (d) detecting the final skin region using the texture complexity (S50).

The present invention basically receives a three-dimensional stereoscopic image. In general, three-dimensional stereoscopic display uses multi-view stereoscopic vision technology to add depth information to a two-dimensional image, which makes the viewer feel as if they are in the scene where the image is being produced. And new generation technology that makes you feel the reality.

Figure 2 shows the overall schematic of the skin color region extraction method according to the invention in the form of a block diagram. The proposed method first converts the input color image into a gray image, and then extracts a three-dimensional depth feature representing the distance between the camera and the object from the left and right images using stereo matching. Then, pixels having similar depth characteristics are labeled, and only those regions having a two-dimensional skin color distribution among the labeled regions are determined as actual skin color regions.

First, an operation (S10) of extracting depth information from a 3D image will be described.

Stereo matching for three-dimensional depth extraction is one of the classic problems in computer vision, and is useful in many applications such as robot navigation, three-dimensional modeling, and image-based rendering. The purpose of stereo matching is to calculate a disparity map for a reference image when two or more images for the same scene are given, where the variation refers to the difference in position between the corresponding two pixels. And in order to calculate the disparity map, it is necessary to solve the problem of correspondence with each pixel [Ref. 8].

In general, in the case of binocular stereo, the epipolar lines are horizontal, assuming that two input images have been taken with a calibrated camera and have been rectified. However, despite these constraints, it is still difficult to accurately determine the variation due to the ill-posed nature of stereo matching. In particular, it is more difficult in the overlapped or lacking texture area [9].

In general, stereo matching algorithms can be broadly classified into local methods and global methods. The local method uses a window of constant size for matching to increase the discriminatory power of the correspondence point search. In this method the correspondence points can be extracted by comparing the contrast values using various matching measures within the local window. This method is very fast, but has the disadvantage of degrading accuracy at regions overlapping locally or lacking texture, and discontinuous boundaries.

The global method uses an energy minimization algorithm with a smoothness constraint to solve the ill-posed problem of stereo matching. Therefore, this method can find correspondence even in areas where texture is lacking. However, due to the gradual constraints, discontinuities in the disparity information are eliminated. Therefore, a stereo matching algorithm is introduced that uses a gentleness constraint that preserves discontinuities. Global methods extract matching points by various minimization techniques. Recently, graph cuts and trust propagation-based algorithms are used because of their superior performance. However, many global methods do not explicitly consider overlapping issues.

In the present invention, the distance is measured using a graph cuts based stereo matching technique. In this method, gray images are used without considering color information.

The graph cut-based method is one of the global minimization methods. One of the many tasks to minimize energy functions in computer vision is to assign a label to every pixel. In the stereo world, a label usually means disparity. This problem aims to find f that assigns the label p ∈ P to f _p ∈ L for each pixel. Where P is the set of pixels and L is the set of labels. And to solve this problem, we can formulate the energy function.

Where D _p Is a data term that indicates how well we can assign the label f _p to pixel p. V _{p , q} represents a change in transition between pixels p and q in a smoothness term. In conclusion, the graph cut is used to find f that minimizes the energy function.

First, the input stereo color image is converted into a gray image using [Equation 1].

[Equation 1]

However, r, g, and b represent r, g, and b color values of the input image, and gray represents contrast values of pixels converted to black and white.

After converting the input image to a gray image, depth information is extracted by matching the left and right images. The basic concept of extracting depth information from left and right images is shown in FIG. 3. As shown in FIG. 3, P is a point in the real world, x _l and x _r are x coordinates where P is bound to the left and right images, f is the focal length of the camera, T is the baseline of the camera, and Z is the depth to be extracted. Let's call it a value. Then, since two triangles (p _l , P, p _r ) and (O _l , P, O _r ) are similar, Equation 2 holds, and if you expand Equation 2 with respect to Z, As shown in [3], the depth information Z can be extracted.

&Quot; (2) "

&Quot; (3) "

Extracting the disparity of the left and right images through stereo matching is performed using a graph cut-based method, which is an existing algorithm, as described above. In Equation 2, the variation represents x _l -x _r , and in Equation 3, x _l -x _r Is simply represented by d. As a result, in Equation 3, the depth information Z is extracted using the variation d.

Next, step (S20) of dividing an area by labeling pixels in depth information will be described.

In the present invention, after extracting depth information, pixels having similar depth values are labeled and divided into area units. That is, the pixels of similar depths are labeled with a two-dimensional image composed of depth information, and the image is divided into regions having similar depth values. In this case, the pixels included in the image having a difference in depth value within a predetermined range are equally labeled, thereby being divided into area units.

4 shows an example of extracting depth from a stereo image. 4 (a) and 4 (b) show left and right images of a stereo image, and (c) shows a result of extracting depth information by applying a stereo matching algorithm from left and right input images.

Next, (c) extracting the skin color region (S30) will be described in more detail with reference to FIG.

In the present invention, after obtaining a skin color region from an image using color, the final skin color region is extracted by combining with the depth information extracted in the previous step.

In general, most existing skin color extraction methods use a predefined skin color distribution model through prior learning, but still have inherent problems. That is, the skin color itself is not the same for each person due to individual skin color differences and race color differences. In addition, the skin color region of the acquired image is not the same due to special makeup or tonal cosmetics, optical equipment used for photographing, and lighting effects. Therefore, the existing algorithm using a predefined skin model is difficult to overcome the various changes mentioned above. The optimal solution to this problem is to generate a skin model that is adaptive to the input image by selecting a unique skin color sample for each person whenever a different image is input, and then use the input image. The skin color area of the skin should be extracted.

To this end, the present invention finds the eye, which is the main component of the face, from the input image, and reliably extracts skin color samples around the found eye area to generate a skin model that is optimally suited for the input image. The skin region is extracted from the input image using this model. In this case, the input image is a color image of any one of the left and right images of the stereo 3D image.

As shown in Figure 5a, (c) extracting the skin color region (S30) is (c1) pre-treatment step (S31); (c2) detecting an eye region (S32); (c3) selecting a predetermined skin map when the eye region is not detected (S33); (c4) when the eye region is detected, generating a skin map from the eye region (S34); (c5) generating a skin color model from the skin map (S35); (c6) extracting the skin color region using the skin color model (S36).

First, as a preprocess, the color space RGB of the input image is converted into the YC _b C _r space which is known to be most suitable for skin color extraction as shown in [Equation 4] (S31).

&Quot; (4) "

Next, both eye regions are detected using EyeMap [Document 5] in which color and contrast based maps are mixed (S32).

Next, if the eye area is not detected, a skin map to be used as a predetermined skin map is selected (S33).

Next, when the eye area is detected, a skin map is generated from the eye area (S34).

A sample located in an area extended five times the minimum inclusion rectangle MER corresponding to the extracted eye region is selected, and a skin map as shown in Equation 5 below is generated from the selected sample (S34). .

&Quot; (5) "

과

는 각각 일반적인 피부 색상의 C_r과 C_b 값을 나타낸다. 그리고 피부 맵은 0에서 255사이의 값의 범위를 가지는데, 선택된 샘플이 평균적인 피부 색상에 근접할수록 255와 가까운 값을 가진다.here,

and

Are the common skin colors C _r and C _{b, respectively} Value. The skin map has a value ranging from 0 to 255. The closer the selected sample is to the average skin color, the closer to 255.

6 shows an example of generating a skin map from an input image and binarizing it. In FIG. 6, a quadrangle around both eyes of the left image represents an area five times larger than an eye area minimum including rectangle. Selecting samples from this area and creating a skin map results in a picture centered in FIG. 6. In other words, applying [Equation 5] to each of the samples located within five times the minimum containing rectangle generates a skin map. Then, when the skin map is binarized into a skin region and a non-skin region, an image located on the right side of FIG. 6 is obtained.

과

는 일반화된 피부 모델의 C_r과 C_b 값이다.In [Equation 5], C _r and C _b refer to the values of C _r and C _b of samples located in a region 5 times extended of the minimum inclusion rectangle (MER) corresponding to the eye region. And

and

Is the C _r and C _b values of the generalized skin model.

The reason for doing this is to primarily determine whether the pixel value of the sample extracted from the 3D input image is a color close to the skin color. In other words, the C _r and C _b values of the generalized skin model are not 100% accurate skin color values for all input images, but have similar values to human skin color.

Next, after binarizing the skin map, a skin color model is generated using only the skin map determined as the skin color (S35), and the skin color region is robust from the entire image using the skin color model generated through the above process. To extract (S36).

That is, the skin map is first generated using the C _r and C _b values of the generalized skin model, and then the samples determined to be non-skin color are first removed from the skin map generated through the binarization of the skin map. Only actual skin color samples are selected to generate a skin color model.

In order to select only skin sample pixels in the skin map, the present invention uses the histogram binarization method proposed by Otsu [Ref. 12]. The Otsu method statistically selects the optimal threshold for binarizing the contrast histogram without prior knowledge and is known to perform well when the histogram has two probability densities. In this paper, a histogram is created for the data contained in the skin map, and the histogram is binarized by the Otsu method to select only skin color pixels except non-skin color pixels.

FIG. 5B is a different representation of the process of FIG. 5A, and shows an overall schematic diagram of a method for robustly extracting a skin color region using a skin model adaptive to the above-described input image. As shown in FIG. 5B, when the human eye does not exist in the input image or when the human cannot close the eye by detecting the eye, the skin color region is extracted using a predefined skin model as in the conventional method. There is no problem with the execution stability of the proposed method.

Next, the candidate group of the skin region is detected by AND combining the labeling region and the skin color region (S40). That is, a three-dimensional skin color region is detected in the input image by combining the labeled region in the depth information image of step S20 and the color-based skin region detected in the color image of the three-dimensional image (S40).

To this end, first, a distance area having the same level is labeled from a distance image. In general, distances of various levels are measured in the same object. Then, a distance corresponding to the two-dimensional skin region is obtained by performing an AND operation on the labeled distance image (depth image) and the extracted skin color image extracted using a skin color model adaptive to the input image as shown in [Equation 5-2]. Select only the image.

&Quot; (5-2) "

In other words, Equation 5-2 can be described as below, where I _depth (x, y) is a depth image and I _{bi _ skin} (x, y) is a color-based binary skin. I _depth '(x, y) means a distance image corresponding to a two-dimensional skin region obtained through an AND operation of Equation 5-2. An algorithm for detecting a skin region by ANDing the labeling region and the skin color region is as follows.

[algorithm]

IF I _{bi _ skin} (x, y) ≡ 0 THEN

I _depth '(x, y) = I _depth (x, y)

ELSE

I _depth '(x, y) = 0

END IF

Then, the candidate group of the skin color region can be extracted for each label of the depth feature from I _skincolor (x, y), which is the skin image extracted based on I _depth '(x, y).

Next, the step (S50) of detecting the final skin region using the texture complexity will be described. That is, the final skin region is detected by evaluating texture complexity in the candidate group of the skin region (S50).

In order to select the final three-dimensional skin region, texture smoothness of the skin region candidate group by distance label is evaluated. In general, since the skin color region has a characteristic that the texture is not rough but gentle, the texture complexity of the three-dimensional skin region candidate group is measured to remove the rough texture regions and to determine only the gentle regions as actual skin regions.

In the present invention, the Sobel edge operator is applied in the corresponding region for the texture complexity measurement, and the edge degree is extracted. That is, the more edges present in the region, the rougher the texture is, and the less edges present, the smoother the texture. In general, the Sobel edge operator is a derivative operator for detecting an edge and performs the derivative once on the x-axis and the y-axis, respectively. The convolution mask corresponding to the Sobel operator is shown in FIG. 7.

The texture complexity T (R _i ) of the i-th candidate skin region R _i extracted through the AND operation between the depth image and the binarized skin image may be extracted as shown in [Equation 6]. In Equation 6, I _gray (x, y) denotes the contrast values at the x and y positions, and w _h (x, y) and w _v (x, y) are the horizontal and vertical masks of the Sobel edge operator. Means. E (x, y) represents the edge degree at x and y positions, and N (R _i ) means the number of pixels belonging to the region R _i .

&Quot; (6) "

In the present invention, after calculating the texture complexity for each candidate skin region using Equation 6, it is determined that only candidate regions included in an appropriate threshold range (or a predetermined threshold range) are actual skin regions. In the present invention, the threshold for texture complexity was determined through experiments using various stereoscopic images.

Next, the effects of the present invention through experiments will be described in more detail with reference to FIGS. 8 to 10.

The computer used for the experiment of the present invention used the Intel Core i7-2600 3.40GHz CPU and 8GB of memory, the operating system of Microsoft Windows 7 was used. As a compiler for implementing application software, the proposed skin color region extraction algorithm is implemented using Microsoft Visual C ++ 2008. In addition, we used various 3D still and video images taken in a general environment without specific constraints to construct an image database for the experiment.

8 shows an example of detecting a skin region from an input image using the present invention. FIG. 8 (a) shows the left image of the input stereo image, and FIG. 8 (b) shows the distance image measured through stereo matching from the input image. 8 (c) shows a binary result image obtained by extracting a skin region using a two-dimensional skin color model, and FIG. 8 (d) shows a three-dimensional image finally detected by combining a distance image and a color-based skin region. Represents a skin area.

As shown in (c) of FIG. 8, the skin color region extracted using the two-dimensional skin color model includes many false detections. In other words, if there is an area similar to the skin color in the background area instead of the human skin area, it is incorrectly detected as the skin area. However, it can be seen that the three-dimensional skin color region detected by combining the two-dimensional skin color feature with the depth feature extracts only the skin region more accurately.

In the present invention, since it is difficult to extract three-dimensional depth information in an area where texture is very poor or absent in the input image, it may be difficult to extract skin color.

In order to quantitatively evaluate the performance of the skin area detection method according to the present invention, a root mean square (RMSE) measure of an error as defined in [Equation 7] was defined. RMSE is a measure that deals with the general and specific aspects of image quality and is often used to measure the difference between actual and measured values, and is known as a good measure of accuracy.

[Equation 7]

In Equation (7), M and N represent horizontal and vertical lengths of an image, and i and j are indexes of rows and columns representing positions of an image. OB (i, j) represents a ground truth binary image, and RB (i, j) represents a binary result image of detecting a skin region.

In order to evaluate the performance more accurately, the original test image was manually converted into skin and non-skin areas and compared with the resultant image. In addition, since there are two kinds of errors, false positive and false negative, in skin region extraction, the overall scale as shown in [Equation 8] is used for the accuracy evaluation [Ref. 11].

&Quot; (8) "

9 and 10 show the results of the performance of comparing and evaluating the conventional two-dimensional skin area detection method and the three-dimensional skin area detection method according to the present invention in a table and a figure, respectively.

As can be seen in FIG. 10, the present invention detects the final skin region based on the 2D skin region detection result, and thus the precision of the proposed method is slightly lower. However, when the color similar to the skin color is distributed in the background area, the two-dimensional skin area detection occurs a little lower recall because a lot of false detection occurs. In contrast, since the present invention verifies the skin area by separating the area based on the distance image, the detection rate of the error can be significantly lowered, so that both precision and recall can obtain good results.

Next, a depth information based human skin region detection system according to an embodiment of the present invention will be described with reference to FIG. 11.

As described above, the depth information based human skin region detection method according to the present invention may be implemented as a program system, and each step of the method may be implemented by configuring one functional means.

The skin region detection system 30 is a system for detecting a human skin region from a stereo three-dimensional image composed of left and right images.

As shown in FIG. 11, a depth information based human skin region detection system according to an exemplary embodiment of the present invention includes a depth information extractor 31 extracting depth information from a stereo three-dimensional image; A labeling performing unit 32 for labeling an image of the depth information as an object; A color gamut detector 33 for detecting a skin color gamut from the 3D image; A candidate region detector (34) for detecting a candidate group of skin regions by AND combining the labeling region and the skin color region; And a final region detector 35 for evaluating the texture complexity in the candidate group of the skin regions to detect the final skin region.

In the present invention, a method of robustly extracting human skin color regions present in a 3D video input by combining depth and color features is proposed. In the present invention, three-dimensional depth information, which is a distance between a camera and an object, is extracted from a left and right image input using a stereo matching technique. Then, regions having similar depth characteristics were clustered, and regions having a two-dimensional skin color distribution among the clustered regions were determined as actual skin color regions.

As mentioned above, although the invention made by this inventor was demonstrated concretely according to the said Example, this invention is not limited to the said Example and can be variously changed in the range which does not deviate from the summary.

10: three-dimensional image 20: computer terminal
30: skin area detection system 31: depth information extraction unit
32: labeling unit 33: color gamut detection unit
34: candidate region detection unit 35: final region detection unit
36: memory

Claims (7)

In the depth information-based human skin region detection method for detecting a human skin region from a stereo three-dimensional image composed of left and right images,
(a) extracting depth information from the stereo 3D image;
(b) labeling the image of the depth information as an object;
(c) detecting a skin color region from the 3D image;
(d) detecting a candidate group of skin regions by AND combining the labeling region and the skin color region; And
(e) evaluating texture complexity in the candidate group of skin regions to detect the final skin region,
In the step (c), the eye region is detected in the 3D image, a skin map is generated from the detected eye region, and a skin color model is generated from the skin map to extract a skin color region. Information based human skin area detection method.
The method of claim 1,
In the step (a), the depth information-based human skin region detection method, characterized in that to extract the depth information using a graph cuts based stereo matching technique.
The method of claim 1,
In the step (a), the depth-based human skin region is characterized in that the left and right color image of the stereo three-dimensional image is converted into a left and right gray image, and depth information is extracted by applying stereo matching between the left and right gray image. Detection method.
delete The method of claim 1,
In step (c), selecting a sample located in the extended region of the minimum inclusion rectangle (MER) corresponding to the extracted eye region, and generates a skin map as shown in [Formula 1] from the selected sample Depth information-based human skin region detection method, characterized in that.
[Equation 1]

However, C _r and C _b are pixel values of the YC _b C _r space located in the extended area of the minimum containing rectangle (MER) corresponding to the eye area.

and

Are the pixel values of the YC _b C _r space of the normal skin color, respectively. The method of claim 1,
In the step (e), based on the depth information, the texture complexity is calculated by extracting the degree of the edge by applying a Sobel edge operator to the region (hereinafter candidate region) belonging to the candidate group of the skin region Method for detecting human skin area.
The method according to claim 6,
Depth information-based human skin region detection method characterized in that the texture complexity T (R _i ) for the i-th candidate skin region belonging to the candidate group of the skin region is calculated by Equation 2.
[Equation 2]

Where I _gray (x, y) is the contrast at the x and y positions,
w _h (x, y) and w _v (x, y) are the horizontal and vertical masks of the Sobel edge operator,
E (x, y) is the edge at x and y positions,
N (R _i ) is the number of pixels belonging to the region R _i .

Images