KR101411309B1 - Digital image processing apparatus which can determine the eye's open-close rate comprised in image - Google Patents

Abstract

The present invention relates to a digital image processing apparatus capable of efficiently managing an image by determining the degree of opening and closing of eyes included in a photographed image, A detection unit; An area ratio calculating unit for calculating a ratio of an area occupied by the pupil in the entire area of the detected eyes; And an eye blinking image detecting unit for detecting an image in which the area ratio is equal to or smaller than a predetermined value.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to a digital image processing apparatus capable of determining an open / close degree of eyes included in an image,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital image processing apparatus capable of efficiently managing an image by determining the degree of opening and closing of eyes included in an image.

2. Description of the Related Art In recent years, digital photographing devices such as a cellular phone, a personal digital assistant (PDA), and the like, including a digital camera, have been widely used. With the spread of such digital photographing apparatus, the use of the digital photographing apparatus is not limited to simply taking a picture, and various multimedia functions such as a potable multimedia player (PMP), an MP3 player, and a digital multimedia broadcasting (DMB)

Since the built-in or built-in memory card for storing the photographed image has to be allocated not only the storage space of the image but also the function for performing the function as the multimedia according to the diversity of the functions for storing the image, A high-capacity, highly integrated memory card is required.

However, since there is not enough space to store the photographed image when the number of images to be photographed is large, a method for more efficient image management needs to be constantly searched even if the capacity of the memory card is increased and the density is increased.

The present invention has been made to solve the above problems and it is an object of the present invention to provide a digital image processing apparatus capable of efficiently managing an image by determining the degree of opening and closing of eyes included in an image.

The present invention provides a digital image processing apparatus capable of determining the degree of opening and closing of eyes included in a photographed image.

According to another aspect of the present invention, the degree of opening / closing of the eye can be determined by a ratio of the area occupied by the pupil including the iris and the pupil to the area of the entire eye along the contour of the eye.

According to still another aspect of the present invention, the ratio of the area can be determined by a ratio of the color area occupied by the color of the pupil to the area of the entire eye along the contour of the eye.

According to another aspect of the present invention, the digital image processing apparatus may automatically delete an image including eyes whose color area ratio is less than or equal to a predetermined value.

According to another aspect of the present invention, the digital image processing apparatus may further include display means for classifying and displaying an image including eyes having the color area ratio equal to or less than a predetermined value.

According to another aspect of the present invention, in the digital image processing apparatus, a user can selectively delete an image among images displayed on the display means.

According to another aspect of the present invention, there is provided an image processing apparatus including an eye detecting unit detecting an eye included in an image using a face recognition technique; An area ratio calculating unit for calculating a ratio of an area occupied by the pupil in the entire area of the detected eyes; And an eye blinking image detecting unit for detecting an image in which the area ratio is equal to or smaller than a predetermined value.

According to another aspect of the present invention, the area ratio calculation unit may calculate an area ratio of a color occupied by the pupil in the entire area of the detected eyes.

According to another aspect of the present invention, the digital image processing apparatus may further include an eye blinking image deletion unit that deletes the image having the area ratio smaller than a predetermined value.

According to another aspect of the present invention, the blinking image deletion unit may delete an image whose predetermined elapsed time has elapsed.

According to the digital image processing apparatus of the present invention as described above, the ratio of the area occupied by the pupil in the entire area of the eye included in the photographed image is calculated to efficiently detect unnecessary images to close or close the eyes , These detected images can be deleted from the storage device, and the storage space can be efficiently used.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a view schematically showing a configuration of a digital image processing apparatus according to an embodiment of the present invention. The digital image processing apparatus according to the present embodiment can be applied to various image processing apparatuses including a digital camera, a mobile phone equipped with a camera module, a PDA, and the like, but a digital camera will be described as an example in this embodiment. Also, it goes without saying that the present invention can be applied to a digital camera as well as a compact digital camera shown in the drawing, as well as a single lens reflex (SLR) camera.

A digital camera 100 that captures an image and generates and stores an image file includes a shutter-release button 11, a power button 13, a wide angle-zoom button 15w, a telephoto button, A zoom 15t, a function button 17, a display unit 19, and the like.

The shutter-release button 11 is opened and closed to expose a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) to light for a predetermined period of time and interlocked with an iris (not shown) And records the image on the CCD.

The power button 13 is input to supply power to the image processing apparatus to operate.

The wide angle-zoom button 15w or the telephoto-zoom 15t widens the angle of view or narrows the angle of view depending on the input, in particular, when changing the size of the selected exposure area. When the wide angle-zoom button 15w is input, the size of the selected exposure area is reduced, and when the telephoto-zoom button 15t is input, the size of the selected exposure area is increased.

The function button 17 includes a total of five buttons including an up button, a down button, a left button, a right button, and a menu / OK button. The function buttons 17 are input to execute various menus related to the operation of the digital image processing apparatus 100, and the respective keys may be used as shortcut keys.

2 is a block diagram showing a configuration of a digital image processing apparatus according to the present embodiment and includes a power supply unit 110, a user input unit 120, a photographing unit 130, an image processing unit 140, a storage unit 150, (160) and a controller (170).

The power supply unit 110 supplies power to the digital image processing apparatus 100 to operate the power supply, or stops the power supply to the digital image processing apparatus 100 by shutting off the power.

The user input unit 120 inputs signals for controlling various operations of the digital camera by a user's operation and includes a shutter-release button 11 (see FIG. 1), a power button 13, a wide-angle zoom button 15w Or a telephoto-zoom button 15t and a function button 17 or the like.

The photographing unit 130 converts an optical signal of an image of a subject into an electric signal, and includes an optical system including a lens unit (not shown) and a filter unit (not shown) and an image sensor (not shown). The lens portion may include a zoom lens, a focus lens, and a compensation lens. An image sensor (not shown) such as a CCD or a CMOS accumulates an amount of light input through the lens unit and outputs an image photographed by the lens unit according to the accumulated light amount, in accordance with a vertical synchronizing signal. Image acquisition by a digital image processing apparatus is performed by an image sensor such as a CCD, which converts light reflected from a subject into an electrical signal. In order to obtain a color image using an image sensor, a color filter is required. Mostly, a filter (not shown) called a CFA (color filter array) is employed. The CFA has a regularly arranged structure that passes only light representing one color per pixel, and has various shapes according to the arrangement structure.

An analog image signal output from an image sensor (not shown) is converted into a digital signal by an A / D converter (not shown), and the digital image signal is converted into an original ) Data.

The image processing unit 140 processes the digitally converted RAW data so that it can be displayed. The image processing unit 140 removes the black level due to the dark current generated in the CCD and CFA filters, which are sensitive to the temperature change. In addition, gamma correction is performed to encode information in accordance with nonlinearity of human vision, and CFA interpolation is performed to interpolate Bayer patterns embodied with gamma corrected predetermined data RGRG lines and GBGB lines into RGB lines. Also, the color values of the U and V signals are corrected using the standard color coordinate system, and the noise is removed by converting the interpolated RGB signal into the YUV signal, edge compensation processing the Y signal by filtering the Y signal by the high band filter, , And compresses and processes the Y, U, and V signals from which noise has been removed to generate an image file such as a JPEG file.

The generated image file is stored in the storage unit 150 such as a memory card or displayed on the display unit 160 according to the classification according to the user setting.

The storage unit 150 may store a memory card of various sizes such as a smart card, a Compact flash (CF) memory, a memory stick, and a secure digital (SD) memory card in order to finally store the image file .

In addition, the digital image processing apparatus used in the present invention may include an EEPROM (Electrically Erasable and Programmable Read Only Memory) and a digital A flash memory for storing setting data necessary for the operation of the camera processor, and the like.

All operations of the image processing unit 140 operate under the control of the control unit 170. The control unit 170 detects the eyes using the face recognition technique in the photographed image, calculates the area ratio occupied by the pupil in the entire area of the eyes included in the image, detects the images having the predetermined area ratio From the storage unit. The control unit 170 includes an eye detection unit 171, an area ratio calculation unit 172, a blinking image detection unit 173, and a blinking image deletion unit 174.

The eye (eye) detecting unit 171 detects whether or not an image is included in the photographed image using the face recognition technique.

In order to detect the eyes from the photographed images using the face recognition technology, a principal component analysis (PCA) -based Eigenface method in which the determinant of the covariance matrix is maximized through a linear transformation, data belonging to the same class are gathered , And Fisher's face method based on FLD (Fisher's linear discriminant) which obtains the conversion of data belonging to another class into a space far from each other can be used.

The area ratio calculation unit 172 calculates the ratio of the area occupied by the pupil to the area of the entire eye along the contour of the detected eye to determine the degree of opening and closing of the eye included in the image. Here, the pupil is a concept that includes iris and pupil.

In the case of a photograph with a portrait, the subject usually shoots a photo with the eyes wide open toward the front, but occasionally the photograph is taken at the moment of closing or closing the eyes. In the case of a photograph taken at the moment of closing or closing the eyes, the area of the pupil with respect to the area of the entire eye is smaller than that of the case where the eye is properly opened.

At this time, the area ratio of the pupil to the whole eye can be calculated using the characteristic that the white part of the eye and the pupil have different colors. For example, the margin of the eye excluding the iris and the pupil (white part) is usually white, whereas the pupil including the iris and pupil is colored. These pupils may have various colors such as black, brown, blue and green depending on race. Therefore, the area ratio calculation unit 172 determines whether the eye is opened or closed by determining the ratio of the area of the color of the pupil portion to the area of the white portion of the white portion of the eye.

Eye blinking image detecting unit 173 detects an image of the photographed images which are to be closed or to be wiped, based on the values calculated by the area ratio calculating unit 172. [ Here, the term "blinking image" refers to an image in which an eye is closed or closed, and more precisely, an image in which the area ratio of the pupil in the entire eye area is equal to or smaller than a predetermined value. The term "blinking" is used below to denote a state in which the eyes are closed and / or a state in which the eyes are to be closed.

If the ratio of the area of the pupil relative to the area of the entire eye derived by the area ratio calculation unit 172 is a value set by the user or is equal to or smaller than a predetermined value determined in advance by the program, the eye blinking image detecting unit 173 detects, As a blinking image. Here, as a predetermined value, which is a criterion for judging the blinking image, for example, in the case of a photograph properly taken without sensing the eyes, considering the case that the ratio of the area of the pupil to the total area of the eyes is 50 or more, Images with a predetermined value of 50 or less can be detected. Of course, the predetermined value is only one example, and it is needless to say that the predetermined value can be changed to a desired value by the setting of the user or by the program.

The above-mentioned contents will be described in more detail with reference to Figs. 3 and 4. Fig. FIG. 3 is a conceptual diagram schematically showing a step of detecting an eye from a photographed image, and FIG. 4 is a conceptual diagram schematically showing a step of calculating a ratio of an area of a pupil to an area of an entire eye.

Referring to FIG. 3, first, the eye detecting unit 171 detects an eye part IV using various methods described above from the photographed image.

Fig. 4 is an enlarged view of the eye part IV in Fig. 3, in which the entire eye along the contour of the eye is divided into a pupil A including an iris a1 and a pupil a2, And a white part (B).

The area ratio calculation unit 172 calculates the ratio of the area of the pupil portion A to the area A + B of the entire eye using the area of the whitish portion B of the eye and the area of the pupil portion A do. At this time, the color of the whitish portion (B) and the pupil portion (A) of the eye can be analyzed and the above-mentioned area ratio can be calculated using the area of the color of each portion. Here, the whitish portion (B) of the eye does not necessarily have an RGB value that comes to the center of the color coordinate system, but it is acceptable if it has an RGB value included in a predetermined range of the color coordinate system so as to give the observer a perception of a white system. In addition, since the color of the pupil portion A can have various colors such as black, brown, blue, and green depending on the race as described above, it is possible for the observer to know that the color of the pupil portion is the white portion B of the eye It is acceptable if it has an RGB value included in a predetermined range of the color coordinate to such an extent that it is recognizable that the color has a distinctive color from the white system.

If the result of the area ratio calculated by the area ratio calculating unit 172 is equal to or less than a predetermined value, for example, 50 or less, the eye blinking image detecting unit 173 determines that the eye is closed or the image is to be wrapped do. On the other hand, an image judged as a blinking image may be automatically displayed by the blinking image deletion unit 174 or may be selectively displayed by the user after being displayed on the display unit 160.

FIG. 5 shows images displayed by the blinking image detecting unit on the display unit in which the area ratio is equal to or smaller than a predetermined value.

For example, it is assumed that four images having an area ratio of 50 or less are detected by the area ratio detecting unit 172. Among these images, there are images P1 taken while the eyes are closed by mistake, images P3 taken at the moment of closing the eyes, but also images laughing P2 and winking images P4 . The user may not want to delete the laughing image or the winking image. Therefore, when the blinking image deletion unit 174 automatically deletes images having an area ratio less than a predetermined value, it is possible to quickly delete unnecessary images when the storage space is insufficient by quickly deleting images. However, It is possible to prevent the images including smiley faces and winking faces from being deleted by the user.

The photographs P1, P2, P3 and P4 illustrated in the figure include only one person's face. However, the present invention is applicable to a case where a plurality of persons are included.

FIG. 6 shows a method of selecting a type of images that the user desires to delete among the images taken including the present embodiment. Referring to the drawings, the user is shown on the display unit with a blinking image search menu 6A, an image search menu 6B with a lapse of a predetermined time, and an in-focus image search menu 6C. In other words, the user can search for not only blinking images according to the present embodiment, but also old images, which are unnecessarily occupying the storage space, or images that are out of focus, and delete them from the storage unit.

 According to the digital image processing apparatus of the present invention, the ratio of the area occupied by the pupil in the entire area of the eye included in the photographed image can be calculated to efficiently detect unnecessary images to close or close the eyes. The detected images can be deleted from the storage device, and the storage space can be efficiently used.

Further, the digital image processing apparatus according to the present invention is not limited to the above-described image photographing apparatus, and is not limited to the name as long as it can store and reproduce images such as personal computers, electronic albums, digital photo frames, and kiosks. Applicable.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. Accordingly, the true scope of the present invention should be determined by the appended claims

1 is a view schematically showing a configuration of a digital image processing apparatus according to an embodiment of the present invention.

2 is a block diagram showing a configuration of a digital image processing apparatus according to the present embodiment.

 3 is a schematic conceptual diagram showing a step in which an eye is detected from a photographed image.

4 is a schematic conceptual diagram showing a step of calculating the area ratio of the pupil to the entire eye area.

FIG. 5 shows images displayed on the display unit, the areas of which are detected by the eye blinking image detecting unit, and the area ratio is less than a predetermined value.

Fig. 6 shows various image retrieval methods to which the present embodiment is applied.

Claims (10)

It is possible to determine the degree of opening and closing of eyes included in the photographed image, The degree of opening and closing of the eye is determined by the ratio of the area occupied by the pupil including the iris and the pupil to the area of the entire eye along the contour of the eye, The ratio of the area is determined by the ratio of the area of the color of the pupil occupied by the color of the pupil to the area of the entire eye along the outline of the eye, The color of the pupil is a color including black, brown, blue, and green predetermined according to the color of a rabbit pupil, And judges that an image of which the ratio of the area of the color of the pupil is not more than 50 to an image which is closed or the eyes are to be closed. delete delete The method according to claim 1, Wherein the digital image processing apparatus automatically deletes an image including an eye having an area ratio of the color of the pupil of 50 or less. The method according to claim 1, Wherein the digital image processing apparatus further comprises display means for classifying and displaying an image including an eye having an area ratio of the color of the pupil of 50 or less. 6. The method of claim 5, Wherein the digital image processing apparatus can selectively delete an image among images displayed on the display means by a user. An eye detector for detecting eyes included in the image using a face recognition technology; An area ratio calculating unit for calculating a ratio of an area of a color of the pupil occupied by the pupil in the entire area of the detected eyes; And And an eye blinking image detecting unit for detecting an eye blinking image by determining an image in which the ratio of the area of the color of the pupil is 50 or less to an image for closing or closing the eyes, Wherein the color of the pupil when the area ratio calculation unit calculates the area ratio of the color of the pupil includes color of black, brown, blue, and green predetermined according to the color of the rabbit pupil. delete 8. The method of claim 7, Wherein the digital image processing apparatus further comprises an eye blinking image deleting unit for deleting an image having an area ratio of the color of the pupil of 50 or less. 10. The method of claim 9, Wherein the blinking image deletion unit deletes an image in which a predetermined photographing elapsed time has been exceeded.

Images