JP2013219561A - Imaging device, control method therefor, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately control white balance even in the case of photographing at a changed zoom magnification.SOLUTION: A WB correction unit 103 calculates a first WB correction value on the basis of image data captured by a solid-state image sensor 101. Then, the WB correction unit 103 monitors the calculated first WB correction value during change of zoom magnification at a predetermined interval, and calculates a second WB correction value on the basis of the monitored first WB correction value.

Description

  The present invention relates to a white balance control technique for image data.

  In general, an imaging apparatus using an imaging element such as a digital camera or a digital video camera has a white balance control function for adjusting the color tone of an image obtained by imaging. White balance control includes manual white balance control and automatic white balance control. Manual white balance control is control in which a white subject is imaged in advance and a white balance coefficient is calculated, and the calculated white balance coefficient is applied to the entire screen. On the other hand, the auto white balance control automatically detects a white portion from captured image data, calculates a white balance coefficient from the average value of each color component of the entire screen, and applies the calculated white balance coefficient to the entire screen. Control.

  Here, the conventional auto white balance control will be briefly described. In the conventional auto white balance control, a color evaluation value is obtained for each block of captured image data, and when the color evaluation value is included in the white detection area, it is determined that the block is white. Then, the white balance coefficient is calculated by obtaining the integral value of the pixels of the block determined to be white.

  However, when white is not present in the captured image data, the integrated value of the pixels of the block whose color evaluation value is included in the white detection range is close to zero, and thus a correct white balance coefficient cannot be calculated. Furthermore, when zoom shooting is performed, the probability of white being present is lower than when shooting wide, and it is difficult to specify the color temperature of the environmental light source. For example, when a person's up is photographed by zooming, the ratio of white to the screen decreases.

  In addition, when a person's up is photographed by zooming, the following problems also occur. For example, in a light source such as a fluorescent lamp, human skin color is distributed on the low color temperature side in the white detection range. In other words, in a scene where there is little white on the screen and the human skin is up, by misclassifying the skin color as white under a low color temperature light source, the human skin may become pale, resulting in an image unintended by the photographer. There is sex.

  On the other hand, for example, Patent Document 1 discloses a technique for controlling a control range according to a zoom magnification. In the technique disclosed in Patent Document 1, the control range is wide when the zoom magnification is low, and the control range is narrow when the zoom magnification is high. Thereby, even if the occupation area of the chromatic color in the photographed image data increases, it is possible to suppress the fading phenomenon that makes the chromatic color whitish.

Japanese Patent No. 264168

  However, the technique disclosed in Patent Document 1 may not be able to follow the environmental light source. For example, when the light source is switched after the zoom magnification is changed, the control range may be narrowed to follow the environment light source. Also, even when the occupying area of the chromatic color is large in the scene before the zoom magnification change, there are cases where the environment light source cannot be followed before and after the zoom magnification change.

  Therefore, an object of the present invention is to perform appropriate white balance control even when the zoom magnification is changed and the image is taken.

  The image pickup apparatus of the present invention includes an image pickup unit, a first calculation unit that calculates a first white balance correction value based on image data picked up by the image pickup unit, and a zoom during the image pickup process by the image pickup unit. Changing means for changing the magnification, monitoring means for monitoring the first white balance correction value calculated by the first calculation means during a zoom magnification change by the changing means, and monitoring by the monitoring means And a second calculating means for calculating a second white balance correction value based on the first white balance correction value.

  According to the present invention, appropriate white balance correction can be performed even when the zoom magnification is changed and the image is taken.

It is a figure which shows the structure of the imaging device which concerns on embodiment of this invention. It is a flowchart which shows the calculation process of the 1st WB correction value in a WB control part. It is a figure which shows the graph used in the case of white detection. It is a flowchart which shows the calculation process of the 2nd WB correction value in a WB control part. It is a figure which shows typically the case where a zooming process is performed. It is a figure which shows the relationship between zoom magnification and color temperature.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments to which the invention is applied will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a diagram illustrating a configuration of an imaging apparatus according to an embodiment of the present invention. In FIG. 1, reference numeral 101 denotes a solid-state imaging device composed of a CCD, a CMOS, or the like, and its surface is covered with an RGB color filter such as a Bayer array so that color photography can be performed. When the subject image is formed on the solid-state image sensor 101, the solid-state image sensor 101 generates image data (image signal), and the generated image data is stored in the memory 102.

  The control unit 114 calculates a shutter speed and an aperture value that brighten the entire image, and calculates a driving amount of the focus lens so as to focus on a subject in the focusing area. Then, the exposure value (shutter speed, aperture value) and the driving amount of the focus lens calculated by the control unit 114 are output to the imaging control circuit 113 and controlled based on each value. The white balance (WB) control unit 103 calculates a WB correction value based on the image data stored in the memory 102, and uses the calculated WB correction value to perform WB correction on the image data stored in the memory 102. I do. The details of the method for calculating the WB correction value by the WB control unit 103 will be described later.

  A color conversion matrix (MTX) circuit 104 converts the image data corrected by the WB control unit 103 into color difference signals RY and BY with a color gain so that the image data is reproduced with an optimum color. A low-pass filter (LPF) circuit 105 limits the bands of the color difference signals RY and BY. A CSUP (Chroma Supress) circuit 106 suppresses a false color signal in a saturated portion of the image data band-limited by the LPF circuit 105. On the other hand, the image data that has undergone WB correction by the WB control unit 103 is also output to the luminance signal generation circuit 111. The luminance signal generation circuit 111 generates a luminance signal Y from the input image data. The edge enhancement circuit 112 performs edge enhancement processing on the generated luminance signal Y.

  The RGB conversion circuit 107 converts the color difference signals RY and BY output from the CSUP circuit 106 and the luminance signal Y output from the edge enhancement circuit 112 into RGB signals. A gamma (γ) correction circuit 108 performs gradation correction on the converted RGB signal. The color luminance conversion circuit 109 converts the RGB signal subjected to gradation correction into a YUV signal. The JPEG compression circuit 110 compresses the YUV signal output from the color luminance conversion circuit 109 by the JPEG method or the like, and records it as image data on an external recording medium or an internal recording medium.

  FIG. 2 is a flowchart showing a calculation process of the first WB correction value (first white balance correction value) in the WB control unit 103. Hereinafter, the calculation process of the first WB correction value in the WB control unit 103 will be described in detail with reference to FIG. The process shown in FIG. 2 and the process shown in FIG. 4 to be described later are processes realized by the control unit 114 reading out and executing necessary data and programs from a ROM or the like and controlling the WB control unit 103.

In step S201, the WB control unit 103 reads the image data stored in the memory 102, and divides the image data into arbitrary m blocks. In step S202, the WB control unit 103 calculates a color average value (R [i], G [i], B [i]) by adding and averaging pixel values to each color for each block (1 to m). . Then, the WB control unit 103 calculates a color evaluation value (Cx [i], Cy [i]) using the following formula.
Cx [i] = (R [i] −B [i]) / Y [i] × 1024
Cy [i] = (R [i] + B [i] -2G [i]) / Y [i] × 1024
However, Y [i] = (R [i] + 2G [i] + B [i]) / 4

  The WB control unit 103 performs white detection using a graph having coordinate axes as shown in FIG. In FIG. 3, the negative direction of the x coordinate (Cx) represents a color evaluation value when white of a high color temperature subject is photographed, and the positive direction represents a color evaluation value when white of a low color temperature subject is photographed. The y-coordinate (Cy) means the degree of the green component of the light source, and the green component increases as it goes in the negative direction, that is, it indicates a fluorescent lamp.

  In step S203, the WB control unit 103 determines whether or not the color evaluation value (Cx [i], Cy [i]) of the i-th block calculated in step S202 is included in the preset white detection range 301. Determine. The white detection range 301 is obtained by plotting color evaluation values calculated by photographing white under different light sources in advance along the black body radiation axis. The white detection range 301 can be set differently depending on the shooting mode. For example, when a strobe light is emitted, a known light source can be set with a limited range such as a white detection range 302. If the color evaluation value (Cx [i], Cy [i]) is included in the white detection range 301, the process proceeds to step S204. On the other hand, when the color evaluation values (Cx [i], Cy [i]) are not included in the white detection range 301, the process skips step S204 and proceeds to step S205.

  In step S204, the WB control unit 103 determines that the i-th block is white, and integrates the color average values (R [i], G [i], B [i]) of the block. On the other hand, when step S204 is skipped, the process proceeds to step S205 while the color average value of the block is not integrated (added). Note that the processing of steps S203 and S204 can be expressed by the following equation.

  Here, in the above equation, when the color evaluation value (Cx [i], Cy [i]) is included in the white detection range 301, Sw [i] is set to 1, and the color evaluation value (Cx [i], If Cy [i]) is not included in the white detection range 301, Sw [i] is set to zero. Thereby, in steps S203 and S204, the color average values (R [i], G [i], B [i]) are substantially integrated or not integrated.

  In step S205, the WB control unit 103 determines whether the processes in steps S202 to S204 have been executed for all blocks. If the process has been executed for all blocks, the process proceeds to step S206. On the other hand, if the process has not been executed for all the blocks, the process returns to step S202.

In step S206, the WB control unit 103 calculates a first WB correction value (WBCol_R1, WBCol_G1, WBCol_B1) using the following formula from the obtained integrated value (SumR1, SumG1, SumB1) of the color evaluation values. To do. Step S206 is a processing example of the first calculation unit.
WBCol_R1 = SumY1 × 1024 / SumR1
WBCol_G1 = SumY1 × 1024 / SumG1
WBCol_B1 = SumY1 × 1024 / SumB1
However, SumY1 = (SumR1 + 2 × SumG1 + SumB1) / 4

  FIG. 4 is a flowchart showing the calculation process of the second WB correction value (second white balance correction value) in the WB control unit 103. Hereinafter, the calculation process of the second WB correction value in the WB control unit 103 will be described in detail with reference to FIG.

  In step S401, the WB control unit 103 stores the WB correction value before the zoom magnification change. For example, the WB control unit 103 stores a WB correction value calculated in the past when EVF (Electronic View Finder) is driven. The WB correction value stored here is obtained by the first WB correction value calculation process shown in FIG.

  In step S402, the WB control unit 103 determines whether or not the zoom magnification has been changed and shot. When the zoom magnification is changed and the image is taken, the process proceeds to step S403. On the other hand, if the image has been shot without changing the zoom magnification, the process proceeds to step S407. In step S407, the WB control unit 103 performs normal WB control using the WB correction value (first WB correction value) stored in step S401.

  In step S403, the WB control unit 103 calculates and monitors WB correction values at predetermined intervals during zoom magnification change. Each WB correction value monitored at a predetermined interval is stored. In the present embodiment, the WB correction value is monitored according to a predetermined zoom position. For example, in the case of a 28 mm-560 mm angle of view in terms of 35 mm, the WB correction value is monitored according to a predetermined zoom position such as 28 mm, 56 mm, 84 mm,. Note that the monitoring interval of the WB correction value may be varied according to the zoom speed, or the WB correction value may be monitored at a predetermined time interval.

  After the zoom magnification is changed, in step S404, the WB control unit 103 sets a WB control range (WB correction value range). The setting of the WB control range here is performed based on the WB correction value before the zoom magnification change. For example, a range having a predetermined width around the WB correction value before the zoom magnification change may be set as the WB control range. The setting of the WB control range may be variable depending on the zoom magnification before or after the zoom magnification change. For example, when the zoom magnification is high, the occupying area of the chromatic color increases, and the possibility that the chromatic color is determined to be white increases. Therefore, the WB control range is set to be narrow.

  In step S405, the WB control unit 103 determines whether the scene has changed during or after the zoom magnification change. Here, as a determination element for a scene change, a change in the scene is detected by monitoring changes in information obtained from the gyro sensor and brightness (photometric value) information of the environment light source. That is, when information obtained from the gyro sensor is used, it is possible to detect whether or not the image is panned and photographed. When the brightness (photometric value) information of the environment light source is used, the change of the environment light source is detected. be able to. If the scene has not changed, the process proceeds to step S406. On the other hand, if the scene has changed, the process proceeds to step S407 described above.

  In step S406, the WB control unit 103 calculates a second WB correction value based on the WB control range set in step S403 and the monitoring result monitored in step S404. Step S406 is a processing example of the second calculation unit.

  Hereinafter, the second WB correction value calculation process in step S406 will be described in detail with reference to FIGS. In FIGS. 5 and 6, zooming is performed as zoom (0) → zoom (1) → zoom (2) → zoom (3) → zoom (4).

  FIG. 5A schematically shows a case where the zooming process is performed on the face area of the main subject 501. As shown in FIG. 5A, when the zoom process is performed on the face area of the main subject 501, the ratio of false detection that the face color is a low color temperature light source gradually increases. The color temperature is reduced. That is, the state transition of the color temperature detected here is as shown in FIG. As shown in FIG. 6A, in the case of a state transition that tends to gradually decrease from the color temperature (WB correction value) detected before the zoom magnification change, the WB correction value before the zoom magnification change is more reliable. It is determined that the degree is high. Therefore, the WB correction value before the zoom magnification change is used as the second WB correction value. Further, contrary to the example shown in FIG. 6A, even in the case of a state transition that tends to gradually increase from the detected color temperature (WB correction value) before the zoom magnification change, before the zoom magnification change. It is determined that the WB correction value is more reliable. Therefore, in this case, the WB correction value before the zoom magnification change is used as the second WB correction value. In addition, when it is determined that the reliability of the WB correction value before the zoom magnification change is higher, the upper limit value or the lower limit value of the WB control range or a value in the vicinity thereof among the WB correction values calculated during the zoom magnification change. The WB correction value corresponding to the above may be output as the second WB correction value.

  Also, as shown in FIG. 5B, when there are subjects 502 of various colors in the background, the reliability as the WB correction value may be low before the zoom magnification change. That is, as shown in FIG. 6B, when the detected color temperature (WB correction value) varies as the state transition, it is determined that the reliability of the WB correction value before the zoom magnification change is low. The WB correction value included in the WB control range is weighted and added to obtain the second WB correction value. That is, the WB control unit 103 calculates the second WB correction value using the following equation.

  Here, in the above formula, when the WB correction value (Cx [j], Cy [j]) monitored when the zoom magnification is changed is included in the WB control range shown in FIG. 6B, Sw [j] is Set to 1. On the other hand, if the WB correction value (Cx [j], Cy [j]) is not included in the WB control range shown in FIG. 6B, Sw [j] is set to 0. As a result, processing is performed to determine whether or not to add the WB correction value.

  As described above, an appropriate white balance correction value can be obtained even when the zoom magnification is changed and an image preferable for the user can be provided.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

  101: Fixed imaging device, 102: Memory, 103: WB control unit, 104: Color conversion matrix circuit, 105: LPF circuit, 106: CSUP circuit, 107: RGB conversion circuit, 108: γ correction circuit, 109: Color luminance conversion Circuit: 110: JPEG compression circuit, 111: Luminance signal generation circuit, 112: Edge enhancement circuit, 113: Imaging control circuit, 114: Control unit

Claims (11)

Imaging means;
First calculation means for calculating a first white balance correction value based on image data picked up by the image pickup means;
Changing means for changing a zoom magnification at the time of image pickup processing by the image pickup means;
Monitoring means for monitoring the first white balance correction value calculated by the first calculating means at a predetermined interval during the zoom magnification change by the changing means;
An image pickup apparatus comprising: a second calculation unit that calculates a second white balance correction value based on the first white balance correction value monitored by the monitoring unit. Further comprising setting means for setting a range of white balance correction values based on the first white balance correction value calculated by the first calculation means before the zoom magnification change by the changing means;
The second calculating unit is configured to determine a second white balance correction value based on the first white balance correction value monitored by the monitoring unit and the range of the white balance correction value set by the setting unit. The imaging apparatus according to claim 1, wherein the imaging device is calculated.   The imaging apparatus according to claim 2, wherein the setting unit sets a range of the white balance correction value according to a zoom magnification before or after the zoom magnification is changed by the changing unit. A determination means for determining a reliability of the first white balance correction value calculated by the first calculation means before the zoom magnification change by the changing means;
The second calculating means uses the first white balance correction value calculated by the first calculating means before the zoom magnification change by the changing means in accordance with the determination result by the determining means. The imaging apparatus according to claim 1, wherein the imaging apparatus calculates the white balance correction value. A determination means for determining a reliability of the first white balance correction value calculated by the first calculation means before the zoom magnification change by the changing means;
The second calculation unit is configured to determine the white balance among the first white balance correction values calculated by the first calculation unit during the zoom magnification change by the change unit according to the determination result by the determination unit. The first white balance correction value corresponding to an upper limit value, a lower limit value, or a value in the vicinity thereof in the correction value range is calculated as the second white balance correction value. Imaging device. A determination means for determining a reliability of the first white balance correction value calculated by the first calculation means before the zoom magnification change by the changing means;
The second calculation unit is configured to determine the white balance among the first white balance correction values calculated by the first calculation unit during the zoom magnification change by the change unit according to the determination result by the determination unit. The imaging apparatus according to claim 2, wherein the second white balance correction value is calculated based on a first white balance correction value belonging to a range of correction values. A detecting means for detecting a change in the scene during or after the zoom magnification change by the changing means;
The said 2nd calculation means calculates the said 2nd white balance correction value, when the change of a scene is not detected by the said detection means, The one of Claims 1 thru | or 6 characterized by the above-mentioned. Imaging device.   The imaging apparatus according to claim 7, wherein the detection unit detects a change in a scene based on information obtained from a gyro sensor.   The imaging apparatus according to claim 7, wherein the detection unit detects a change in a scene based on brightness of an environmental light source. A method for controlling an imaging apparatus having imaging means,
A first calculation step of calculating a first white balance correction value based on image data imaged by the imaging means;
A change step of changing a zoom magnification at the time of image pickup processing by the image pickup means;
A monitoring step of monitoring the first white balance correction value calculated by the first calculation step at a predetermined interval during the zoom magnification change by the changing step;
And a second calculation step of calculating a second white balance correction value based on the first white balance correction value monitored in the monitoring step. A program for causing a computer to execute a control method of an image pickup apparatus having an image pickup means,
A first calculation step of calculating a first white balance correction value based on image data imaged by the imaging means;
A change step of changing a zoom magnification at the time of image pickup processing by the image pickup means;
A monitoring step of monitoring the first white balance correction value calculated by the first calculation step at a predetermined interval during the zoom magnification change by the changing step;
A program for causing a computer to execute a second calculation step of calculating a second white balance correction value based on the first white balance correction value monitored in the monitoring step.

Images