JP5245648B2 - Image processing apparatus and program - Google Patents

Description

  The present invention relates to an image processing apparatus and a program for performing a so-called multiplane addition process in which a plurality of images obtained by photographing the same subject are superimposed.

  Since the dynamic range of solid-state image sensors used in digital cameras, video cameras, etc. (hereinafter referred to as cameras) is narrow, it is difficult to set appropriate exposure for all subjects when imaging subjects with large differences in brightness. It is. Therefore, when the exposure is adjusted to the dark portion, the bright portion is whitened (whiteout), and conversely, when the exposure is adjusted to the bright portion, the dark portion is blackened (black crushing). Therefore, by performing a so-called multi-plane addition process that superimposes a short exposure image obtained by photographing the same subject with a short exposure time and a long exposure image obtained by a long exposure time, an appropriate exposure image is obtained for both bright and dark portions. An image processing apparatus and method for obtaining the above are known (for example, Patent Document 1).

JP 2001-008216 A

  However, when the long exposure image is simply superimposed on the short exposure image and the luminance is added, an appropriate image is obtained for the dark portion of the short exposure image, but overexposure occurs in the bright portion of the short exposure image. . In other words, the simple multi-plane addition processing has a problem that black crushing is eliminated but whiteout is not eliminated.

  The present invention has been made in view of such a background, and provides an image processing apparatus and a program that eliminates whiteout and blackout that occur when a subject having a large difference in brightness is imaged.

An image processing apparatus of the present invention is made to solve the above-described problem, and includes an imaging unit, an imaging control unit that controls the imaging unit to continuously shoot the same subject under the same imaging conditions, A selection unit that selects a reference image from a plurality of images obtained as a result of continuous shooting by the imaging control unit, and a luminance that calculates a luminance distribution by analyzing pixels included in the reference image selected by the selection unit Based on the distribution calculation means and the luminance distribution of the reference image calculated by the luminance distribution calculation means, a threshold value for dividing the reference image into a bright area and a dark area is determined, and the dark area of the reference image is determined. a dark region specifying unit for specifying, in the dark region of the reference image specified by the dark area specifying means, superimposing images of the region corresponding to the dark region in the plurality of images other than the reference image Characterized in that it comprises a that image polymerized unit.

  Further, the threshold value may be a median value of luminance in the luminance distribution.

In addition, the image superimposing unit multiplies the luminance of a pixel to which an image in the region corresponding to the dark region in a plurality of images other than the reference image belongs by a positive coefficient, and corresponds to the dark image in the reference image. While superimposing images of regions in a plurality of images other than the reference image, the coefficient may be obtained by a function that gradually increases as the luminance of the pixel moves away from the threshold value.
Further, the selection unit may select the reference image from images excluding the first image and the last image from among the plurality of continuously shot images.
Furthermore, the selection means may select the reference image from the plurality of continuously shot images based on an arbitrary operation from the outside.

Further, the program of the present invention, the computer comprising an image processing apparatus having an imaging unit, an imaging control unit that controls to continuous shooting the same subject at the same photographing condition to the image pickup unit, the imaging control unit A selection means for selecting a reference image from a plurality of images obtained as a result of continuous shooting by means of, a luminance distribution calculation means for calculating a luminance distribution by analyzing pixels included in the reference image selected by the selection means, and Based on the luminance distribution of the reference image calculated by the luminance distribution calculating means, a dark area specifying means for determining a threshold value for classifying the reference image into a bright area and a dark area and specifying the dark area of the reference image , the dark region of the reference image specified by the dark area specifying means, superimposing the image of the region corresponding to the dark region in the plurality of images other than the reference image It is intended to function as an image polymerization unit.

  According to the present invention, in the so-called multi-plane addition process, blackout and whiteout are eliminated, and a high-quality image can be obtained.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Embodiment 1]
FIG. 1 is a diagram illustrating a configuration of a digital camera device 1 according to the embodiment. As shown in FIG. 1, the digital camera device 1 includes an optical lens device 2, a CCD 3, an A / D converter 4, an image storage unit 5, a reference image selection circuit 6, a reference image storage unit 7, an additional image storage unit 8, A luminance histogram analysis circuit 9, a positional deviation correction circuit 10, an addition processing circuit 11 and a control unit 12 are provided.

  The optical lens device 2 includes a lens, a diaphragm, a focus adjustment device, a shutter device, and the like, and forms an image of a subject on the CCD 3.

  The CCD 3 is a so-called solid-state imaging device, which decomposes an image formed on the CCD 3 into a large number of pixels, converts the luminance of each pixel into a voltage level, and outputs the voltage to the A / D converter 4.

  The A / D converter 4 converts the luminance of each pixel input as a voltage strength into a digital signal and sends it to the image storage unit 5.

  The image storage unit 5 is a storage device that stores the image sent from the A / D converter 4. That is, the image storage unit 5 stores the subject image captured by the optical lens device 2 in the form of a set of digital signals. The digital camera device 1 has a continuous shooting function and is configured to automatically take a plurality of images by pressing the shutter button once. In addition, since photographing of the subject is performed a plurality of times at predetermined time intervals, a plurality of substantially identical images obtained by photographing the same subject are stored in the image storage unit 5.

  The reference image selection circuit 6 reads a plurality of images taken of the same subject from the image storage unit 5, selects an image located in the center when the images are arranged in time series, and stores the reference image as a reference image. Send to part 7. A plurality of images other than the reference image (hereinafter referred to as additional images) are sent to the additional image storage unit 8.

  The luminance histogram analysis circuit 9 reads the reference image from the reference image storage unit 7, classifies the pixels of the reference image into a plurality of luminance classes having a certain luminance width, and distributes the number of pixels belonging to each luminance class ( Luminance histogram). A median value of the luminance histogram is calculated, and this value is set as a threshold value for dividing the reference image into a bright region and a dark region. Then, an area having pixels equal to or smaller than the threshold value, that is, an image of a dark area is sent to the addition processing circuit 11.

  Since the position of the subject in the shooting range may be different between the reference image and the additional image, the positional deviation correction circuit 10 reads the reference image and the additional image from the reference image storage unit 7 and the additional image storage unit 8, respectively. Correction is performed so that the position of the subject in the additional image is the same as the position of the subject in the reference image. Then, the corrected additional image is sent to the addition processing circuit 11.

  The addition processing circuit 11 reads the additional image corrected by the positional deviation correction circuit 10 and specifies the area of the additional image corresponding to the dark area. Thereafter, the luminance of the pixel in the additional image area corresponding to the dark area is added to the luminance of the pixel in the reference image to generate a composite image.

  The control unit 12 is a computer that controls the digital camera device 1 and controls the addition processing circuit 11 and the like according to a program written in a storage unit (not shown) to execute image processing and the like.

  Next, FIG. 2 is a flowchart illustrating an image processing procedure executed by the control of the control unit 12 when the shutter button of the digital camera device 1 is pressed. As described above, the digital camera device 1 has a continuous shooting function, and in the present embodiment, it is assumed that the same subject is photographed three times when the shutter button is pressed. Moreover, since the control part 12 controls each part of the digital camera apparatus 1, every reference of the control part 12 is abbreviate | omitted. Hereinafter, these image processing procedures will be described with reference to FIG.

  First, when the shutter button is pressed, three images photo_10, photo_11 and photo_12 as shown in FIG. 3 are taken and stored in the image storage unit 5 (step S11).

  Next, the reference image selection circuit 6 selects the image photo_11 taken second from the three images photo_10, photo_11, and photo_12 as a reference image (step S12). The reference image photo_11 is sent to the reference image storage unit 7, and the images photo_10 and photo_12 are sent to the additional image storage unit 8.

  Next, the luminance histogram analysis circuit 9 analyzes the reference image photo_11 and generates a luminance histogram as shown in FIG. 4 (step S13).

  Further, the luminance histogram analysis circuit 9 calculates a median luminance from the luminance histogram shown in FIG. The value is set as a threshold value b, and an area having a luminance equal to or lower than the threshold value b in the reference image photo_11 is set as a dark area (step S14). In the present embodiment, the reference image photo_11 of the reference image photo_11 shown in FIG. 5 includes the mountain 21, the tree 22, and the background 23 as dark areas, the sun 24, the track 25, the person 26, and the box 27 as bright areas, and the reference image photo_11 includes two areas. It shall be classified into

  Next, the positional deviation correction circuit 10 reads the reference image photo_11 and the additional images photo_10 and photo_12 from the reference image storage unit 7 and the additional image storage unit 8, and corrects the additional images photo_10 and photo_12 with reference to the reference image photo_11. The corrected photo_10 and photo_12 are sent to the addition processing circuit 11 (step S15).

  Finally, as shown in FIG. 6, the addition processing circuit 11 extracts the mountain 21, the tree 22, and the background 23 from the additional images photo_10 and photo_12, and generates photo_13 and photo_14. Further, a composite image photo_15 is generated by superimposing photo_13 and photo_14 on the reference image photo_11 (step S16).

  As described above, according to the first embodiment, since the area of the additional image corresponding to the dark area of the reference image is superimposed on the reference image, it is possible to eliminate black crushing and whiteout.

  Note that the number of images to be combined is not limited to three. Further, the same image may be duplicated to obtain an additional image. The reference image may be arbitrarily selected from a plurality of photographed images by the user.

  Note that the threshold value b is not limited to the median value of luminance in the luminance histogram. For example, the threshold value b may be an average value of luminance, or the luminance of the luminance class having the maximum number of pixels in the luminance histogram may be set as the threshold value b.

  Although an example in which the image processing apparatus is configured by combining dedicated hardware is shown here, the present invention is not limited to such a configuration. For example, all or part of the image processing apparatus may be configured by software. Further, the camera body portion (optical lens device 2, CCD 3, A / D converter 4, and image storage unit 5) may be separated from other portions. Further, the computer may function as the image processing apparatus of the present invention by causing the computer to read and execute a program describing the image processing method according to the present invention.

[Embodiment 2]
In the first embodiment, an additional image area corresponding to the dark area of the reference image is superimposed on the reference image to synthesize the image. In this case, the luminance corresponding to the number of additional images is a pixel belonging to the dark area of the reference image. Therefore, the difference in luminance between the pixels near the threshold value in the dark region and the pixels near the threshold value in the bright region becomes large, and there is a possibility that a discontinuity in luminance occurs in the image after the addition process. In order to avoid this, as shown in FIG. 7, in the second embodiment, step S21 is added to the procedure of the first embodiment, and a positive weighting factor is applied to the luminance of the pixels belonging to the area of the additional image corresponding to the dark area. Multiplane addition processing is performed by multiplying by.

The positive weighting factor is obtained by a function in which the luminance of the pixel belonging to the region of the additional image corresponding to the dark region gradually increases as the distance from the threshold increases, and this function is 0 when the luminance is equal to the threshold, and the luminance is When equal to 0, it becomes 1. In the present embodiment, the weighting function y (x) represented by the following formula (1) is used using the luminance x and the threshold value b.

  When the weighting function y (x) is superimposed and displayed on the luminance histogram of the additional image, the result is as shown in FIG. FIG. 8B shows a luminance histogram of an image obtained by multiplying the luminance of the pixel belonging to the additional image region corresponding to the dark region by the weight function y (x). As shown in FIG. 8B, the luminance decreases as the pixel has a luminance close to the threshold value b. When the luminance multiplied by the weight function y (x) is added to the luminance of the pixels in the dark area of the reference image in this way, the influence of the addition is reduced for pixels near the dark area threshold value b.

  As described above, in the present embodiment, since the luminance function of the pixel belonging to the area of the additional image corresponding to the dark area is multiplied by the weighting function y (x) and the addition process is performed, the luminance discontinuity of the pixels near the threshold is reduced. It is eliminated and a natural image with no sense of incongruity in brightness is obtained.

[Embodiment 3]
In Embodiments 1 and 2, the reference image is divided into two regions, a dark region and a light region, and addition processing is performed only on the dark region. However, as shown in FIG. 9, in Embodiment 3, the procedure of Embodiment 1 is performed. Step S31 and Step S32 are added to divide the reference image into four regions according to the brightness level, the bright region reduces the number of superimposed images, and the number of superimposed images increases as the region becomes darker. To.

  First, in step S31, the luminance histogram analysis circuit 9 analyzes the reference image photo_11 read from the reference image storage unit 7, and generates a luminance histogram (see FIG. 10). Next, based on the luminance histogram, the reference image photo_11 is divided into four luminance regions according to the brightness level. Here, these four luminance areas are called R0, R1, R2, and R3 in order from the brighter. In this embodiment, the sun 24 of the reference image photo_11 belongs to the luminance region R0, the track 25, the person 26 and the box 27 belong to the luminance region R1, the tree 22 belongs to the luminance region R2, the mountain 21 and the background 23 are the luminance region. They are divided so as to belong to R3 (see FIG. 5).

  Next, step S32 is executed to superimpose the additional image on the reference image. As described above, the number of additional images to be superimposed is set to be small for a bright region and to increase as it becomes darker. More specifically, the luminance region R0 is 0; that is, it is not overlapped. For the luminance regions R1, R2, and R3, one, two, and three additional images are overlaid, respectively.

  Therefore, in the present embodiment, as shown in FIG. 11, the luminance region R0 (sun 1) of the reference image photo_11 is not overlaid, and the luminance region R1 (track 25, person 26 and box 27) One additional image area corresponding to R1 is superimposed. Then, two additional image regions corresponding to the luminance region R2 are superimposed on the luminance region R2 (tree 22). Then, three additional image regions corresponding to the luminance region R3 are superimposed on the luminance region R3 (mountain 21, background 23) to generate a composite image photo_30.

  As described above, in the third embodiment, the reference image is divided into a plurality of areas according to the brightness, and the number of images corresponding to the brightness is superimposed, so that the brightness is adjusted more finely. The synthesized image can be obtained.

  In the third embodiment, the reference image is divided into four areas according to the brightness, but the number of the areas can be changed as necessary. Also, the number of overlaps for each region is merely an example.

It is a block diagram of the digital camera apparatus which concerns on Embodiment 1 of this invention. It is a flowchart which shows the image processing method by the digital camera apparatus which concerns on Embodiment 1 of this invention. It is a figure which shows the picked-up image which concerns on Embodiment 1 of this invention. It is a brightness | luminance histogram of the reference | standard image which concerns on Embodiment 1 of this invention. It is a figure which shows the reference | standard image which concerns on Embodiment 1 of this invention. It is a figure which shows the addition process of the image which concerns on Embodiment 1 of this invention. It is a flowchart which shows the image processing method by the digital camera apparatus which concerns on Embodiment 2 of this invention. It is a brightness | luminance histogram of the additional image which concerns on Embodiment 2 of this invention. It is a flowchart which shows the image processing method by the digital camera apparatus which concerns on Embodiment 3 of this invention. It is a brightness | luminance histogram of the reference | standard image which concerns on Embodiment 3 of this invention. It is a figure which shows the addition process of the image which concerns on Embodiment 3 of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Digital camera apparatus, 2 ... Optical lens apparatus, 3 ... CCD, 4 ... A / D converter, 5 ... Image memory | storage part, 6 ... Reference | standard image selection circuit, 7 Reference image storage unit, 8 ... Additional image storage unit, 9 ... Luminance histogram analysis circuit, 10 ... Position shift correction circuit, 11 ... Addition processing circuit, 12 ... Control unit, b ... threshold, 21 ... mountain, 22 ... tree, 23 ... background, 24 ... sun, 25 ... truck, 26 ... people, 27 ... box, y ..Weighting function, R0 ... luminance region, R1 ... luminance region, R2 ... luminance region, R3 ... luminance region

Claims (6)

Imaging means;
Imaging control means for controlling the imaging means to continuously shoot the same subject under the same shooting conditions;
Selecting means for selecting a reference image from a plurality of images obtained as a result of continuous shooting by the imaging control means;
A luminance distribution calculating means for analyzing a pixel included in the reference image selected by the selecting means and calculating a luminance distribution;
Based on the luminance distribution of the reference image calculated by the luminance distribution calculating means, a threshold value for categorizing the reference image into a bright region and a dark region is determined, and a dark region specification for specifying the dark region of the reference image is determined. Means,
Image superimposing means for superimposing an image of an area corresponding to the dark area in a plurality of images other than the reference image on the dark area of the reference image specified by the dark area specifying means;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the threshold value is a median value of luminance in the luminance distribution. The image polymerization unit, multiplied by the positive coefficient to the luminance of the image belongs pixels in the region corresponding to the dark region in the plurality of images other than the reference image, the reference image, the reference corresponding to the dark region While superimposing the image of the area in multiple images other than the image ,
The image processing apparatus according to claim 1, wherein the coefficient is obtained by a function that gradually increases as the luminance of the pixel moves away from the threshold value. The said selection means selects the said reference | standard image from the image except the image imaged first and the image image | photographed lastly among the several images by which the said continuous shooting was carried out, The Claim 1 thru | or 3 characterized by the above-mentioned. The image processing apparatus according to any one of the above. The image processing apparatus according to claim 1, wherein the selection unit selects the reference image from the plurality of continuously shot images based on an arbitrary operation from the outside. A computer provided in an image processing apparatus having an imaging means ;
Imaging control means for controlling the imaging means to continuously shoot the same subject under the same shooting conditions;
Selecting means for selecting a reference image from a plurality of images obtained as a result of continuous shooting by the imaging control means;
A luminance distribution calculating means for analyzing a pixel included in the reference image selected by the selecting means and calculating a luminance distribution;
Based on the luminance distribution of the reference image calculated by the luminance distribution calculating means, a threshold value for categorizing the reference image into a bright region and a dark region is determined, and a dark region specification for specifying the dark region of the reference image is determined. means,
Image superimposing means for superimposing an image of an area corresponding to the dark area in a plurality of images other than the reference image on the dark area of the reference image specified by the dark area specifying means;
A program characterized by functioning as

Images