JP2008017037A - Solid imaging apparatus, and defective pixel correcting method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suppress frequencies of decreases in resolution and resolving feeling, occurrence of a false color, etc., and to prevent a defective pixel correction from being excessively performed. <P>SOLUTION: A defective level of a defective pixel of an imaging element is detected, a peripheral luminance level of a defective pixel periphery is calculated, and defective pixel correction of only a defective pixel which is visually conspicuous is performed to obtain an excellent image whose decreases in resolution and resolving feeling, false color generation, etc., are minimized and also to prevent the defective pixel correction from being excessively performed. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a solid-state imaging device and a defective pixel correction method, and more particularly to a technique suitable for performing defective pixel correction efficiently.

  In a solid-state imaging device (for example, CCD, CMOS, etc.), it is conventionally known that defective pixels are generated due to crystal defects, dust, or the like. The defective pixels are mainly classified into two types. The first is called a black flaw and refers to a defective pixel whose output value is lower than the value that should be output. As a phenomenon that appears in actual use, an image with a uniform brightness should be originally obtained when a subject with a white surface is photographed, but an image having black spots in the white surface due to black scratches. As the cause, it can be considered that dust is inferior in the degree, and unevenness in opening is in the lighter degree.

  The second is a so-called white scratch, which indicates a defective pixel whose output value is higher than the value that should be output. As a phenomenon that appears in actual use, an image with a black surface should be originally obtained when a subject with a black surface is photographed, but an image having white spots in the black surface due to white scratches. A possible cause is an increase in dark signal mainly due to crystal defects.

  In the recent trend of increasing the number of pixels and miniaturization, the number of defective pixels described above is increasing, and there is concern about the influence on the image and the camera system. As the number of defective pixels increases, it becomes difficult to deal with all defective pixels, and various defective pixel correction methods have been proposed as countermeasures against them.

  For example, the defective pixel correction method described in Patent Document 1 is effective when detecting a defective pixel in a solid-state imaging device including an effective pixel region in which an imaging surface is output as an image and an invalid pixel region other than the effective pixel region. Detection of defective pixels in the pixel area is performed with priority over detection of defective pixels in the invalid pixel area. Thus, a method has been proposed in which defect correction is effectively performed without increasing the number of data banks that store data on defective pixels.

  Further, in the defective pixel correction method described in Patent Document 2, image data is temporarily stored in the image storage unit, the defective pixel included in the read image data is detected by the defective pixel detection unit, and the first storage unit has a defect. Store all pixels. On the other hand, the correction target selection unit prioritizes defects in the vicinity of the screen center in the image data read from the first storage unit, and selects the correction target image sensor in consideration of the number of corrections. In other words, a method is proposed that reduces the burden with a simple configuration and easily provides an image without defective pixels by selecting a region in which a defective image is conspicuous and accurately while satisfying the restriction on the number of corrections. Has been.

JP-A-8-317292 JP 2002-84463 A

  In the defective pixel correction methods described in Patent Document 1 and Patent Document 2, defective pixel correction is performed preferentially in an area where defective pixel correction is preferentially required, and defective pixel correction is performed in an area where priority is not required. It is a method to limit. However, in the area where defective pixel correction is preferentially required, there is a high possibility that an interpolation error will occur when the number of correction target pixels is very large or when correction target pixels are concentrated in a certain part of the screen. . For this reason, in actual use, there has been a problem that the frequency at which the resolution, the sense of resolution are lowered, and the occurrence of false colors is increased.

  The reason why these phenomena occur is that defective pixel correction normally interpolates from adjacent pixels of the same color, but if there are a plurality of defective pixels in the adjacent pixels of the same color, interpolation is further performed from neighboring pixels of the same color. Therefore, since the positional relationship between the defective pixel and the adjacent pixel of the same color used in the interpolation becomes far, there is a high possibility that the defective pixel is corrected using a correction value different from the correct correction value when the defective pixel is corrected. I have.

  In addition, in the defective pixel correction methods described in Patent Document 1 and Patent Document 2, correction is performed on defective pixels detected regardless of the luminance of the subject. For this reason, in the case of this defective pixel correction method, the correction is always performed in the same manner regardless of the low luminance at which the defective pixel is visually noticeable and the high luminance at which the defective pixel is not visually noticeable. In this way, performing defective pixel correction regardless of how conspicuous the defective pixel is, as described above, when the number of correction target pixels becomes very large, or when correction target pixels concentrate on a certain part, There was a problem that the possibility of occurrence of an interpolation error increases.

  In view of the above-described problems, the present invention makes it possible to suppress the frequency with which the resolution, the sense of resolution decrease, the occurrence of false colors, and the like, and the defective pixel correction of the solid-state imaging device is excessively performed. It aims to be able to prevent.

  The solid-state imaging device of the present invention converts an analog signal obtained by photoelectrically converting incident light from a subject with a solid-state imaging device into a digital signal, performs signal processing on the digital signal, and defective pixels of the solid-state imaging device A solid-state imaging device that performs correction, wherein the solid-state imaging device detects video signal level detection means that detects an output level of the analog signal or the digital signal, and an output level detected by the video signal level detection means And defective pixel correction operation determining means for determining whether or not to perform the defective pixel correction.

  The defective pixel correction method of the present invention converts an analog signal obtained by photoelectrically converting incident light from a subject with a solid-state image sensor into a digital signal, and performs signal processing on the digital signal to detect defects in the solid-state image sensor. A defective pixel correction method for a solid-state imaging device that performs pixel correction, according to a video signal level detection step for detecting an output level of the analog signal or the digital signal, and an output level detected in the video signal level detection step And a defective pixel correction operation determination step for determining whether or not to perform defective pixel correction of the solid-state imaging device.

  According to the present invention, it is determined whether or not to perform defective pixel correction according to the output level, and defective pixel correction of the solid-state imaging device is performed. As a result, it is possible to suppress the occurrence of the occurrence of false colors with reduced resolution and resolution, and it is possible to prevent the defective pixel correction of the solid-state imaging device from being performed excessively.

(First embodiment)
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a functional configuration example of a solid-state imaging apparatus (here, a digital still camera) according to the present embodiment.
First, the function of each part in FIG. 1 will be described. Reference numeral 10 denotes a solid-state imaging device, which is a digital still camera in this embodiment.

  The internal function will be described in detail. Reference numeral 12 denotes a solid-state image sensor that converts light into an electrical signal. In the present embodiment, a CCD is used. Reference numeral 11 denotes a lens that collects light from the subject onto the CCD 12. A timing generator (TG) 13 generates timing for driving the CCD 12.

  An A / D conversion circuit 14 converts an electrical signal output from the CCD 12, that is, an analog video signal into a digital video signal. After being converted into a digital video signal by the A / D conversion circuit 14, it is once recorded as RAW image data in an image memory 16 (here, DRAM) of the storage unit 15.

  The defective pixel address memory 17 (referred to as ROM here) in the storage unit 15 stores the coordinates, X address, and defective pixel coordinates obtained by defective pixel detection at the time of factory adjustment or camera activation in advance. Y address is recorded. Further, the defective pixel level memory 18 (herein referred to as ROM) records the defect level obtained by detecting defective pixels in advance at the time of factory adjustment or camera activation.

  Reference numeral 19 denotes a luminance level determination circuit that determines the luminance levels of the peripheral pixels of the defective pixel, and reference numeral 20 denotes an image processing circuit that converts RAW data recorded in the image memory 16 into a final output image format. After image processing by the image processing circuit 20, the captured image is recorded in a final format on a recording medium 21 (here, a compact flash (registered trademark) card).

  Reference numeral 22 denotes a control unit (herein referred to as a CPU), which controls the timing generator 13, the A / D conversion circuit 14, the storage unit 15, the luminance level determination circuit 19, the image processing circuit 20, and the recording medium 21. Control.

  Next, the defective pixel detection processing procedure in this embodiment will be described with reference to the flowchart shown in FIG. As described above, the X address and Y address of the defective pixel and the defect level of the defective pixel must be detected in advance and stored in the camera memory. In this embodiment, an example in which defective pixels are detected in a factory before product shipment is shown.

  First, in step S101, defective pixel detection is started. Here, since the number of existing pixels, addresses, and defect levels differ depending on the CCD mounted, it is necessary to detect each defective product.

  Next, in step S102, a subject with uniform brightness, for example, a subject such as a luminance box, is photographed to detect defective pixels. The photographing condition is preferably a condition that does not have shading that makes the entire surface of the photographed image uniform, for example, a condition that is narrower than the F value. In step S103, the captured image is converted from an analog video signal to a digital signal by the A / D conversion circuit 14, and recorded in the image memory 16 as a RAW image.

  Next, the process proceeds to step S104, and a difference value from the surrounding pixels of the same color is calculated for each pixel of the RAW image. Details of the difference value calculation method will be described later with reference to FIG. In the present embodiment, the CCD 12 is described on the premise of the primary color Bayer arrangement, but other color filter configurations may be used.

For example, when the inspection target pixel is R33 located in the center of FIG. 4, the average value of the surrounding pixels of the same color of the inspection target pixel (R11, R13, R15, R31, R35, R51, R53, and R55 in this embodiment) AVER33. AVER33 is calculated | required by the following formula | equation (1).
AVER33 = (R11 + R13 + R15 + R31 + R35 + R51 + R53 + R55) / 8 (1)

Furthermore, when the difference value between the addition average values AVER33 and R33 is DR33, DR33 is obtained by the following equation (2).
DR33 = | R33-AVER33 | (2)

  Returning to the flowchart of FIG. In step S104, after calculating the difference value, in step S105, the defective pixel determination level is further set to K, and K and DR33 are compared in magnitude. As a result of this comparison, if K <DR33, the process proceeds to step S106, where R33 is determined to be a defective pixel, and the X address and Y address (see FIG. 5) are stored in the defective pixel address memory 17 of the storage unit 15 in the digital still camera 10. In the example shown in FIG. 4, X = 3, Y = 3) is recorded.

Further, the defect level of the defective pixel is calculated by the following equations (3) and (4). If the defect level of R33 is LEVR33,
When R33 ≧ AVER33, LEVR33 = R33 / AVER33 (3)
When R33 <AVER33, LEVR33 = AVER33 / R33 (4)
And Then, the defect level (LEVR33) of R33 is recorded in the defective pixel level memory 18 of the storage unit 15 in the digital still camera 10.

  On the other hand, if K ≧ DR33 as a result of the comparison in step S105, the process proceeds to step S107, and it is determined that R33 is not a defective pixel because the original value of R33 is output. Next, in step S108, it is determined whether or not defective pixels have been determined up to the final pixel of the RAW image. If the determination of the defective pixel is not completed as a result of the determination, the process returns to step S104, and the defective pixel detection operation for each pixel is performed in the same manner until the completion of all pixels from step S104 to S108.

  On the other hand, if the determination of the defective pixel is completed as a result of the determination in step S108, the process proceeds to step S109, and a defective pixel correction table is created. FIG. 5 is a diagram illustrating an example of a defective pixel correction table. As shown in FIG. 5, the brightness level (12 bits) of the peripheral pixels of the same color of the defective pixel and the defective pixel level are classified into five levels, and are created with two parameters of ○ and ×. Details of the creation method and the usage method will be described later with reference to the flowchart shown in FIG. When the defective pixel correction table is created in step S109, the defective pixel detection at the factory before product shipment is terminated (step S110).

Next, an example of the defective pixel correction method of this embodiment will be described with reference to the flowchart shown in FIG.
First, in step S201, shooting is started by a photographer's operation. Next, in step S <b> 202, the captured RAW image is temporarily recorded in the image memory (DRAM) 16 of the storage unit 15 in the digital still camera 10.

  Next, in step S203, the address of the defective pixel detected in step S106 in FIG. Further, in step S204, the defective pixel level of the defective pixel detected in step S106 in FIG.

Next, in step S205, the peripheral luminance level around the defective pixel is calculated. As for the peripheral luminance level calculation method around the defective pixel, if R33 shown in FIG. 4 is a defective pixel and the peripheral luminance level is YR33, in this embodiment, the following equation (5) is used to calculate the defective pixel. A peripheral brightness level is calculated.
YR33 = ((R13 + R31 + R35 + R53) / 4 + (B22 + B24 + B42 + B44) / 4 + (Gr23 + Gb32 + Gb34 + Gr43) / 4 × 2) / 4 (5)

  Next, in step S206, with reference to the defective pixel correction table created in step S109 of FIG. 2, it is determined whether or not the defective pixel correction is performed on the defective pixel acquired in step S203. In the present embodiment, the defective pixel correction table includes five levels of peripheral luminance levels around the defective pixel calculated in step S109 in FIG. 2 (described in 12 bits), and the defective pixel level of the defective pixel is set to levels 1 to 5 (level 1). It is created by two parameters classified into five levels.

  Whether or not to perform defective pixel correction is determined based on these two parameters. In this embodiment, the defective pixel is less noticeable when the periphery of the defective pixel has high luminance, and the defective pixel level is low at low luminance. Only when it is low, defective pixel correction is performed.

  The characteristic that the defective pixel is less noticeable when the periphery of the defective pixel has high brightness will be described with reference to FIG. FIG. 6 is a diagram showing input / output characteristics of luminance values, that is, luminance gamma characteristics. The X axis represents the luminance value (12 bits) of the input RAW image, and the Y axis represents the luminance value (8 bits) of the final output image format.

  For example, a case will be described in which it is assumed that a defective pixel having an output value of −20% with respect to a value that should be output is present in each of the high luminance portion and the low luminance portion. As indicated by (1) in FIG. 6, the defective pixel in the high luminance portion has a small decrease in output value because the gradient of the gamma characteristic is small even when the output is decreased by −20%. On the other hand, as shown in (2) in FIG. 6, the defective pixel in the low luminance part has a large gradient of the gamma characteristic even when the decrease is the same −20%, so that the output value decreases greatly.

  Taking advantage of such characteristics, in step S206, it is determined that defective pixel correction is to be performed when the pixel is a defective pixel corresponding to a circle in FIG. If it is, it is determined not to perform defective pixel correction.

If defective pixel correction is to be performed as a result of the determination in step S206, the process proceeds to step S207, and the value of the defective pixel is replaced by interpolation from surrounding pixels of the same color. Specifically, for example, when R33 in FIG. 4 is a defective pixel correction necessary pixel, R33 is replaced with the same color surrounding pixel average value AVER33 by using the following equation (6).
R33 = AVER33 = (R11 + R13 + R15 + R31 + R35 + R51 + R53 + R55) / 8 (6)

  On the other hand, if the result of determination in step S206 is that defective pixel correction is not to be performed, processing proceeds to step S208, where the defective pixel correction is not performed and is left unattended. Next, in step S209, it is determined whether or not the defective pixel correction determination has been completed up to the final defective pixel. If the result of this determination is that the defective pixel correction determination has not been completed up to the final defective pixel, the process returns to step S203, and all defective pixels recorded in the defective pixel address memory detected in step S106 of FIG. Repeat until the defective pixel correction is determined. On the other hand, as a result of the determination in step S209, when the determination of the defective pixel correction is completed up to the final defective pixel, the process proceeds to step S210, and the defective pixel correction is completed.

  As described above, in the present embodiment, the defective pixel correction is performed only on the pixel that is visually conspicuous of the defective pixel from the peripheral luminance level around the defective pixel and the defective pixel level of the defective pixel. As a result, it is possible to provide a good image with a reduction in resolution feeling minimized.

(Other embodiments according to the present invention)
Each means constituting the solid-state imaging device and each step of the defective pixel correction method in the embodiment of the present invention described above can be realized by operating a program stored in a RAM or ROM of a computer. This program and a computer-readable recording medium recording the program are included in the present invention.

  Further, the present invention can be implemented as, for example, a system, apparatus, method, program, or recording medium. Specifically, the present invention may be applied to a system including a plurality of devices. The present invention may be applied to an apparatus composed of a single device.

  In the present invention, a software program (in the embodiment, a program corresponding to the flowcharts shown in FIGS. 2 and 3) that realizes the functions of the above-described embodiments is directly or remotely supplied to the system or apparatus. This includes the case where the system or the computer of the apparatus is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

  Examples of the recording medium for supplying the program include a floppy (registered trademark) disk, a hard disk, an optical disk, and a magneto-optical disk. Further, there are MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, there is a method of connecting to a homepage on the Internet using a browser of a client computer. The computer program itself of the present invention or a compressed file including an automatic installation function can be downloaded from the homepage by downloading it to a recording medium such as a hard disk.

  It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  As another method, the program of the present invention is encrypted, stored in a recording medium such as a CD-ROM, distributed to users, and encrypted from a homepage via the Internet to users who have cleared predetermined conditions. Download the key information to be solved. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. Furthermore, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can be realized by the processing.

  As another method, the program read from the recording medium is first written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Then, based on the instructions of the program, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are also realized by the processing.

It is a block diagram which shows the function structural example of the digital still camera which concerns on embodiment of this invention. It is a flowchart which shows an example of the detection process procedure of the defective pixel in embodiment of this invention. It is a flowchart which shows an example of the defective pixel correction | amendment procedure in embodiment of this invention. It is a figure which shows an example of the pixel arrangement | sequence which concerns on embodiment of this invention. It is a figure which shows an example of the defective pixel correction table which concerns on embodiment of this invention. It is a figure which shows the input / output characteristic which concerns on embodiment of this invention.

Explanation of symbols

10 Digital still camera 11 Lens 12 CCD
DESCRIPTION OF SYMBOLS 13 Timing generator 14 A / D conversion circuit 15 Memory | storage part 16 Image memory 17 Defective pixel address memory 18 Defective pixel level memory 19 Luminance level determination circuit 20 Image processing circuit 21 Recording medium 22 Control part

Claims (6)

A solid-state imaging device that converts an analog signal obtained by photoelectrically converting incident light from an object into a digital signal, performs signal processing on the digital signal, and corrects a defective pixel of the solid-state imaging device. And
Video signal level detection means for detecting an output level of the analog signal or the digital signal;
A solid-state imaging device, comprising: a defective pixel correction operation determination unit that determines whether or not to perform defective pixel correction of the solid-state imaging device according to an output level detected by the video signal level detection unit.   The solid-state imaging device according to claim 1, wherein the video signal level detection unit generates luminance signal information from an output level of the analog signal or the digital signal. A defect level detecting means for detecting a defect level of a defective pixel in the solid-state imaging device;
Whether or not the defective pixel correction operation determination unit performs the defective pixel correction of the solid-state imaging device from the defect level detected by the defect level detection unit and the information of the luminance signal generated by the video signal level detection unit. The solid-state imaging device according to claim 2, wherein the solid-state imaging device is determined. A defect in a solid-state imaging device that converts an analog signal obtained by photoelectrically converting incident light from a subject into a digital signal to a digital signal and performs signal processing on the digital signal to correct a defective pixel of the solid-state imaging device A pixel correction method,
A video signal level detection step of detecting an output level of the analog signal or the digital signal;
A defective pixel correction method comprising: a defective pixel correction operation determination step for determining whether or not to perform defective pixel correction of the solid-state imaging device according to the output level detected in the video signal level detection step .   5. The defective pixel correction method according to claim 4, wherein in the video signal level detection step, information on a luminance signal is generated from an output level of the analog signal or the digital signal. A defect level detection step of detecting a defect level of a defective pixel in the solid-state imaging device;
Whether the defective pixel correction of the solid-state imaging device is performed based on the defect level detected in the defect level detection step and the luminance signal information generated in the video signal level detection step in the defective pixel correction operation determination step 6. The defective pixel correction method according to claim 5, wherein it is determined whether or not.

Images