JP2006238060A - Image processor and digital camera provided with the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor capable of providing images of high quality, using a simple configuration. <P>SOLUTION: The image processor for processing an RGB signal 101 inputted from an image sensor, where RGB are Bayer-arrayed and obtaining a luminance signal and a color difference signal, is provided with an RGB signal level independent detecting part 103 for independently detecting the level of the RGB signal 101, according to each color and a median filter 105 for determining the flaws of the RGB signal inputted via an AFE 102 according to the signal level detected by the RGB signal level independent detection part 103 and the color, and when a flaw is determined, the flaw is corrected. Since the flaw of the RGB signal is determined, according to the signal level and color in each pixel; and when it is determined that there is a, the flaw is corrected by this constitution, high-quality noise reduction is executed, and high-quality images of are provided. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus that performs video signal processing corresponding to an image sensor and a digital camera equipped with the image processing apparatus, and more particularly to noise removal from an image sensor output.

  In recent years, there has been a remarkable shift from analog technology to digital technology in the camera industry. In particular, digital still cameras that require neither film nor development are booming, and camera-equipped cameras are becoming the mainstream.

  At present, CCD sensors are the mainstream as sensors for digital still cameras, but MOS sensors are also expected to be promising as sensors that change to CCDs because of improved characteristics. Image sensors are required to have higher performance, and it is necessary to provide technical proposals that emphasize high resolution and achieve high quality output.

FIG. 4 shows a basic block diagram of a conventional digital still camera equipped with an image processing apparatus that performs video signal processing corresponding to an image sensor.
In FIG. 4, 401 is an image sensor, 402 is a timing generator (TG) that generates a drive pulse of the image sensor, 403 is to remove noise from the output of the image sensor, perform gamma correction on the input signal, and raise the low illuminance part. CDS / AGC circuit for controlling gain, 404 an analog-digital converter (ADC), 405 a digital signal processing circuit (DSP) (details will be described later), 406 a memory circuit for storing image data and various data, Reference numeral 407 denotes a microcomputer (CPU) that controls the camera. In FIG. 4, light that has entered the image sensor 401 through a lens is converted into an electrical signal by a photodiode, and is output as an analog continuous signal by vertical driving and horizontal driving. Drive timing pulses necessary for the operation of the image sensor 401 are generated from the timing generator 402. In the signal output from the image sensor 401, the 1 / f noise is effectively reduced in the CDS / AGC circuit 403, and after that, the input signal is subjected to gamma correction, the low illuminance portion is raised, and the gain is controlled. The signal is input to the ADC 404 and converted into a digital signal. The digitized signal is input to the DSP 405, and various processes such as color separation, color matrix processing, and luminance processing are performed to obtain a luminance signal and a color difference signal, and are stored in the memory circuit 406.

  Of the basic configuration of the digital still camera shown in FIG. 4, the basic configuration of a conventional image processing apparatus mainly composed of the DSP 405 will be described with reference to FIG. 501 is an RGB signal input from the image sensor 401, 502 is an AFE corresponding to the CDS / AGC circuit 403 and ADC 404 in FIG. 4, 507 is an RGB signal processing unit corresponding to a part of the DSP 405 in FIG. 4, and 503 is in FIG. This is a CPU corresponding to the CPU 407. The RGB signal processing unit 507 is a median filter 504 that removes scratch noise from the sensor, and a luminance signal / color signal processing unit 505 that performs processing to convert the RGB signal from which noise has been removed by the median filter 504 into a luminance signal and a color difference signal. And a fixed LPF (Low Pass Filter) 506 that limits the band of the converted luminance signal and color difference signal, and the CPU 503 sets a scratch determination threshold level of the median filter 504.

  In the conventional image processing apparatus, the CPU 503 sets the scratch determination threshold level of the median filter 504 for each screen in accordance with the brightness of the subject and the AFE gain, thereby removing scratch noise (noise reduction). It is carried out.

Further, Patent Document 1 discloses an imaging apparatus that performs noise removal of RGB signals as in the above-described noise removal method (noise reduction method).
In Patent Document 1, a RGB signal is converted into a luminance signal and a color difference signal, and thereafter, correlation indicating the degree of correlation in at least one or more directions around the pixel of pixel data generated based on the luminance signal. The value is detected for each pixel, and the noise reduction gain for the luminance signal is set for each pixel based on the detected correlation value, and the color difference space of the pixel data is detected for each pixel based on the color difference signal. The noise reduction gain for the color difference signal is set for each pixel based on the color difference space, and then the noise reduction depth is controlled for each pixel based on the noise reduction gain for the set luminance signal. Noise of the luminance signal of each pixel is removed, and noise reduction is performed based on the noise reduction gain for the color difference signal. The depth of the action is controlled for each pixel are removing noise of the color difference signals of the respective pixels.
JP 2001-189944 A

  However, in the conventional noise reduction method using the image processing apparatus shown in FIG. 5, when the gain of the AFE 502 is increased, the high frequency component of the signal increases along with the scratch signal level of the image sensor 401. At this time, the scratch determination threshold level of the median filter 504 is increased. If is constant, there is a problem that video signals are lost due to erroneous determination of a scratch signal and a high-frequency image signal. Also, since the AFE 502 performs gamma correction on the input signal to raise the low illuminance portion, if the scratch determination threshold level is constant within one screen, scratch correction cannot be performed in the high luminance portion, and noise And the filter characteristics of the fixed LPF 506 are constant, so that there is a problem that the noise in the low illuminance part becomes conspicuous.

  In the noise reduction method of Patent Document 1, noise reduction is performed based on a luminance signal and a color difference signal that have already been subjected to image processing, and a pixel and a luminance signal and a color difference signal converted from an RGB signal are processed. Since the noise reduction gain is controlled every time, there is a problem that the configuration becomes complicated.

  Accordingly, the present invention solves the above-described conventional problems, and provides an image processing apparatus capable of providing a high-quality image with a simple configuration, and a digital camera including the image processing apparatus. It is intended.

  In order to achieve the above-described object, according to the first aspect of the present invention, an RGB signal input from an image sensor having a predetermined pixel arrangement is processed to obtain a luminance signal and a color difference signal, and output them. An RGB signal level independent detection unit that independently detects an RGB signal level input from the image sensor according to a color; and a signal level detected by the RGB signal level independent detection unit; It is characterized by comprising scratch noise removing means for determining a scratch of the input RGB signal according to the color, and correcting it when it is determined to be a scratch.

  According to the above configuration, for each pixel, it is determined whether or not there is a scratch in the scratch noise removing unit according to the signal level and color of the pixel detected by the RGB signal level independent detection unit. Is corrected. As described above, the scratch determination and correction are executed for each pixel according to the signal level and the color, so that high-quality noise reduction is performed and a high-quality image is provided.

  The invention described in claim 2 is the invention described in claim 1, wherein the RGB signal input from the image sensor is multiplied by a predetermined gain to perform gain correction and output to the scratch noise removing means. And a gain correction unit, wherein the scratch noise removing means determines the scratch in consideration of the gain.

  According to the above configuration, whether or not there is a scratch is determined for each pixel according to the signal level and color of the detected pixel and the gain of the gain correction unit that corrects the gain of the RGB signal. In the event of a failure, correction is performed. In this way, the accuracy of the scratch determination is improved by taking into account the input signal level that changes depending on the gain of the gain correction unit that corrects the gain of the RGB signal.

  The invention according to claim 3 is the invention according to claim 1, wherein the scratch noise removing means is arranged in horizontal, vertical, and diagonal directions between a pixel of interest for performing noise removal and a peripheral pixel of the pixel of interest. Based on the correlation, it is determined whether or not the pixel of interest is flawed. If it is determined that the pixel of interest is flawed, the pixel of interest is a signal level difference from the pixel of interest among the peripheral pixels. Is a median filter that corrects the pixel by replacing it with the smallest pixel.

  According to the above configuration, the noise removal unit is configured by a median filter, and the determination and correction of the scratches on the target pixel are performed based on the correlation between the target pixel on which noise removal is performed and the surrounding pixels of the target pixel.

  The invention according to claim 4 is the invention according to claim 1, wherein the scratch noise removing means is configured to detect a scratch determination threshold in which a difference in signal level between a target pixel and a peripheral pixel of the target pixel is a predetermined pixel unit. When the level exceeds the level, the median filter determines that the pixel of interest has a flaw, and the RGB signal level independent detection unit includes a signal level to be detected and a flaw determination threshold level for each pixel for each color type. Is stored as table data, and a scratch determination threshold level determined in units of pixels determined by referring to the table data is output to the scratch noise removing means.

  According to the above configuration, the RGB signal level independent detection unit searches the table data based on the detected signal level and color of the target pixel, obtains a scratch determination threshold level in units of pixels (target pixel), and outputs it to the median filter. In the median filter, if the difference in signal level between the pixel of interest and the surrounding pixels of the pixel of interest exceeds the scratch determination threshold level of the input pixel unit, it is determined as a scratch.

  The invention according to claim 5 is the invention according to claim 1, wherein the RGB signal corrected by the scratch noise removing means is subjected to predetermined processing to obtain a luminance signal and a color difference signal. A color signal processing unit; and a low-pass filter that limits a band of the luminance signal and the color difference signal obtained by the luminance signal / color signal processing unit, and the coefficient of the low-pass filter is detected by the RGB signal level independent detection unit. It is determined according to the signal level and color.

  According to the above configuration, in addition to the correction of scratches by the scratch noise removing means, the band of the luminance signal and the color difference signal is limited by the low-pass filter, so that higher quality noise reduction is performed and a high quality image is provided. Is done.

  The invention according to claim 6 is the invention according to claim 5, wherein the RGB signal level independent detection unit holds the correspondence between the signal level and the filter coefficient for each color type as table data, A filter coefficient determined by referring to the table data is output to the low-pass filter.

  According to the above configuration, the RGB signal level independent detection unit searches the table data based on the detected signal level and color of the target pixel, obtains the filter coefficient for each pixel (target pixel), and outputs the filter coefficient to the low-pass filter. In the low-pass filter, the bands of the luminance signal and the color difference signal are limited.

  According to a seventh aspect of the present invention, in the digital camera, the digital camera processes an image sensor in which pixels are arranged in a Bayer array and an RGB signal output from the image sensor. The image processing apparatus includes a memory circuit that stores an image processed by the image processing apparatus.

  According to the above configuration, it is possible to provide a higher quality digital camera by including the image processing device that performs high quality noise reduction and provides a high quality image.

  The image processing apparatus of the present invention performs adaptive noise reduction for each pixel on the basis of the input RGB signal level and color detected for each pixel in the previous stage of the scratch noise removing unit, and thus has a simple configuration. The detection accuracy of the image information in the frame is improved, high-quality noise reduction can be realized more flexibly, and a high-quality image can be provided.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[Embodiment 1]
FIG. 1 is a basic configuration diagram of an image processing apparatus according to Embodiment 1 of the present invention. The image processing apparatus will be described as being used in the digital camera having the basic configuration shown in FIG.

  As shown in FIG. 1, an RGB signal 101 is input from an image sensor in which RGB are arranged in a Bayer array. The Bayer array refers to a color array in which “R” and “B” are arranged on one diagonal, and two pixels “G” are arranged on the other diagonal, with 4 pixels as a structural unit.

  The input RGB signal 101 is input to the AFE 102 (corresponding to the CDS / AGC circuit 403 and ADC 404 in the background art; an example of a gain correction unit). The AFE 102 performs analog gain correction by multiplying the RGB signal 101 by an AFE gain set by the CPU 104 described later. The RGB signal subjected to the analog gain correction is converted into a luminance signal (Y) and a color difference signal (Cr, Cb) by the RGB signal processing unit 108 and output.

  The CPU 104 sets the AFE gain of the AFE 102 according to the characteristics, brightness, etc. of the subject, and outputs the set AFE gain to the RGB signal level independent detection unit 103 to be described later to control the entire control.

An RGB signal level independent detection unit 103 is newly provided.
The RGB signal level independent detection unit 103 performs level detection independently from the RGB signal that has been subjected to analog gain correction by the AFE 102 according to the color, that is, for each of the R signal, G signal, and B signal. The RGB signal level independent detection unit 103 also detects a scratch determination threshold level (described later) that is linked to the detected signal level and the information of the AFE gain input from the CPU 104 for each color type (R signal, G signal, and B signal). Table data corresponding to the threshold level for determining scratches in the median filter 105 is stored (built-in) so as to be sequentially readable, and the scratch determination threshold level for each pixel determined by referring to the table data will be described later. This is output to the median filter 105. That is, a signal level is detected for each pixel, table data is selected (referenced) in consideration of whether the pixel corresponds to an R signal, a G signal, or a B signal, and the selected (referenced) table data is selected. The scratch determination threshold level for each pixel is determined by searching based on the detected signal level, and is output to the median filter 105. Thus, the scratch determination threshold level for each pixel is adaptively determined with respect to the input RGB signal level and AFE gain setting input to the median filter 105.

The RGB signal processing unit 108 includes a median filter 105 constituting a scratch noise removing unit, a luminance signal / color difference signal processing unit 106, and a fixed LPF 107.
The luminance signal / color difference signal processing unit 106 converts the RGB signal that has been detected and corrected by the median filter 105 into a luminance signal and a color difference signal, and obtains the luminance signal and the color difference signal.

The fixed LPF 107 limits the band of the luminance signal and the color difference signal obtained by the luminance signal / color difference signal processing unit 106 (performs noise reduction again).
The median filter 105 will be described in detail.

  The median filter 105 is a pixel-based scratch determination threshold level set (input) by the RGB signal level independent detection unit 103 (a scratch determination formed according to the signal level and color detected by the RGB signal level independent detection unit 103). This is a filter that detects and corrects flaws in the input RGB signal that has been subjected to analog gain correction by the AFE 102 based on the threshold level. Here, scratches are often caused by defective pixels of the image sensor.

The flaw detection and correction of the median filter 105 will be described in detail with reference to FIG.
As shown in FIG. 2, the median filter 105 looks at the correlation of the “horizontal direction”, “vertical direction”, and “diagonal direction” between the target pixel on which noise removal is performed and peripheral pixels of the same color as the target pixel. Whether or not the pixel of interest is flawed, the signal level of the pixel of interest and the signs of the differences between the signal levels of the surrounding pixels in the horizontal, vertical and diagonal directions all match, and When the smallest absolute value is larger than the scratch determination threshold level of the target pixel input from the RGB signal level independent detection unit 103, it is determined as a scratch, and the target pixel is determined as a difference from the target pixel among the surrounding pixels. Correction is performed by replacing the pixel with the smallest surrounding pixel.

  For example, as shown in FIG. 2, when the target pixel is an R signal pixel, dn (n) obtained by subtracting the signal level of each peripheral pixel R of the same color in the horizontal, vertical, and diagonal directions from the signal level of the target pixel RA. = 1 to 7) are coincident and the smallest value dmin among the absolute values of these differences is larger than the scratch determination threshold level of the pixel of interest RA, the pixel of interest RA is scratched. The correction is performed by replacing the target pixel RA with the peripheral pixel R having the smallest difference from the target pixel RA among the peripheral pixels R.

  Further, when the signs of dn are all negative, black scratches, and when all signs are positive, white scratches. When black scratches or white scratches, the pixel-level scratch determination threshold level input from the RGB signal level independent detection unit 103 is input. Is corrected. That is, if the average value of the signal level of the surrounding pixels is low in the case of black scratches, the determination threshold level is corrected to be small, and if the average value of the signal levels of the peripheral pixels is high in the case of white scratches, The determination threshold level for each pixel is greatly corrected. Thereby, even in an image whose signal level changes greatly in the screen, the detection rate of scratches increases.

  The RGB signal processing unit 108 and the RGB signal level independent detection unit 103 are incorporated in the DSP 405 in FIG. Further, the DSP 405 and the CPU 104 may be configured with different semiconductor chips or may be configured within the same semiconductor chip.

  As described above, according to the first embodiment, the signal level difference between the target pixel and the surrounding pixels also changes in magnitude according to the magnitude of the AFE gain. Therefore, the input signal level is detected, and the AFE gain at the time of signal input is detected. By setting a scratch determination threshold level for each pixel in conjunction with the information and performing noise reduction for determining and correcting scratches in the median filter 5, that is, input RGB detected for each pixel in the preceding stage of the median filter 5 By performing adaptive noise reduction for each pixel based on the signal level and color, adaptive noise reduction can be performed for each pixel with a simple configuration, and image information within one frame can be detected. Improve accuracy, achieve higher quality noise reduction more flexibly, and provide high quality images . In addition, the scratch detection rate can be controlled. Random noise can also be reduced by detecting the low illuminance part and color data of the input RGB signal based on the signal levels of the peripheral pixels around the pixel of interest and correcting (setting) the scratch determination threshold level to be small.

Further, according to the first embodiment, it is possible to provide a higher-quality digital camera by including an image processing device that performs high-quality noise reduction and provides a high-quality image.
[Embodiment 2]
A second embodiment of the present invention will be described with reference to the drawings.

FIG. 3 is a basic configuration diagram of the image processing apparatus according to the second embodiment. Constituent elements similar to those of the first embodiment are denoted by the same reference numerals and description thereof is omitted.
In the image processing apparatus according to the second embodiment, the fixed filter 107 according to the first embodiment is configured by an n-tap vertical LPF 307 and an n-tap horizontal LPF 308, and the scratch determination threshold level of the median filter 105 and the n-tap horizontal / vertical are set. Both of the coefficients of the LPFs 307 and 308 are controlled by the RGB signal level independent detection unit 303. The first embodiment is different from the first embodiment in that the filter coefficients of the n-tap vertical LPF 307 and the n-tap horizontal LPF 308 are controlled in units of pixels at the latter stage of image processing.

  The RGB signal level independent detection unit 303 adds the signal level and the LPFs 307 and 308 in addition to the table data corresponding to the signal level and the scratch determination threshold level for each color type (R signal, G signal, and B signal). The second table table corresponding to the filter coefficients is held (built-in). Then, filter coefficients in pixel units obtained by searching the second table table based on the color type and signal level are output to the LPFs 307 and 308. Thereby, in the LPFs 307 and 308, noise removal is performed based on the filter coefficient in units of pixels based on the color type and the signal level, and more effective noise removal is realized.

  The median filter 105 determines and corrects scratches similar to those in the first embodiment, and determines the brightness and color around the pixel of interest based on the levels of the R, G, and B signals around the pixel of interest. The noise of the pixel of interest is removed based on the brightness and color around the pixel of interest and the scratch determination threshold level of the pixel of interest input from the RGB signal level independent detection unit 103. For example, the skin-colored part of the screen performs strong noise removal (corrects the scratch judgment threshold level to reduce noise), and the achromatic part weakens the noise removal (corrects the scratch judgment threshold level to remove noise). In this way, the human face portion can be processed into a soft image with less noise, and the hair portion can be processed into an image that is rich in detail and can be distinguished one by one.

  As described above, according to the second embodiment, the scratch determination threshold level of the median filter 105 and the LPFs 307 and 308 are compared with the signal level and color of the input RGB signal 101 and with respect to the gain of the AFE 102 at the time of signal input. The filter coefficient can be set adaptively for each pixel, and more effective noise removal (noise reduction) can be realized.

  The image processing apparatus according to the present invention has an effect that more effective noise removal can be realized, and can be widely applied not only to a digital still camera but also to a digital movie.

1 is a basic configuration diagram of an image processing apparatus according to Embodiment 1 of the present invention. It is a crack judgment explanatory view of the median filter of the image processing apparatus. It is a basic block diagram of the image processing apparatus in Embodiment 2 of this invention. It is a basic lineblock diagram of a digital still camera. It is a basic block diagram of the conventional image processing apparatus.

Explanation of symbols

101 Input RGB signal 102 AFE
103, 303 RGB signal level independent detection unit 104 CPU
105 Median Filter 106 Luminance Signal / Color Signal Processing Unit 107 Fixed LPF
108 RGB signal processing unit 307 n-tap vertical LPF
308 n-tap horizontal LPF
401 Image sensor 402 TG
403 CDS / AGC circuit 404 ADC
405 DSP
406 Memory circuit

Claims (7)

An image processing apparatus that processes an RGB signal input from an image sensor having a predetermined pixel arrangement to obtain a luminance signal and a color difference signal, and outputs the luminance signal and a color difference signal.
An RGB signal level independent detection unit for independently detecting the level of the RGB signal input from the image sensor according to the color;
A scratch noise removing unit that determines a scratch of the input RGB signal according to the signal level and color detected by the RGB signal level independent detection unit, and corrects the scratch when the scratch is determined as a scratch. A featured image processing apparatus. A gain correction unit that performs gain correction by multiplying the RGB signal input from the image sensor by a predetermined gain, and outputs the signal to the scratch noise removing unit;
The image processing apparatus according to claim 1, wherein the scratch noise removing unit determines a scratch in consideration of the gain. The scratch noise removing means determines whether or not the target pixel is scratched based on a horizontal, vertical, and diagonal correlation between the target pixel on which noise removal is performed and a peripheral pixel of the target pixel. When it is determined that there is a scratch, the median filter performs correction by replacing the target pixel with a pixel having the smallest signal level difference from the target pixel among the peripheral pixels. The image processing apparatus according to claim 1. The scratch noise removing means is formed by a median filter that determines that there is a scratch on the target pixel when a difference in signal level between the target pixel and a peripheral pixel of the target pixel exceeds a scratch determination threshold level in a predetermined pixel unit.
The RGB signal level independent detection unit holds, as table data, the correspondence between the signal level to be detected and the scratch determination threshold level of the pixel unit for each type of color, and for each pixel type determined by referring to the table data The image processing apparatus according to claim 1, wherein a scratch determination threshold level is output to the scratch noise removing unit. A luminance signal / color signal processing unit that performs predetermined processing on the RGB signal corrected by the scratch noise removing unit to obtain a luminance signal and a color difference signal;
A low-pass filter that limits the band of the luminance signal and the color difference signal obtained by the luminance signal / color signal processing unit;
The image processing apparatus according to claim 1, wherein a coefficient of the low-pass filter is determined according to a signal level and a color detected by the RGB signal level independent detection unit. The RGB signal level independent detection unit holds the correspondence between the signal level and the filter coefficient for each color type as table data, and outputs the filter coefficient determined by referring to the table data to the low-pass filter. The image processing apparatus according to claim 5, wherein: An image sensor with pixels arranged in a Bayer array;
The image processing apparatus according to any one of claims 1 to 6, which processes RGB signals output from the image sensor;
A digital camera comprising: a memory circuit that stores an image processed by the image processing apparatus.

Images