JP2000165892A - Pixel interpolation method and image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a pixel interpolation method to prevent deterioration in image quality due to interpolation and an image processor. SOLUTION: The image processing unit is provide with a picture interpolation processing section 7. The processing section 7 compares a difference between signal quantities of two green(G) pixels longitudinally adjacent to a target pixel other than the G pixel in a color filter 2 having three primary colors R(red), G(green) and B(blue) with G pixels as specified color arranged in a checkered way with a difference between signal quantities of two green(G) pixels laterally adjacent to the target pixel, discriminates a direction with a smaller difference to be a consecutive direction, discriminates the longitudinal (lateral) direction to be the consecutive direction when the differences are the same, and interpolates the target pixel with a signal quantity averaging the signal quantities of the G pixels in the discriminated direction.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pixel interpolation method for obtaining a video signal and an image processing apparatus used in an electronic still camera or a video camera having an optical system lens.

[0002]

2. Description of the Related Art Conventionally, this type of image processing apparatus has been used, for example, in an electronic still camera for recording a still image.

As shown in FIG. 9, a conventional image processing apparatus 1 includes a solid-state imaging device 3 having a color filter 2,
It is composed of a pixel interpolation unit 4 and an RGB image generation unit 5. As shown in FIG. 10, this color filter 2
It has R (red), G (green), and B (blue), and is a checkered array of G (green).

The image processing apparatus 1 generates an RGB color image based on a signal output of a solid-state imaging device 3, and applies a G pixel signal to R (red) and B (blue) pixels of a color filter 2. In the case of interpolation, it is necessary to interpolate R (red) and B (blue) pixels as missing parts in order to extract a G (green) signal and process it as a video signal as shown in FIG. The pixel interpolator 4 averages the G signals of a total of 4 pixels in the vertical and horizontal directions adjacent to the pixel to be interpolated, and outputs the averaged G (green) signal to the R (red) and B ( (RGB), and the RGB image is generated by the RGB image generation unit 5.

In addition, instead of the above-mentioned G (green) signal, R
When interpolating with (red) signals, as shown in FIG.
G (green) checkerboard arrangement of color filter 2 to R (red)
In order to extract the signal and process it as a video signal, it is necessary to interpolate the G (green) and B (blue) pixels as missing parts. Similarly, after simple averaging, processing such as filtering is performed. To generate RGB images. In addition,
The same applies to the case where interpolation is performed using a B (blue) signal.

[0006]

However, in the image processing by the conventional pixel interpolation method, if it is assumed that a signal amount having a distribution as shown in FIG. R of filter 2 (red)
In the case where the G signal is interpolated into the pixel portions of B and B (blue), the G (green) signal of a total of 4 pixels in the vertical and horizontal directions of the pixel to be interpolated is averaged and interpolated. Therefore, there is a problem that the image quality after the interpolation becomes sharp, and the image quality is severely deteriorated, for example, the resolution is reduced and a false color is generated.

Further, instead of the G (green) signal, R (red)
In the case of interpolating with a signal, assuming that a signal amount having a distribution as shown in FIG. 15 is incident on the CCD having the above-described color filter 2, the R (red) signal is simply averaged and interpolated. The signal amount distribution shown in FIG. 16 is obtained, and similarly to the case of the G (green) signal described above, the contour of the interpolated image becomes unclear, the resolution is reduced, and a false color is generated. There is. It should be noted that there is a similar problem when the interpolation is performed using the B (blue) signal.

SUMMARY OF THE INVENTION An object of the present invention is to provide a pixel interpolation method and an image processing apparatus for preventing deterioration of image quality due to interpolation.

[0009]

According to the pixel interpolation method of the present invention, a pixel signal output from a target pixel other than a specific color of a color filter having three primary colors and having specific colors among them arranged in a checkered pattern is specified. When interpolating based on a pixel signal output from a color pixel, the difference between the signal amount of two vertical pixels adjacent to the target pixel and the signal amount of two horizontal pixels adjacent to the target pixel And the difference is determined as a direction having continuity.If the difference is the same, the vertical and horizontal directions are determined as directions having continuity, and pixels in the determined direction are determined. The target pixel is interpolated by a signal amount obtained by averaging the signal amounts of the above.

According to the present invention, it is possible to provide a pixel interpolation method capable of sharply restoring the outline of an image after interpolation, improving resolution and suppressing the occurrence of false colors, and preventing deterioration in image quality due to interpolation. it can.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS According to the first aspect of the present invention, a pixel signal output from a target pixel other than a specific color of a color filter having three primary colors and having a specific color among the three primary colors arranged in a checkered pattern. Is interpolated based on a pixel signal output from a pixel of a specific color, a difference between signal amounts of two pixels in the vertical direction adjacent to the target pixel and two pixels in the horizontal direction adjacent to the target pixel. The difference between the signal amounts is compared, the smaller of the differences is determined as a direction having continuity, and if the difference is the same, the vertical and horizontal directions are determined as directions having continuity, and the determined direction is determined. A pixel interpolation method of interpolating the target pixel with a signal amount obtained by averaging the signal amounts of the pixels of the pixel signal, wherein the pixel signal output from the target pixel other than the specific color in the pixel signal output from the specific color The contour of the interpolated image after interpolation is restored sharply and the resolution Improved can be suppressed false color generation, it is possible to provide a pixel interpolation method for preventing deterioration of image quality due to interpolation.

According to a second aspect of the present invention, a pixel signal output from a color filter having three primary colors and a specific color among them arranged in a checkered pattern is a pixel output from a pixel other than the specific color. When interpolating based on the signal, the difference between the signal amount of two vertical pixels adjacent to the pixel other than the specific color and the difference of the signal amount of two horizontal pixels adjacent to the pixel other than the specific color is calculated. In comparison, the smaller of the differences is determined as a direction having continuity, and if the differences are the same, the vertical and horizontal directions are determined as directions having continuity, and the signal amount of pixels in the determined direction is determined. A two-dimensional contour component of the pixel of the specific color is extracted from the interpolated image obtained by interpolating the pixel other than the specific color with the averaged signal amount, and the pixels other than the specific color are extracted from the color filter based on the contour component. Pixel interpolation to interpolate the output pixel signal Even if the pixel signal output from the color filter is interpolated with the pixel signal output from the target pixel other than the specific color, the contour of the interpolated image is clearly restored and the resolution is improved. , The generation of false colors can be suppressed, and a pixel interpolation method that prevents the deterioration of image quality due to interpolation can be provided.

The invention according to claim 3 of the present invention is characterized in that R
A pixel signal output from a target pixel other than G (green) of a color filter having (red), G (green), and B (blue) and a specific color among them arranged in a checkered pattern is defined as G (green). When performing interpolation based on a pixel signal output from a pixel, a total of 2 pixels of 5 pixels vertically and 5 pixels horizontally centering on the pixel to be interpolated
Using the G pixels existing in the area of 5 pixels, a difference between two pixels in the vertical direction and a difference between two pixels in the horizontal direction among four G (green) pixels at the upper left of the pixel to be interpolated is calculated. Compare,
It is determined that the direction having the smaller difference has continuity, and if the difference is the same, both directions are determined to have continuity, and the upper right, lower left, and lower right areas of the pixel to be interpolated are determined. Similarly, the continuity is determined in the same manner, and the interpolation direction of each pixel to be interpolated is determined based on the above determined direction by averaging two pixels in the vertical direction, averaging two pixels in the horizontal direction,
This is a pixel interpolation method in which one of the pixel averages is selected and a G (green) interpolation signal of the pixel to be interpolated is set as a G (green) interpolation signal. The pixel signal output from the G (green) pixel is R (red) And a pixel that can clearly restore the contour of the interpolated image obtained by interpolating the pixel signal output from the B (blue) pixel, improve the resolution and suppress the generation of false colors, and prevent the deterioration of image quality due to the interpolation. An interpolation method can be provided.

The invention described in claim 4 of the present invention is characterized in that R
(Red), G (green), B (blue) color filters,
In a system for generating an RGB color image based on the signal output of a solid-state imaging device in which the array is a checkerboard array of G, the image obtained by image processing including interpolation processing and the like is used.
This is a pixel interpolation method in which a contour component of a two-dimensional G signal is extracted, and an R or B image signal is interpolated using the contour component. Even if the pixel signal output from the color filter is interpolated with the pixel signal, the contour of the interpolated image can be clearly restored, the resolution can be improved, and the generation of false colors can be suppressed. Can be provided.

According to a fifth aspect of the present invention, there is provided an image processing apparatus provided with an image interpolation processing section for performing processing according to the pixel interpolation method according to the first aspect. According to a sixth aspect of the present invention, there is provided an image processing apparatus provided with an image interpolation processing unit for performing processing according to the pixel interpolation method according to the second aspect.

Hereinafter, a pixel interpolation method and an image processing apparatus according to the present invention will be described based on specific embodiments. (Embodiment 1) An image processing apparatus 6 according to Embodiment 1 of the present invention
As shown in FIG. 1, R (red) and G as three primary colors
(Green) and B (blue), of which G is a specific color.
G of color filter 2 in which (green) is arranged in a checkered pattern
(Green) G in the vertical direction adjacent to the target pixel other than the pixel
The difference between the signal amounts of the two pixels of (green) and the difference of the signal amounts of the two pixels of G (green) in the horizontal direction adjacent to the target pixel are compared. Judging in a certain direction,
If the difference is the same, the vertical and horizontal directions are determined to have continuity, and the target pixel is interpolated with a signal amount obtained by averaging the signal amounts of G (green) pixels in the determined direction. A part 7 is provided.

Since the configuration is the same as the conventional one except for the image interpolation processing unit 7, a detailed description of the components other than the image interpolation processing unit 7 is omitted here. Here, processing of the image interpolation processing unit 7 for interpolating pixel signals output from R (red) and B (blue) pixels of the color filter 2 as target pixels with pixel signals output from G (green) pixels. The operation will be described.

As shown in FIG. 11, an image signal at the position of the G (green) pixel can directly obtain a signal for the G (green) pixel. For example, on the arrangement of the G (green) pixels, FIG.
When a signal having a distribution shown in FIG. 3 is input, as shown in FIG.
Of G (green) pixels existing in an area of a total of 25 pixels of 5 pixels vertically and 5 pixels centered on X, four G (green) pixels on the upper left centered on X are used for interpolation. The difference between the two pixels in the vertical direction of the target pixel to be processed and the difference between the two pixels in the horizontal direction
When comparing the difference between the pixels, as shown in FIG. 2, the difference between the two pixels in the vertical direction (4-4 = 0) is smaller than the difference between the two pixels in the horizontal direction (8-4 = 4). It is determined that the vertical direction indicated by the arrow A1 has continuity.

Similarly, the continuity of each of the upper right, lower left, and lower right target pixels is determined using four G pixels of upper right, lower left, and lower right with X as the center. Specifically, in the upper right, the direction in which the difference is small is continuous in the vertical direction indicated by arrow A2, in the lower left, the direction in which the difference is small is vertical in the direction indicated by arrow A3, and in the lower right, the direction in which the difference is small is continuous in the vertical direction indicated by arrow A4. Judge that there is.

In this way, for the remaining target pixels, the difference between two pixels in the vertical direction and the difference between two pixels in the horizontal direction are compared to determine a direction having continuity. FIG. 4 shows the continuity determination results.

Based on the result of this continuity determination, the amount of signal to be interpolated into the R (red) and B (blue) pixels of the color filter 2 is calculated. The target pixel determined to have continuity in the vertical direction is a signal amount obtained by averaging the sum of the two pixels in the vertical direction, and the difference between the two pixels in the vertical direction and the difference between the two pixels in the horizontal direction are the same. The target pixel determined to have continuity in the direction is a signal amount obtained by averaging the sum of the four pixels in the vertical and horizontal directions.
An interpolation signal for a G (green) pixel is generated. The interpolation signal of the G (green) pixel generated in this way has a distribution signal amount as shown in FIG. 5, and it can be seen that the signal amount of the distribution before interpolation shown in FIG. 2 is completely restored. .

With this configuration, the outline of the interpolated image obtained by interpolating the pixel signals output from the R (red) and B (blue) pixels with the pixel signals output from the G (green) pixels is clearly restored. Therefore, the resolution can be improved, the generation of false colors can be suppressed, a high-quality image signal can be generated, and the deterioration of image quality due to interpolation can be prevented.

In addition, since the structure is simple, real-time driving by hardware is possible, and an interpolation method is automatically determined in response to a change in the situation of an imaging screen, so that high-quality video signals can be processed at high speed. Is possible.

(Embodiment 2) As shown in FIG. 6, an image processing apparatus according to Embodiment 2 of the present invention employs a color filter 2 of Embodiment 1 other than G (green) as a specific color. The difference between the signal amounts of two pixels in the vertical direction adjacent to the pixel and G
The difference between the signal amounts of two pixels in the horizontal direction adjacent to the pixel other than (green) is compared, and the smaller of the differences is determined as the direction having continuity. Is determined as a direction having continuity, and a two-dimensional G (green) pixel is obtained from an interpolated image obtained by interpolating pixels other than G (green) with a signal amount obtained by averaging the signal amounts of the pixels in the determined direction. An image interpolation processing unit 9 is provided for extracting a contour component and interpolating a pixel signal output from a color filter with R (red) pixels as pixels other than G (green) based on the contour component. .

Since the image interpolation processing unit 9 is the same as the image interpolation processing unit 7 of the first embodiment except for the image interpolation processing unit 9, a detailed description of the components other than the image interpolation processing unit 9 is omitted here. .

Here, instead of the interpolation of G (green) in the first embodiment, pixel signals from R (red) pixels are output from G (green) and B (blue) pixels of the color filter 2. The processing operation of the image interpolation processing unit 9 for interpolating the pixel signals will be described. Note that R (red) and B (blue) of the color filter 2
Embodiment 1 described above up to the processing of interpolating the pixel signal output from the pixel with the pixel signal output from the G (green) pixel
Since the processing is the same as that described above, the subsequent processing will be specifically described.

As shown in FIG. 5, based on the interpolation result of the G (green) signal, the G (green) signal
FIG. 7 shows a result obtained by extracting a horizontal edge signal as a two-dimensional contour component of a (green) pixel, and halving the result of applying a commonly used secondary high-pass filter. . When this value is added (subtracted) to the R (red) signal obtained by simply averaging four pixels, a signal as shown in FIG. 8 is obtained.
Comparing the result shown in FIG. 8 with the distribution before interpolation shown in FIG. 2, it can be seen that the R (red) signal is completely restored.

With this configuration, the outline of the interpolated image obtained by interpolating the pixel signals output from the G (green) and B (blue) pixels with the pixel signals output from the R (red) pixels is clearly restored. Therefore, the resolution can be improved, the generation of false colors can be suppressed, a high-quality image signal can be generated, and the deterioration of image quality due to interpolation can be prevented. In addition, the simple structure enables real-time driving by hardware, and automatically determines the interpolation method in response to changes in the situation of the imaging screen, enabling high-speed processing of high-quality video signals. is there.

In the second embodiment, the interpolation is performed using the R (red) signal.
Since the arrangement is the same as that of (red), interpolation can be performed by performing the same processing, and the same effect is obtained.

In each of the above-described embodiments, a checkered color filter in which a specific color of the three primary colors is G (green) is used, but a checkered color filter in which R (red) or B (blue) is used. The same effect is obtained even when the color filters are arranged.

[0031]

As described above, according to the pixel interpolation method of the present invention, a pixel signal output from a target pixel other than a specific color of a color filter having three primary colors and having a specific color among them arranged in a checkered pattern is obtained. In performing interpolation based on a pixel signal output from a pixel of a specific color, a difference between signal amounts of two vertical pixels adjacent to the target pixel and a signal of two horizontal pixels adjacent to the target pixel are used. Compare the difference of the amount, the smaller of the difference is determined as a direction having continuity, if the difference is the same, the vertical and horizontal directions are determined as a direction having continuity, the direction of the determined direction By interpolating the target pixel with a signal amount obtained by averaging the pixel signal amounts, the interpolated image of the pixel signal output from the target pixel other than the specific color is interpolated with the pixel signal output from the specific color. The outline can be clearly restored, and the resolution Can be above, it is possible to suppress the occurrence of a false color, it is possible to generate an image signal of high quality, it is possible to provide a pixel interpolation method for preventing deterioration of image quality due to interpolation. In addition, the simple structure enables real-time driving by hardware, and automatically determines the interpolation method in response to changes in the situation of the imaging screen, enabling high-speed processing of high-quality video signals. is there.

Further, a pixel signal output from a color filter having three primary colors and a specific color among them arranged in a checkered pattern is interpolated based on pixel signals output from pixels other than the specific color. Also, comparing the difference between the signal amounts of two pixels in the vertical direction adjacent to the pixel other than the specific color and the difference between the signal amounts of two pixels in the horizontal direction adjacent to the pixel other than the specific color, Is determined as the direction having continuity, and if the difference is the same, the vertical and horizontal directions are determined as the direction having continuity, and the signal amount obtained by averaging the signal amounts of the pixels in the determined direction is used. Extracting a two-dimensional contour component of the pixel of the specific color from the interpolated image obtained by interpolating pixels other than the specific color,
By interpolating a pixel signal output from a color filter with a pixel other than the specific color based on the contour component, a pixel signal output from the color filter with a pixel signal output from a target pixel other than the specific color Has the same effect as described above.

Further, the image processing apparatus of the present invention has a difference in signal amount between two pixels in the vertical direction adjacent to a target pixel other than the specific color of a color filter having three primary colors and having the specific colors arranged in a checkered pattern. And the difference between the signal amounts of the two pixels in the horizontal direction adjacent to the target pixel, and the smaller of the differences is determined as the direction having continuity. Is determined as a direction having continuity, and an image interpolation processing unit is provided for interpolating the target pixel with a signal amount obtained by averaging the signal amounts of the pixels in the determined direction, so that the pixel interpolation method of the present invention is realized. it can.

The difference between the signal amounts of two pixels in the vertical direction adjacent to the pixels other than the specific color of the color filter having the three primary colors and having the specific colors arranged in a checkered pattern, and the pixels other than the specific color Is compared with the difference between the signal amounts of the two pixels in the horizontal direction that are adjacent to each other, and the smaller of the differences is determined as the direction having continuity, and if the difference is the same, the vertical and horizontal directions are the directions having continuity. And extracting a two-dimensional contour component of the pixel of the specific color from an interpolated image obtained by interpolating pixels other than the specific color with a signal amount obtained by averaging the signal amounts of the pixels in the determined direction,
In the case where an image interpolation processing unit that interpolates a pixel signal output from a color filter with a pixel other than the specific color based on the outline component is provided, a pixel signal output from the color filter is converted into a pixel other than the specific color. The pixel interpolation method according to the present invention in which interpolation is performed based on the pixel signal output from the device can be realized.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus according to a first embodiment of the present invention.

FIG. 2 is a diagram in which an input signal amount of the solid-state imaging device according to the first embodiment is modeled by numerals;

FIG. 3 is an image diagram showing a continuity determination portion according to the first embodiment;

FIG. 4 is an image diagram showing a determination result of continuity according to the first embodiment;

FIG. 5 is a diagram showing a G (green) signal amount after interpolation according to the embodiment;

FIG. 6 is a block diagram illustrating a configuration of an image processing apparatus according to a second embodiment of the present invention;

FIG. 7 is a diagram modeling a contour of a G (green) signal according to the second embodiment;

FIG. 8 is a diagram showing an R (red) signal amount after interpolation according to the embodiment;

FIG. 9 is a block diagram illustrating a configuration of a conventional image processing apparatus.

FIG. 10 is a diagram showing an RGB arrangement of a conventional color filter.

FIG. 11 is a diagram showing a filter array in which only a G array is extracted from a color filter.

FIG. 12 is a diagram illustrating a filter array in which only an R array is extracted from a color filter.

FIG. 13 is a diagram in which the input signal amount of a conventional solid-state imaging device is modeled by numerals.

FIG. 14 shows G after interpolation performed by a conventional pixel interpolation unit.
(Green) Diagram showing signal volume

FIG. 15 is a diagram in which the input signal amount of a conventional solid-state imaging device is modeled by numerals.

FIG. 16 shows an interpolated R interpolated by a conventional pixel interpolating unit.
(Green) Diagram showing signal volume

[Explanation of symbols]

 2 Color filter 6 Image processing device 7 Pixel interpolation processing unit 8 Image processing device 9 Pixel interpolation processing unit

 ──────────────────────────────────────────────────の Continued on the front page (72) Inventor Yoshihisa Shimazu 1006 Kazuma Kadoma, Osaka Pref. Matsushita Electric Industrial Co., Ltd. Term (reference) 5B050 AA04 BA11 DA04 EA13 EA16 5C065 AA01 AA03 BB10 CC01 CC08 DD01 DD17 EE05 EE06 GG13 GG17 GG22 5C066 AA01 BA20 CA05 DC06 EC01 EC11 EF14 EF15 GA01 KD02 KD06 KE05 KM02 KM05 K05 KM05 K05 KM05

Claims (6)

[Claims] 1. A pixel signal output from a target pixel other than a specific color of a color filter having three primary colors and having specific colors arranged in a checkered pattern based on pixel signals output from pixels of a specific color. When interpolating, the difference between the signal amounts of two pixels in the vertical direction adjacent to the target pixel and the difference between the signal amounts of two horizontal pixels adjacent to the target pixel are compared, and the difference is small. Is determined as the direction having continuity, and if the difference is the same, the vertical and horizontal directions are determined as the direction having continuity, and the signal amount is obtained by averaging the signal amounts of the pixels in the determined direction. A pixel interpolation method for interpolating pixels. 2. When interpolating a pixel signal output from a color filter having three primary colors and a specific color among them arranged in a checkered pattern based on a pixel signal output from a pixel other than the specific color, The difference between the signal amounts of the two pixels in the vertical direction adjacent to the pixel other than the color is different from the difference between the signal amounts in the horizontal direction adjacent to the pixel other than the specific color.
The difference between the signal amounts of the pixels is compared with each other, and the smaller one of the differences is determined as the direction having continuity, and if the difference is the same, the vertical and horizontal directions are determined as the direction having continuity. The two-dimensional contour component of the pixel of the specific color is extracted from the interpolated image obtained by interpolating the pixel of the specific color with the signal amount obtained by averaging the signal amount of the pixel in the direction determined, and the identification is performed based on the contour component. A pixel interpolation method for interpolating pixel signals output from a color filter with pixels other than color. 3. A color filter G having R (red), G (green), and B (blue), wherein specific colors among them are arranged in a checkered pattern.
A pixel signal output from a target pixel other than (green) is G
When performing interpolation based on the pixel signal output from the (green) pixel, G pixels existing in an area of a total of 25 pixels of 5 pixels vertically and 5 pixels horizontally around the pixel to be interpolated are used,
By comparing the difference between two pixels in the vertical direction and the difference between two pixels in the horizontal direction among the four G (green) pixels at the upper left of the pixel to be interpolated, it is determined that the direction having the smaller difference has continuity. If the difference is the same, the two directions are determined to have continuity, and the continuity is similarly determined for the upper right, lower left, and lower right areas of the pixel to be interpolated. Based on the determined direction, the interpolation direction of each pixel to be interpolated is calculated by averaging two pixels in the vertical direction and two pixels in the horizontal direction.
A pixel interpolation method in which one of pixel average and vertical and horizontal four pixel average is selected and used as a G (green) interpolation signal of a pixel to be interpolated. 4. An RGB color image based on a signal output of a solid-state imaging device in which R (red), G (green), and B (blue) color filters are arranged and the arrangement is a G checkered arrangement. A pixel interpolation method for extracting a contour component of a two-dimensional G signal obtained by image processing including interpolation processing or the like, and interpolating an R or B image signal using the contour component. 5. An image processing apparatus provided with an image interpolation processing section for performing processing according to the pixel interpolation method according to claim 1. 6. An image processing apparatus provided with an image interpolation processing unit for performing processing according to the pixel interpolation method according to claim 2.