JP3406674B2 - Two-chip imaging device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a two-plate type image pickup device, and more particularly to improvement of color resolution of an image.

[0002]

The human eye is less sensitive to color than it is to luminance. Therefore, NTSC (National T
In the standards such as the Elevision System Committee), the color resolution is set lower than the luminance resolution.
In the plate type image pickup device, the G and R / B methods are mainly used. That is, in this image pickup apparatus, as shown in FIG. 14, for example, a filter F1 that transmits green light in all pixels transmits red (red) light onto one CCD (solid-state image sensor). A filter F2 in which the pixels to be turned on and the pixels to transmit blue light are alternately arranged is provided on another CCD. These CC
The output signal of D is superimposed on the corresponding pixels in the video processing circuit. That is, the brightness of one pixel on the CCD is represented by the signal from one pixel, but the color is the signal of two adjacent pixels (R corresponding to red).
(A pixel signal including a signal and a pixel signal including a B signal corresponding to blue).

[0003]

As described above, the conventional apparatus is constructed so that the color signal is generated by synthesizing the information of two pixels, and the information amount of the color signal is sufficient as compared with the luminance signal. Not so, good image quality is not obtained. When an image signal having such a low color resolution is input to a computer and image processing is performed on a pixel-by-pixel basis, deterioration of the image quality is often noticeable. To deal with this, it is possible to obtain a RGB signal by adopting a so-called three-plate type image pickup device in which three CCDs are provided and each CCD is provided with a filter for transmitting red, green and blue respectively. Such a configuration causes a problem that the scale of the device becomes large and the circuit configuration becomes complicated.

Further, in the conventional apparatus, the sensitivity is determined according to the performance of the optical system such as the taking lens and the prism, and there is a problem that an image having sufficient contrast cannot be obtained when the brightness of the subject is too low.

According to the present invention, it is possible to increase the color resolution without complicating the circuit structure, to generate an image signal whose image quality does not deteriorate even when image processing is performed by a computer, etc. It is an object of the present invention to provide a two-plate type imaging device capable of performing the above.

[0006]

A first two-plate type image pickup device according to the present invention has complementary color difference line-sequential pixel spectral characteristics that change regularly, and the pixel spectral characteristics are horizontal for one pixel. First and second image sensors that are offset in direction,
Based on signals of a total of 4 pixels obtained from the first and second image sensors for 2 pixels arranged in the vertical direction, 1
First color signal extraction means for extracting the R signal, G signal and B signal of the pixel, and 2 in each of the horizontal and vertical directions
Second color signal extraction means for extracting the R signal, G signal and B signal of one pixel on the basis of signals of a total of eight pixels obtained from the first and second image sensors for four pixels arranged side by side, And a control unit for selecting and operating one of the first and second color signal extraction units.

A second two-plate type image pickup device according to the present invention has a complementary-color-difference line-sequential pixel spectral characteristic that changes regularly, and the pixel spectral characteristics are shifted from each other by one pixel in the horizontal direction. R of 1 pixel based on a signal obtained from a total of 4 pixels of two pixels arranged in the vertical direction at positions where the first and second image sensors and the first and second image sensors are optically equal to each other. A total of four pixels lined up in the horizontal direction and in the vertical direction at positions where the first color signal generating means for generating the signal, the G signal and the B signal and the first and second image sensors are optically equal to each other. R signal of 1 pixel, G based on the signal obtained from 8 pixels
First color signal generating means for generating a signal and a B signal,
And a control means for selecting and operating one of the first and second color signal generating means.

A third two-plate type image pickup device according to the present invention has a complementary-color-difference line-sequential pixel spectral characteristic that regularly changes, and the pixel spectral characteristics are shifted from each other by one pixel in the horizontal direction. R of 1 pixel based on a signal obtained from a total of 4 pixels of two pixels arranged in the vertical direction at positions where the first and second image sensors and the first and second image sensors are optically equal to each other. A first color signal generating means for generating a signal, a G signal and a B signal, and a vertical in the first and second image sensors respectively adjacent to the two pixels used in the first color signal generating means. R of 1 pixel based on a signal obtained from a total of 8 pixels including a total of 4 pixels of 2 pixels arranged in each direction and the 4 pixels used in the first color signal generation means.
It is characterized in that it is provided with a second color signal generation means for generating a signal, a G signal and a B signal, and a control means for selecting and operating one of the first and second color signal generation means.

[0009]

The present invention will be described below with reference to illustrated embodiments. FIG. 1 is a block diagram of a still video camera which is an embodiment of the present invention.

The system control circuit 10 is a microcomputer and controls the entire still video camera.

The image pickup optical system 11 includes a lens 12 and a diaphragm 13. The lens 12 is driven by the zoom drive circuit 14 during the zooming operation, and is driven by the focus drive circuit 15 during the focusing operation. The aperture of the diaphragm 13 is adjusted by the iris drive circuit 16 during exposure control.
The zoom drive circuit 14, the focus drive circuit 15, and the iris drive circuit 16 are controlled by the system control circuit 10.

The light beam that has passed through the image pickup optical system 11 is guided to the first and second CCDs (image sensors) 22 and 23 through the prism 21, and the same subject image is formed on these CCDs 22 and 23. It Further, this light beam is guided to the finder optical system 25 via the prism 21 and the mirrors 24 and 29. First and second CCDs 22, 2
3 are provided with filters 51 and 52, respectively. The CCDs 22 and 23 are driven by the CCD driver 26, and thereby, the image signals corresponding to the subject images formed on the CCDs 22 and 23 are supplied to the correlated double sampling (CDS) circuits 31 and 32. The CCD driver 26 operates by the pulse signal output from the synchronization signal generation circuit 27 controlled by the system control circuit 10.

The image signals input to the CDS circuits 31 and 32 are subjected to predetermined processing such as gamma correction in the preprocessing circuits 33 and 34 after the reset noise is removed. Then, this image signal is converted into a digital signal in the A / D converters 35 and 36, and the image memories 41 to 44.
Stored in. Image memory 41 to which image signals are stored
The address of 44 is controlled by the system control circuit 10 via the address control circuit 45.

The video processing circuit 46 includes image memories 41-4.
The image signal stored in 4 is subjected to the processing described later, whereby the R signal, the G signal, and the B signal are output together with the luminance signal. These signals are output to the computer or display device via the interface circuit.

The manual switch 47 connected to the system control circuit 10 is provided for operating the present still video camera, and the display element 48 is provided for displaying the content of the operation by the manual switch 47 and the like. The mode switching switch 49 is a switch for switching shooting by the still video camera between the high resolution mode and the high sensitivity mode.

The prism 21 has first and second light separation surfaces 21a and 21b.
By the action of a and 21b, the first and second CCD2
The amount of light guided to 2, 23 and the finder optical system 25 is separated into a ratio of 4: 4: 2, for example, and light of the same intensity is guided to each CCD 22, 23. That is, a part of the light rays that have entered the prism 21 from the imaging optical system 11 are reflected by the first light splitting surface 21a and guided to the second CCD 23, and the other light rays pass through the first light splitting surface 21a. The light is transmitted and guided to the second light separation surface 21b. At the second light separation surface 21b, a part of the light rays is reflected and guided to the first CCD 22, and the other light rays, that is, the first and second light separation surfaces 21a and 21b.
The light beam that has passed through is emitted to the outside of the prism 21.
This light beam is reflected by the mirrors 24 and 29 and guided to the finder optical system 25. The mirrors 24 and 29 are members for shifting the optical axis of the finder optical system 25 from the optical axis of the photographing optical system 11, and can be omitted.

It is preferable that the light amounts guided to the first and second CCDs 22 and 23 are the same, but they are not necessarily the same, and when these light amounts are different, CC
The magnitudes of these output signals may be made the same by adjusting the gains of the outputs of D22 and D23.

FIG. 2 shows the first and second CCDs 22 and 2.
3 shows an array of color filters 51, 52 provided on the light receiving surface of No. 3. These color filters 51,
Reference numeral 52 is a complementary color checkered color filter having the same configuration. In these color filters 51 and 52, filter elements that transmit magenta (Mg), yellow (Ye), cyan (Ce), and green (G) are alternately arranged. That is, in addition to green (G), magenta (Mg), yellow (Ye), and cyan (Ce) having different spectral characteristics of complementary colors are included in a total of 4 pixels in which two pixels are arranged in each of the horizontal direction and the vertical direction. Three pixels are provided.

The positional relationship between the second color filter 52 and the CCD 23 is shown by the CCD of the first color filter 51.
Compared with the positional relationship with respect to 22, the second color filter 52 is provided so as to be shifted by one pixel with respect to the CCD 23 in the horizontal direction (left direction in FIG. 2). For example, paying attention to the pixel P at the upper left corner of the screen, the first color filter 51 is magenta, but the second color filter 52 is green.

As described above, the pixel spectral characteristics of the CCDs 22 and 23 are regularly changed and are of the complementary color difference line sequential type. Further, the pixel spectral characteristic of the second CCD 23 is horizontally displaced from the pixel spectral characteristic of the first CCD 22 by one pixel. Therefore, the four pixels of the two pixels arranged in the vertical direction of the first CCD 22 and the two pixels arranged in the vertical direction of the second CCD 23 at the same optical position as the two pixels.
The pixels have different spectral characteristics. That is, for example, when one of the two pixels is Mg and Ye, the other two pixels are G and Cy.

The output signal of the first CCD 22 and the second CC
The output signal of D23 is converted into a digital signal by the image memory 4
1 to 44 are temporarily stored, they are read from these memories and processed in the video signal processing circuit 46. That is, the corresponding pixels are superimposed on each other, and the R signal, the G signal, and the B signal corresponding to each pixel are extracted from the superimposed signals to obtain a video signal.

FIG. 3 shows this superposed state. As understood from this figure, the magenta (Mg) of the filter 51 of the first CCD 22 and the green (G) of the filter 52 of the second CCD 23 are the same as the green (G) of the filter 51 and the magenta (Mg) of the filter 52. ), Yellow (Ye) of the filter 51 and cyan (Cy) of the filter 52 correspond to the same pixel, and cyan (Cy) of the filter 51 and yellow (Ye) of the filter 52 correspond to the same pixel, respectively. In FIG. 3, Px is a horizontal pixel interval (1 pitch), and Py is a vertical pixel interval (1 pitch).
Pitch) respectively.

Next, the extraction of the R signal, G signal and B signal when the mode changeover switch 49 is set to the high resolution mode will be described. First, the extraction of the R signal will be described. The R signal contained in magenta (Mg) is converted to R _Mg ,
B signal is B _Mg , R signal included in yellow (Ye) is R
_{When Ye} , G signal are G _Ye , G signal contained in cyan (Cy) is G _Cy , and B signal is B _Cy , Mg = R _Mg + B _Mg , Ye = R _Ye + G _Ye , Cy = G _Cy + B
It can be represented as _Cy .

The R signal is a magenta (M
g) and yellow (Ye), and the four pixels of green (G) and cyan (Cy) (pixels shaded in FIG. 3) superimposed on them are obtained by the following formula. R _S = (Mg + Ye) -α (G + Cy) = R _Mg + B _Mg + R _Ye + G _Ye −αG-αG _Cy −αB _Cy = R _Mg + R _Ye + G _Ye −α (G + G _Cy ) + B _Mg −αB _Cy = R _Mg + R _Ye (1) However, in order for this expression (1) to hold, the condition is that α = G _Ye / (G + G _Cy ) = B _Mg / B _Cy .

Similarly, the B signal can be obtained by the following equation. B _S = (Mg + Cy) -β (G + Ye) = R _Mg + B _Mg + G _Cy + B _Cy -βG-βR _Ye -βG _Ye = B _Mg + B _Cy + G _Cy -β (G + G _Ye ) + R _Mg -βR _Ye = B _Mg + B _Cy (2) However, in order for this equation (2) to hold, it is a condition that β = G _Cy / (G + G _Ye ) = R _Mg / R _Ye holds.

Regarding the G signal, a luminance signal (Y) and
It is obtained from R _S and B _S obtained by the equations (1) and (2). That is, G _S = Y−R _S −B _S = (Mg + Cy + G + Ye) −R _S −B _S = G + G _Ye + G _Cy (3)

Next, consider the spectra of the RGB signal and the luminance signal Y thus obtained.

First, the spectrum of the luminance signal Y will be described.
Let S _O (x, y) = Σ _m Σ _n δ (x−2mP _x , y−4 nP _y ), where δ is a delta function, x is a horizontal coordinate, and y is a vertical direction. Coordinates, m and n are integers). And M
The optical image distributions for the g, G, Ye, and Cy pixels are I _A1Mg (x, y) and I _A1G (x, respectively) for the first CCD 22.
y), I _A1Ye (x, y), I _A1Cy (x, y), and the second C
For CD23, I _A2Mg (x, y) and I _A2G (x, y, respectively)
y), I _A2Ye (x, y), and I _A2Cy (x, y), and spectra are obtained for a combination of 4 pixels which are connected by a line L in FIG.

Here, the first and second CCDs 22, 2
Assuming that the output of each pixel of 3 is equal, I _AMg (x, y) = I _A1Mg (x, y) = I _A2Mg (x, y) I _AG (x, y) = I _A1G (x, y ) = I _A2G (x, y) I _AYe (x, y) = I _A1Ye (x, y) = I _A2Ye (x, y) I _ACy (x, y) = I _A1Cy (x, y) = I _A2Cy Let (x, y). Then, the luminance signal components of the first and second fields are represented by I _Y1 (u, v) = I _AMg (u, v) exp (-jP _y v) + I _AG (u, v) exp (-j
P _y v) ＋ I _AYe (u, v) ＋ I _ACy (u, v) I _Y2 (u, v) ＝ I _AMg (u, v) ＋ I _AG (u, v) ＋ I _AYe (u, v) ex
p (-jP _y v) + I _ACy (u, v) exp (-jP _y v) (where u is the horizontal angular spatial frequency, v is the vertical angular spatial frequency, and j is an imaginary number), First and second
The luminance signal spectrum of the field is Y ₁ (u, v) = I _Y1 (u, v) * {S _O (u, v) (1 + exp (-jP _x u)) (1 + exp (-j2P _y v))} = I _Y1 (u, v) * {S _O (u, v) 4exp (-jP _x u / 2) exp (-jP _y v) ・ cos (P _x u / 2) cos (P _y v)} ( 4) Y ₂ (u, v) = I _Y2 (u, v) * {S _O (u, v) exp (-jP _y v) (1 + exp (-jP _x u)) ・ (1 + exp (-j2P _y v ))} = I _Y2 (u, v) * {S _O (u, v) 4exp (-jP _x u / 2) exp (-j2P _y v) ・ cos (P _x u / 2) cos (P _y v )} (5) Where S _O (u, v) is the basic sampling sequence S
_In the spectrum of _O (x, y), S _O (u, v) = (1 / 8P _x P _y ) Σ _m Σ _n δ (u −2πm / 2
p _x , v−2πn / 4P _y ). The * symbol represents convolution integral.

From the expressions (4) and (5), the components of 1 / 2Px, 1 / 4Py, and 3 / 4Py disappear in the luminance signal spectrum, and (0, 1 / Px, 1 / Py) I _Y1
(u, v) + I _Y2 (u, v)) The spectrum is convoluted (convolution integral), and at (1 / 2Py) (I
It can be seen that the _Y1 (u, v) -I _Y2 (u, v)) spectrum is combo rooted. Therefore, the luminance signal spectrum is as shown in FIG. In FIG. 5, the horizontal axis is the spatial frequency in the horizontal direction, and the vertical axis is the spatial frequency in the vertical direction.

As shown in this figure, in the horizontal direction, (I _Y1 (u, v) + I _Y2 (u,
v)) signal sideband components appear, and (I _Y1 (u, v) −I _Y2 (u,
The sideband component of the v)) signal appears. In addition, this 1/2
The amplitude of the sideband component at Py is small, and the image distribution I
_{As A} (x, y) is flat (uniform), v = 2πm / P
In case of frequency of y (m is an integer), Mg + G and Ye + C
When the values of y are equal, I _Y1 = I _Y2 , so this sideband component disappears.

The spectrum of the R signal is also obtained in the same manner as the case of the luminance signal. That is, the first and second C
Assuming that the output of each pixel of CD22 and CD23 is equal,
And R signal component of the second field as I _R1 (u, v) = I _AMg (u, v) exp (-jP _y v) −α I _AG (u, v) ex
p (-jP _y v) + I _AYe (u, v) -αI _ACy (u, v) I _R2 (u, v) = I _AMg (u, v) -αI _AG (u, v) + I _AYe (u, v)
exp (-jP _y v) −αI _ACy (u, v) exp (-jP _y v), the spectrum of the R signal in the first and second fields is R ₁ (u, v) = I _R1 ( _{u, v) * {S O} (u, v) (1 + exp (-jP x u)) (1 + exp (-j2P y v))} = I R1 (u, v) * {S O (u, v) 4exp (-jP _x u / 2) exp (-jP _y v) ・ cos (P _x u / 2) cos (P _y v)} (6) R ₂ (u, v) = I _R2 (u, v) * _{{S O (u, v)} exp (-jP y v) (1 + exp (-jP x u)) · (1 + exp (-j2P y v))} = I R2 (u, v) * {S O (u, _{v) 4exp (-jP x u /} 2) exp (-j2P y v) · cos (P x u / 2) cos (P y v)} and comprising (7).

As understood from the equations (6) and (7), the spectrum of the R signal appears at the same place as the luminance signal (FIG. 6 (a)). Further, when the image distribution I _A (x, y) has a frequency of v = 2πm / Py (m is an integer) such as flat, or when the values of Mg−αG and Ye−αCy are equal,
Since I _R1 = I _R2 , the 1/2 Py component disappears due to the interlacing as well as the luminance signal.

The spectrum of the B signal is also obtained in the same manner as the case of the luminance signal. That is, the first and second C
Assuming that the output of each pixel of CD22 and CD23 is equal,
And the B signal component of the second field, I _B1 (u, v) = I _AMg (u, v) exp (-jP _y v) −βI _AG (u, v) exp
(-jP _y v) −βI _AYe (u, v) + I _ACy (u, v) I _B2 (u, v) = I _AMg (u, v) −βI _AG (u, v) −βI _AYe (u, v)
If exp (-jP _y v) + I _ACy (u, v) exp (-jP _y v) is set, the spectrum of the B signal in the first and second fields is B ₁ (u, v) = I _B1 (u , v) * {S _O (u, v) (1 + exp (-jP _x u)) (1 + exp (-j2P _y v))} = I _B1 (u, v) * {S _O (u, v) 4exp ( -jP _x u / 2) exp (-jP _y v) -cos (P _x u / 2) cos (P _y v)} (8) B ₂ (u, v) = I _B2 (u, v) * { S _O (u, v) exp (-jP _y v) (1 + exp (-jP _x u)) ・ (1 + exp (-j2P _y v))} = I _B2 (u, v) * {S _O (u, v _{) 4exp (-jP x u / 2} ) exp (-j2P y v) · cos (P x u / 2) cos (P y v)} is (9).

As can be understood from the equations (8) and (9), the spectrum of the B signal appears at the same position as the luminance signal as in the case of the R signal (FIG. 6B). Also the image distribution I
_{When A} (x, y) has a frequency of v = 2πm / Py (m is an integer) such as flat, Mg−βG and −βYe + Cy
When the values are the same, I _B1 = I _B2 , so that the 1/2 Py component disappears due to the interlacing as well as the luminance signal.

The spectrum of the G signal can also be obtained in the same manner as described above. That is, the first and second CCDs
Assuming that the output of each pixel of 22 and 23 is equal, the spectrum of the B signal component of the first and second fields is calculated as I _G1 (u, v) = I _Y1 (u, v) -I _R1 (u, v ) −I _B1 (u, v) ＝ −
I _AMg (u, v) exp (-jP _y v) + (1-α-β) I _AG (u, v) exp (-jP
_y v) + (2-β) I _AYe (u, v) + (2-α) I _ACy (u, v) I _G2 (u, v) = I _Y2 (u, v) -I _R2 (u, v) −I _B2 (u, v) ＝ −
I _AMg (u, v)-(1-α-β) I _AG (u, v) + (2-β) I
_AYe (u, v) ・ exp (-jP _y v) ＋ (2-α) I _ACy (u, v) exp (-jP
_y v), G ₁ (u, v) = I _G1 (u, v) * {S _O (u, v) (1 + exp (-jP _x u)) (1 + exp (-j2P _y v) )} = I _G1 (u, v) * {S _O (u, v) 4exp (-jP _x u / 2) exp (-jP _y v) ・ cos (P _x u / 2) cos (P _y v) } (10) G ₂ (u, v) = I _G2 (u, v) * {S _O (u, v) exp (-jP _y v) (1 + exp (-jP _x u)) ・ (1 + exp (-j2P _y v))} = I _G2 (u, v) * {S _O (u, v) 4exp (-jP _x u / 2) exp (-j2P _y v) ・ cos (P _x u / 2) cos (P _y v)} (11).

As can be understood from the equations (10) and (11), a spectrum appears in the G signal at the same place as the luminance signal as in the R signal and the B signal (FIG. 6C). Also, the image distribution I _A (x, y) is v = 2πm such as flat.
In the case of the frequency of / Py (m is an integer), -Mg + (1-
If the values of α-β) G and (2-α) Cy are equal, I
_{Since G1} = I _G2 , the 1/2 Py component disappears due to the interlacing as well as the luminance signal.

Next, the spectra of the respective color signals according to the present embodiment shown in FIGS. 6A to 6C are compared with the spectra when a G, R / B type filter as shown in FIG. 14 is used. Compare.

FIG. 7 shows the result of analyzing the spectrum in the case of using the G, R / B type filter by the same method as described above. As shown in this figure, in the R signal and the B signal, the sideband component is (1 / 2Px, 1 / 4P
y). Therefore, according to Nyquist's theorem, the cut-off frequency of the low-pass filter provided to cut the sideband wave component to obtain the fundamental wave component of the color signal is 1 / 4Px in the horizontal direction and 1 / 8Py in the vertical direction. Limited to.

On the other hand, according to the structure using the complementary color checkered color filter as in the present embodiment, as shown in FIG. 6, the sideband components are (1 / Px, 0) and (1 / 2Py, 0). Therefore, the cutoff frequency of the low-pass filter provided to obtain the fundamental wave component of the color signal is 1/2 in the horizontal direction.
Px may be lower than 1/4 Py in the vertical direction. That is, according to the present embodiment, the cutoff frequency of the low-pass filter can be increased by a factor of two as compared with the comparative example of FIG. 7, whereby the spectral range of the color signals reproducible by the CCDs 22 and 23 is the same as the conventional one. It has been enlarged and the resolution has improved.

Regarding the G signal, in the present embodiment, (0, 1
/ 2Py), a sideband component appears at the position where the sideband component does not appear at this position in the comparative example, but the sideband component in the vertical direction disappears due to interlacing as described above. It doesn't really matter.

FIG. 8 shows the reproducible spectral range in this example and the comparative example. In the comparative example, the spectral range of the color signal is ¼ Px as indicated by a broken line B1.
However, in this embodiment, it is enlarged to 1/2 Px as shown by the broken line B2. The spectral range of the luminance signal is shown by the solid lines S1 and S in the comparative example and the present example.
It is the same as indicated by 2.

Next, the actual method of extracting the RGB signals will be described with reference to FIG. In the pixel arrangement of this figure,
The first and second CCDs 22 and 23 are represented by parameters A and B, respectively, the horizontal direction is represented by a parameter i, and the vertical direction is represented by a parameter j. In this figure, among the pixels shown in an overlapping manner, the one located above is the first C
Corresponding to the CD22, the one below is the second CCD
23. The signal from each pixel is
For the CCD 22 of, VA, i, j = Mg, VA, i + 1, j = G VA, i, j + 1 = Ye, VA, i + 1, j + 1 = Cy VA, i, j + 2 = G, VA, i + 1, j + 2 = Mg VA, i, j + 3 = Ye, VA, i + 1, j + 3 = Cy, and for the second CCD 23, VB , i, j = G, VB, i + 1, j = Mg VB, i, j + 1 = Cy, VB, i + 1, j + 1 = Ye VB, i, j + 2 = Mg, VB, i The arrangement is such that + 1, j + 2 = G VB, i, j + 3 = Cy and VB, i + 1, j + 3 = Ye. Where i = 1,3,5, ...; j = 1,5,
It shall take the values of 9, ...

In the first scan, the signal of the first field, that is, VA, i, j = Mg, VA, i + 1, j = G VA, i, j + 2 = G, VA, i + 1, j + 2 = Mg VB, i, j = G, VB, i + 1, j = Mg VB, i, j + 2 = Mg, VB, i + 1, j + 2 = G are extracted, and in the second scan Signal of the second field, that is, VA, i, j + 1 = Ye, VA, i + 1, j + 1 = Cy VA, i, j + 3 = Ye, VA, i + 1, j + 3 = Cy VB , i, j + 1 = Cy, VB, i + 1, j + 1 = Ye VB, i, j + 3 = Cy, VB, i + 1, j + 3 = Ye are extracted.

The above all pixel signals are used as the image memories 41 to 44.
Memorized in. These pixel signals are sent to the video processing circuit 46.
Read out and calculated, for the i-th pixel of the odd-even scan line of the odd field, Ri, k = (VA, i, j + VA, i, j + 1) -α (VB, i, j + VB, i, j + 1) = (Mg + Ye) -α (G + Cy) (12) Ri, k + 1 = (VB, i, j + 2 + VA, i, j + 3) -α (VA, i, j + 2 + VB, i, j + 3) = (Mg + Ye) −α (G + Cy) (13) Bi, k = (VA, i, j + VB, i, j + 1) −β (VB, i, j + VA, i, j + 1) = (Mg + Cy) -β (G + Ye) (14) Bi, k + 1 = (VB, i, j + 2 + VB, i, j + 3) -β (VA, i, j +2 + VA, i, j + 3) = (Mg + Cy) -β (G + Ye) (15) Gi, k = (VA, i, j + VA, i, j + 1 + VB, i, j + VB, i, j + 1) -pRi, k -qBi, k = (Mg + Ye + G + Cy) -pRi, k -qBi, k (16) Gi, k + 1 = (VB, i, j + 2 + VA, i, j + 3 + VA, i, j + 2 + VB, i, j + 3) -pRi, k + 1-qBi, k + 1 = (Mg + Ye + G + Cy) -pRi, k + 1-qBi, k + 1 (17) RGB signal It is obtained.

Similarly, Ri + 1, k = (VB, i + 1, j + VB, i + 1, j + 1)-. Alpha. (VA , i + 1, j + VA, i + 1, j + 1) = (Mg + Ye) -α (G + Cy) (18) Ri + 1, k + 1 = (VA, i + 1, j + 2 + VB, i + 1, j + 3) -α (VB, i + 1, j + 2 + VA, i + 1, j + 3) = (Mg + Ye) -α (G + Cy) (19) Bi + 1, k = (VB, i + 1, j + VA, i + 1, j + 1) -β (VA, i + 1, j + VB, i + 1, j + 1) = (Mg + Cy) -β (G + Ye) (20) Bi + 1, k + 1 = (VA, i + 1, j + 2 + VA, i + 1, j + 3) -β (VB, i + 1, j + 2 + VB, i + 1, j + 3) = (Mg + Cy) -Β (G + Ye) (21) Gi + 1, k = (VB, i + 1, j + VB, i + 1, j + 1 + VA, i + 1, j + VA, i + 1, j + 1) -pRi + 1, k-qBi + 1, k = (Mg + Ye + G + Cy) -pRi + 1, k-qBi + 1, k (22) Gi + 1, k + 1 = (VA, i + 1, j + 2 + VB, i + 1, j + 3 + VB, i + 1, j + 2 + VA, i + 1, j + 3) -pRi + 1, k + 1 -qBi + 1, k + 1 = ( g + Ye + G + Cy) - pRi + 1, k + 1 - qBi + 1, k + RGB signals 1 (23) is obtained. Here, p and q may be 1 by referring to the expression (3), but it is preferable that they can be appropriately adjusted according to the value of the Y component.

The constants α, β, p and q are determined by adjusting the parameters of the software executed in the system control circuit 10.

For an even field, by combining the j + 1th pixel and the j + 2nd pixel, the above (1
RGB signals are obtained by the same equations as in equations (2) to (23). Note that the above equation is a linear calculation method without gamma correction, and if the gamma-corrected signals are stored in the image memories 41 to 44, after linear conversion, those calculations are performed. After the calculation, the normal gamma correction is performed.

FIG. 10 shows an example of the arithmetic circuit configuration for carrying out the extraction of the RGB signals described above, and this circuit is provided inside the video signal processing circuit 46. In this figure, the output signal from the first CCD 22 is the switch 5
1 is input to one of the image memories 41 and 42, and the output signal from the second CCD 23 is input to one of the image memories 43 and 44 via the switch 52. Switch 5
1, 52 are switched under the control of the system control circuit 10, and when the image signal of the first field is input, one terminal 51a, 52a side, and when the image signal of the second field is input, the other Terminals 51b, 5
2b side is connected respectively. That is, the image memory 4
The image signals of the first field are stored in 1 and 43, and the image signals of the second field are stored in the image memories 42 and 44.

The signals read from the image memories 41 to 44 are subjected to a predetermined calculation in any of the adders 61 to 65, the subtractors 66 to 69 and the level shift circuits 71 to 76, and the G signal and R signal are supplied. The signal and the B signal are sought.
The G signal is directly output from the G terminal 81, but the R signal and the B signal are output via the switches 53 and 54 to the R terminal 82.
Alternatively, it is output from the B terminal 83. The arithmetic operation by the arithmetic circuit of FIG. 10 is in accordance with the expressions (12) to (23), and the operation of this circuit will be described by taking the expression (12) as an example.

First, the switches 51 and 52 are switched to one of the terminals 51a and 52a, respectively, and the image memory 41 stores the magenta signal (VA, i,
j) is stored, and the green signal (VB, i, j) of the first field is stored in the image memory 43. Then, the switches 51 and 52 are switched to the other terminals 51b and 52b, the yellow signal (VA, i, j + 1) of the second field is stored in the image memory 42, and the second field is stored in the image memory 44. The cyan signal (VB, i, j + 1) of is stored.

The magenta signal (VA, i, of the first field)
j) and the yellow signal (VA, i, j + 1) of the second field is
It is added in the adder 61. The green signal (VB, i, j) of the first field and the cyan signal (VB, i, j + 1) of the second field are added by the adder 63.
The output signal of the adder 61 is multiplied by the coefficient 1 in the level shift circuit 74, and the output signal of the adder 63 is multiplied by the coefficient α in the level shift circuit 73. Subtractor 6
In 7, the output signal of the level shift circuit 73 is subtracted from the output signal of the level shift circuit 74, and (1
The expression 2) is executed. At this time, the switch 53 is switched to the R output terminal side, and the R signal is output from the R terminal 82.

The level shift amounts in the level shift circuits 71 to 76 are controlled by the system control circuit 10 according to the contents of the calculation. That is, the level shift circuits 71 to 74 are set to one of α, β and 1, and the level shift circuits 75 and 76 are set to one of p and q. The switches 53 and 54 are i
When calculating the th pixel, switch to the upper side of the figure
When the first pixel is calculated, it is switched to the lower side of the figure.

FIG. 11 shows another embodiment of the complementary color filter. This filter consists of cyan (Cy), yellow (Ye) and green (G) and has no magenta. In the case of this filter, the B signal, the R signal, and the G signal are obtained from the signals of the four pixels surrounded by the broken line in the figure by the following formulas. B = Cy ₁ − (G ₁ + G ₂ ) / 2 R = Ye ₂ − (G ₁ + G ₂ ) / 2 G = (G ₁ + G ₂ ) / 2 where the subscript 1 is the upper pixel signal in FIG. 11. , Subscript 2
Indicates the lower pixel signal.

The filter shown in FIG. 11 can also achieve the same effect as when the complementary color checkered color filter is used. However, since the arithmetic expression is particularly simple, the circuit structure can be further simplified.

As described above, according to the high resolution mode of the apparatus of this embodiment, the resolution of the color signal can be increased to the same level as that of the luminance signal. Therefore, such an image signal is input to the computer and image processing is performed pixel by pixel. Even if you do
There is no problem of noticeable deterioration of image quality. The present embodiment is a two-plate type filter 51 having the same structure,
Since 52 is used, the resolution of the color signal can be improved without increasing the scale of the device and with a simple and inexpensive circuit configuration.

Next, the high sensitivity mode of the apparatus of this embodiment will be described. This mode is set by switching the mode selector switch 49.

In the high resolution mode, the R signal, the B signal and the G signal are formed by the four pixels connected by the line L as shown in FIG.
Signals are required, but in the high sensitivity mode, these signals are determined by 8 pixels connected by the line LL as shown in FIG. The extraction method of the R signal, the B signal and the G signal in this high sensitivity mode is performed as follows.

Similar to the case of the high resolution mode, in the pixel arrangement of FIG. 9, the signal of the first field in the first scanning, that is, VA, i, j = Mg, VA, i + 1, j = G VA, i, j + 2 = G, VA, i + 1, j + 2 = Mg VB, i, j = G, VB, i + 1, j = Mg VB, i, j + 2 = Mg, VB, i + 1, j + 2 = G is extracted, and the signal of the second field in the second scanning, that is, VA, i, j + 1 = Ye, VA, i + 1, j + 1 = Cy VA, i, j + 3 = Ye, VA, i + 1, j + 3 = Cy VB, i, j + 1 = Cy, VB, i + 1, j + 1 = Ye VB, i, j + 3 = Cy, VB, i + 1, j + 3 = Ye is extracted.

The above all pixel signals are the image memories 41 to 44.
(FIG. 1). These pixel signals are read out to the video processing circuit 46, and Ri + 0.5, k = (VA, i, j + VA, i, j +) is calculated for the i-th pixel of the odd-even scanning line of the odd field by calculation. 1 + VB, i + 1, j + VB, i + 1, j + 1) -α (VB, i, j + VB, i, j + 1 + VA, i + 1, j + VA, i + 1, j + 1 ) = 2 (Mg + Ye) -2α (G + Cy) (24) Ri + 0.5, k + 1 = (VB, i, j + 2 + VA, i, j + 3 + VA, i + 1, j + 2 + VB, i + 1, j + 3) -α (VA, i, j + 2 + VB, i, j + 3 + VB, i + 1, j + 2 + VA, i + 1, j + 3) = 2 (Mg + Ye) -2α ( G + Cy) (25) Bi + 0.5, k = (VA, i, j + VB, i, j + 1 + VB, i + 1, j + VA, i + 1, j + 1) -β (VB, i, j + VA , i, j + 1 + VA, i + 1, j + VB, i + 1, j + 1) = 2 (Mg + Cy) -2β (G + Ye) (26) Bi + 0.5, k + 1 = (VB, i, j +2 + VB, i, j + 3 + VA, i + 1, j + 2 + VA, i + 1, j + 3) -β (VA, i, j + 2 + VA, i, j + 3 + VB, i + 1 , j + 2 + VB, i + 1, j + 3) = 2 (Mg + Cy) -2β (G + Ye) (27) Gi + 0.5, k = (VA, i, j + VA, i, j + 1 + VB, i, j + VB, i, j + 1 + VB, i + 1, j + VB, i + 1, j + 1 + VA, i + 1, j + VA, i + 1, j + 1) -pRi + 0.5, k-qBi + 0.5, k = 2 (Mg + Ye + G + Cy) -pRi + 0.5, k-qBi + 0.5, k (28) Gi + 0.5, k + 1 = (VB, i, j + 2 + VA, i, j + 3 + VA, i, j + 2 + VB, i, j + 3 + VA, i + 1, j + 2 + VB, i + 1, j + 3 + VB, i + 1, j + 2 + VA, i + 1, j + 3) -pRi + 0.5, k + 1 -qBi + 0.5, k + 1 = 2 (Mg + Ye + G + Cy) -pRi + 0.5, k + 1- An RGB signal of qBi + 0.5, k + 1 (29) is obtained.

Similarly, Ri + 1 + 0.5, k = (VB, i + 1, j + VB, i + 1, j + 1 + VA) for the i + 1 + 0.5th pixel of the odd-even scanning line in the odd field , i + 2, j + VA, i + 2, j + 1) -α (VA, i + 1, j + VA, i + 1, j + 1 VB, i + 2, j + VB, i + 2, j + 1) = 2 (Mg + Ye) -2α (G + Cy) (30) Ri + 1 + 0.5, k + 1 = (VA, i + 1, j + 2 + VB, i + 1, j + 3 + VB, i + 2, j + 2 + VA, i + 2, j + 3) -α (VB, i + 1, j + 2 + VA, i + 1, j + 3 + VA, i + 2, j + 2 + VB, i + 2, j +3) = 2 (Mg + Ye) -2α (G + Cy) (31) Bi + 1 + 0.5, k = (VB, i + 1, j + VA, i + 1, j + 1 + VA, i + 2, j + VB, i + 2, j + 1) -β (VA, i + 1, j + VB, i + 1, j + 1 + VB, i + 2, j + VA, i + 2, j + 1) = 2 (Mg + Cy)- 2β (G + Ye) (32) Bi + 1 + 0.5, k + 1 = (VA, i + 1, j + 2 + VA, i + 1, j + 3 + VB, i + 2, j + 2 + VB, i + 2 , j + 3) −β (VB, i + 1, j + 2 + VB, i + 1, j + 3 + VA, i + 2, j + 2 + VA, i + 2, j + 3) = 2 (Mg + Cy) -2β (G + Ye) (33) Gi + 1 + 0.5 , k = (VB, i + 1, j + VB, i + 1, j + 1 + VA, i + 1, j + VA, i + 1, j + 1 + VA, i + 2, j + VB, i + 2, j +1 + VB, i + 2, j + VB, i + 2, j + 1) -pRi + 1.5, k-qBi + 1.5, k = 2 (Mg + Ye + G + Cy) -pRi, k-qBi, k (34) Gi + 1 + 0.5, k + 1 = (VA, i + 1, j + 2 + VB, i + 1, j + 3 + VB, i + 1, j + 2 + VA, i + 1, j + 3 + VB, i + 2, j + 3 + VA, i + 2, j + 3 + VA, i + 2, j + 2 + VB, i + 2, j + 3) -pRi + 1.5, k + 1 -qBi + 1.5, k + 1 = 2 ( Mg + Ye + G + Cy) -pRi + 1.5, k + 1-qBi + 1.5, k + 1 (35) RGB signals are obtained. Similar to high resolution mode, p,
Although q may be 1 by referring to the expression (3), it is preferable that q can be appropriately adjusted according to the value of the Y component.

The constants α, β, p and q are determined by adjusting the parameters of software executed in the system control circuit 10.

For an even field, by combining the (j + 1) th pixel and the (j + 2) th pixel, the above (2
RGB signals are obtained by the same equations as 4) to (35). Similar to the high resolution mode, the above equation is a linear calculation method without gamma correction, and if the gamma-corrected signals are stored in the image memories 41 to 44, they are once converted into linear. After that, those calculations are performed, and after the calculations, regular gamma correction is performed.

FIG. 13 compares the pixels used in the high resolution mode and the high sensitivity mode. In this figure, when the horizontal direction is represented by the parameter i and the vertical direction is represented by the parameter j, in the high resolution mode, the result is as shown in FIG.
, Two pixels A _{i, j} , A of the first CCD
_{i, j + 1} and two pixels B _{i, j} , B _{i, j + 1 of} the second CCD
Based on and, the pixel signal at the position indicated by the symbol KA is obtained. This pixel signal KA is located in the same column as the i-th pixel. On the other hand, in the high sensitivity mode, FIG.
, The four pixels A _{i, j} , A of the first CCD
_{i, j + 1} , A _{i + 1, j} , A _{i + 1, j + 1} and four pixels B _{i, j} , B _{i, j + 1} , B _{i + 1, j of} the second CCD Based on B _{i + 1, j + 1} , the pixel signal at the position indicated by the reference KB is obtained. This pixel signal KB is located in the middle column between the i-th pixel and the (i + 1) -th pixel. That is, the horizontal position of each pixel signal in the high-sensitivity mode is deviated from the horizontal position of each pixel signal in the high-resolution mode by 0.5 pixel. Therefore, the subscripts indicating the horizontal direction in formulas (24) to (35) are given. (I + 0.5, i + 1 + 0.5 ...) is added by 0.5 to the subscripts (i, i + 1 ...) indicating the horizontal direction in the expressions (12) to (23).

Further, as clearly shown in FIG. 13, one pixel signal KA in the high resolution mode is obtained based on four pixels, and one pixel signal KB in the high sensitivity mode is obtained based on eight pixel signals. To be Therefore, the amount of light incident on the photodiode corresponding to the pixel signal KB in the high-sensitivity mode is twice the amount of light incident on the photodiode corresponding to the pixel signal KA in the high-resolution mode, without increasing the F value of the taking lens. An image with high sensitivity can be obtained.
In addition, the fact that the amount of incident light in the high-sensitivity mode is twice that in the high-resolution mode is counted in each of the last sides of the equations (24) to (35) as compared with the equations (12) to (23). It can be understood from the fact that 2 is applied.

[0066]

As described above, according to the present invention, it is possible to increase the color resolution with a simple circuit configuration, and it is possible to obtain the effect that the image quality does not deteriorate even when image processing is performed by a computer or the like. Further, according to the present invention, it is possible to obtain an effect that high-sensitivity imaging can be performed without increasing the F value of the taking lens.

[Brief description of drawings]

FIG. 1 is a block diagram showing a circuit configuration of a still video camera to which an embodiment of the present invention is applied.

FIG. 2 is a diagram showing an array of color filters provided on the light receiving surfaces of the first and second CCDs.

FIG. 3 is a diagram showing a state in which corresponding pixels are overlapped with each other regarding output signals of the first and second CCDs.

FIG. 4 is a diagram for explaining a method of extracting color signals in the first and second fields in a high resolution mode.

FIG. 5 is a diagram showing a spectral distribution of a luminance signal in the example.

FIG. 6 is a diagram showing a spectral distribution of color signals in the example.

FIG. 7 is a diagram showing a spectral distribution of color signals in a comparative example.

FIG. 8 is a diagram showing a reproducible spectrum range in a comparative example and an example.

FIG. 9 is a diagram for explaining a color signal extraction method according to the embodiment.

FIG. 10 is a circuit diagram showing a configuration example of a video signal processing circuit.

FIG. 11 is a diagram showing another example of complementary color filters.

FIG. 12 is a diagram for explaining a method of extracting color signals in the first and second fields in a high sensitivity mode.

FIG. 13 is a diagram comparing pixels used to obtain pixel signals in a high resolution mode and a high sensitivity mode.

FIG. 14 is a diagram showing a color filter provided in a conventional two-plate type imaging device.

[Explanation of symbols]

21, 22 CCD 51,52 Filter

─────────────────────────────────────────────────── ─── Continuation of front page (56) References JP-A-5-244610 (JP, A) JP-A-6-86301 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) H04N 9/04-9/11

Claims (7)

(57) [Claims] 1. A first image sensor and a second image sensor, which have regularly-changing complementary color difference line-sequential pixel spectral characteristics and in which the pixel spectral characteristics are offset from each other by one pixel in the horizontal direction, and in the vertical direction. Based on signals of a total of 4 pixels obtained from the first and second image sensors for 2 pixels arranged in
First color signal extraction means for extracting the R signal, G signal and B signal of the pixel, and a total of 8 pixels obtained from the first and second image sensors for 4 pixels in which 2 pixels are arranged in each of the horizontal direction and the vertical direction Second color signal extracting means for extracting the R signal, G signal and B signal of one pixel, and control means for selecting and operating one of the first and second color signal extracting means based on the signal of the pixel. And a two-plate type image pickup device. 2. The two-plate type image pickup device according to claim 1, wherein the first and second image sensors have the same configuration. 3. The two-plate type image pickup device according to claim 1, wherein the first and second image sensors have complementary color filters on the light receiving surface. 4. The two-plate type image pickup device according to claim 3, wherein the color filter is a complementary color checkered color filter. 5. The first color signal extraction means is one pixel of R
The four pixels used to extract the signal, the G signal, and the B signal are magenta, cyan, yellow, and green.
Two different color filter elements, and the second color signal extraction means has one pixel of the R signal, the G signal and the B signal.
The color filter is configured such that eight pixels used for extracting a signal have two pixels each having four different color filter elements of magenta, cyan, yellow, and green. Item 2. A two-panel image pickup device. 6. A first image sensor and a second image sensor, which have regularly-changing complementary color difference line-sequential pixel spectral characteristics and in which the pixel spectral characteristics are horizontally offset from each other by one pixel. An R signal of one pixel, a G signal of one pixel based on a signal obtained from a total of four pixels of two pixels arranged in the vertical direction at positions where the first and second image sensors are optically equal to each other.
First color signal generating means for generating a signal and a B signal, and the first and second image sensors are arranged in a horizontal direction and a vertical direction 4 at positions which are optically equal to each other.
A first color signal generation means for generating an R signal, a G signal, and a B signal of one pixel based on signals obtained from a total of eight pixels for each pixel; and one of the first and second color signal generation means. A two-plate type image pickup device, comprising: a control unit that selects and operates. 7. A first image sensor and a second image sensor, which have regularly-changing complementary color difference line-sequential pixel spectral characteristics, and in which the pixel spectral characteristics are horizontally offset from each other by one pixel. An R signal of one pixel, a G signal of one pixel based on a signal obtained from a total of four pixels of two pixels arranged in the vertical direction at positions where the first and second image sensors are optically equal to each other.
A first color signal generating unit that generates a signal and a B signal, and the first and second image sensors that are respectively adjacent to the two pixels used in the first color signal generating unit are arranged in the vertical direction. An R signal, a G signal, and a B signal of one pixel are obtained based on a signal obtained from a total of 8 pixels including a total of 4 pixels of 2 pixels and the 4 pixels used in the first color signal generation unit. A two-plate type image pickup device comprising: a second color signal generating means for generating; and a control means for selecting and operating one of the first and second color signal generating means.