JP2021182763A - Image pick-up device and imaging apparatus - Google Patents

Abstract

To provide an image pick-up device that can shorten as much as possible the time required for the image pick-up device to output a signal necessary for the drive control of an imaging apparatus.SOLUTION: An image pick-up device comprises: a pixel unit in which a plurality of pixels are arranged which photoelectrically convert light from a subject; a reading unit that reads signals from the pixel unit; and an output unit that outputs, of the signals read by the reading unit, signals from the pixels in the entire area of the pixel unit to the outside of the image pick-up device as signals for creating an image, and outputs, to the outside of the image pick-up device, signals from the pixels in a subject area that is an area in the pixel unit where a subject is present as signals for calculating an evaluation value used for the drive control of an apparatus including the image pick-up device.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image pickup device and an image pickup apparatus.

In recent years, image pickup devices using CMOS image pickup devices and the like have become more sophisticated and multifunctional in order to meet various needs. As the number of pixels and high-speed imaging of CMOS image sensors are increasing, there is an increasing demand for a method capable of reading pixel signals at higher speed.

For example, as a method for performing high-speed reading, as described in Patent Document 1, a method in which an analog / digital conversion circuit (hereinafter referred to as a column ADC) is arranged for each column and digital output is performed has become widespread in recent years. .. By introducing the row ADC, it becomes possible to digitally transmit the pixel signal to the outside of the image pickup device, and with the improvement of the digital signal transmission technology, high-speed reading becomes possible.

On the other hand, as an example of multi-functionalization, for example, an image pickup device capable of acquiring not only light intensity distribution but also light incident direction and distance information has been proposed. Patent Document 2 discloses an image pickup device capable of focus detection using a signal obtained from the image pickup device. By dividing the photodiode (hereinafter referred to as PD) corresponding to one microlens into two, each PD is configured to receive light from different pupil surfaces of the photographing lens. Then, focus detection is performed by comparing the outputs of the two PDs. Further, a normal captured image can be obtained by adding the output signals from the two PDs constituting the unit pixel.

Further, the image pickup apparatus described in Patent Document 3 includes a mode in which an image pickup signal for live view display, a focus detection signal, and an exposure control signal are read out from a solid-state image sensor by one vertical scan of the same frame. ing. The image pickup apparatus described in Patent Document 3 is said to be capable of displaying a live view after speeding up both focus detection control and exposure control.

Japanese Unexamined Patent Publication No. 2005-278135 Japanese Patent No. 3774597 Japanese Unexamined Patent Publication No. 2009-89105

However, when performing focus detection and exposure control in an image pickup device as disclosed in Patent Documents 2 and 3, it is necessary to read out all PD signals, so that the time required to read out PD signals is required. There is a problem that the frame rate becomes long and the frame rate decreases. Even if the signal read time is increased by the read method using the column ADC as in Patent Document 1, it is expected that the number of pixels and the frame rate will be further increased in the future, and the signal read time is desired to be further shortened. Is done.

The present invention has been made in view of the above-mentioned problems, and an object thereof is that the time required to output a signal necessary for drive control of an image pickup device such as a focus detection signal from an image pickup element can be significantly shortened. It is to provide an image pickup element.

The image pickup device according to the present invention includes a pixel unit in which a plurality of pixels for photoelectric conversion of light from a subject are arranged, a reading unit that reads a signal from the pixel unit, and a signal read by the reading unit. As a signal for generating an image, a signal of pixels in the entire region of the pixel portion is output to the outside of the image pickup element, and the pixel is used as a signal for calculating an evaluation value used for drive control of a device provided with the image pickup element. It is characterized by including an output unit that outputs a signal of a pixel in a subject area, which is a region in which the subject exists, to the outside of the image pickup device.

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide an image pickup device that can significantly reduce the time required to output a signal required for drive control of an image pickup device, such as a focus detection signal, from the image pickup device.

The block diagram which shows the structure of the image pickup apparatus which concerns on 1st Embodiment of this invention. The figure explaining the structure of the unit pixel of the image pickup element of 1st Embodiment. A conceptual diagram in which a luminous flux emitted from an exit pupil of a photographing lens is incident on a unit pixel. The block diagram which shows the structure of the image pickup element of 1st Embodiment. The figure explaining the pixel circuit and the readout circuit of the image pickup element of 1st Embodiment. The timing chart which shows the signal reading operation of the image pickup element of 1st Embodiment. The figure explaining the structure of the signal processing part in the image pickup element of 1st Embodiment. The figure which showed the signal output area for focus detection in the pixel area of 2nd Embodiment. The block diagram which shows the structure of the image pickup apparatus which concerns on 3rd Embodiment of this invention. The flowchart which shows the flow of the processing of the imaging surface phase difference AF in 3rd Embodiment. The figure which shows the selection screen of the focal point detection area in 3rd Embodiment. The figure which shows the subject detection result screen in 3rd Embodiment. The figure explaining the structure of the image pickup element of 4th Embodiment. The figure explaining the pixel circuit and the readout circuit of the image pickup element of 4th Embodiment. The overall block diagram of the image pickup element of 5th Embodiment. The overall block diagram of the image pickup element of 5th Embodiment. The overall block diagram of the image pickup element of the 6th Embodiment. The block diagram which shows the structure of the image pickup apparatus which concerns on 7th Embodiment. The block diagram which shows the structure of the image pickup apparatus which concerns on 8th Embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

(First Embodiment)
FIG. 1 is a block diagram showing a configuration of an image pickup device having an image pickup device according to the first embodiment of the present invention. The image pickup device 100 includes a light receiving unit 102, a readout unit 103, a control unit 104, a signal processing unit 105, and an output unit 106. The light receiving unit 102 has a plurality of unit pixels arranged in a matrix and receives an optical image formed by the photographing lens 101. The configuration of the light receiving unit 102 will be described later. The reading unit (AD conversion unit) 103 receives the drive control signal of the control unit 104, performs A / D conversion of the image signal output from the light receiving unit 102, and sends it to the signal processing unit 105.

The signal processing unit 105 performs arithmetic processing such as signal addition, subtraction, and multiplication on the A / D converted image signal, and selection processing of a signal to be output from the image sensor 100 to the outside via the output unit 106. I do. Further, the signal processing unit 105 also performs various corrections such as reference level adjustment and rearrangement of data. These processes are executed by receiving a control signal from the control unit 104. The details of the signal processing unit 105 will be described later, but the signal processing unit 105 performs signal processing for captured images and signal processing for focus detection on the image signal obtained from the light receiving unit 102, and outputs them. Send to unit 106. The output unit 106 outputs the image signal processed by the signal processing unit 105 to the outside of the image pickup device 100.

The image processing unit 107 receives a signal for an captured image from the output unit 106 of the image pickup element 100, and performs image processing such as defect pixel correction, noise reduction, color conversion, white balance correction, and image correction, resolution conversion processing, and an image. Performs compression processing and generates still images and moving images. The phase difference detection unit 108 receives a focus detection signal from the output unit 106 and calculates a phase difference evaluation value for performing focus detection.

The overall control / calculation unit 109 comprehensively drives and controls the image sensor 100 and the entire image sensor. The display unit 110 displays an image after shooting, a live view image, various setting screens, and the like. The recording unit 111 and the memory unit 112 are recording media such as a non-volatile memory or a memory card that record and hold an image signal or the like output from the overall control / calculation unit 109. The operation unit 113 receives a user's command by an operation member provided in the image pickup apparatus, and inputs the command to the overall control / calculation circuit 106. The lens control unit 114 calculates optical system drive information based on the phase difference evaluation value calculated by the phase difference detection unit 108, and controls the focus lens position of the photographing lens 101.

Next, the relationship between the photographing lens 101 and the light receiving unit 102 of the image pickup element 100 in the image pickup apparatus of the present embodiment, the definition of pixels, and the principle of focus detection by the pupil division method will be described.

FIG. 2 is a schematic diagram showing the configuration of the unit pixel 200 of the image pickup device 100. In FIG. 2, the microlens 202 further collects the light imaged on the image pickup device 100 by the photographing lens 101 for each pixel. The photoelectric conversion units 201A and 201B composed of photodiodes (PD) receive light incident on the unit pixel 200, and generate and store signal charges according to the amount of the light received. Since the unit pixel 200 has two photoelectric conversion units under one microlens 202, the two photoelectric conversion units 201A and 201B can each receive light in the exit pupil region divided into two. .. The signal obtained by mixing the two signals of the photoelectric conversion units 201A and 201B for each pixel is the output signal of one pixel for image generation. Further, the focus of the photographing lens 101 can be detected by comparing the signals obtained from the two photoelectric conversion units for each pixel. That is, by performing a correlation calculation between the signal obtained from the photoelectric conversion unit 201A and the signal obtained from the photoelectric conversion unit 201B in a certain region in the unit pixel 200, the focus detection of the phase difference detection method in which the pupil is divided in the left-right direction is performed. Is possible.

FIG. 3 shows a state in which light that has passed through the photographing lens 101 passes through one microlens 202 and is received by the unit pixel 200 of the light receiving unit 102 of the image sensor 100 in a direction perpendicular to the optical axis (Z axis). It is a figure observed from (Y-axis direction). The light that has passed through the exit pupils 302 and 303 of the photographing lens is incident on the unit pixel 200 about the optical axis. At that time, the amount of incident light is adjusted by the lens diaphragm 301. As shown in FIG. 3, the luminous flux passing through the pupil region 302 passes through the microlens 202 and is received by the photoelectric conversion unit 201A, and the luminous flux passing through the pupil region 303 is received by the photoelectric conversion unit 201B through the microlens 202. Therefore, the photoelectric conversion units 201A and 201B receive light in different regions of the exit pupil of the photographing lens, respectively.

The signal of the photoelectric conversion unit 201A that divides the light from the photographing lens 101 into pupils is acquired from a plurality of unit pixels 200 arranged in the X-axis direction, and the subject image composed of these output signal groups is defined as the A image. Similarly, the signal of the photoelectric conversion unit 201B that divides the pupil is acquired from a plurality of unit pixels 200 arranged in the X-axis direction, and the subject image composed of these output signal groups is defined as the B image.

A correlation calculation is performed on the A image and the B image, and the amount of image shift (pupil division phase difference) is detected. Further, by multiplying the amount of image shift by the focal position of the photographing lens 101 and a conversion coefficient determined by the optical system, the focal position corresponding to an arbitrary subject position in the screen can be calculated. By controlling the focus position of the photographing lens 101 based on the focus position information calculated here, phase difference AF (autofocus) on the imaging surface becomes possible. Further, by setting the signal obtained by adding the A image signal and the B image signal as the A + B image signal, this A + B image signal can be used for a normal captured image.

Next, the configuration of the light receiving unit 102 and the reading unit 103 of the image pickup device 100 will be described with reference to FIGS. 4 and 5. FIG. 4 is a block diagram showing a configuration example of the light receiving unit 102 and the reading unit 103 of the image pickup device 100.

The light receiving unit 102 has a pixel unit 401 and a drive circuit unit 402. A plurality of unit pixels 200 are arranged in a matrix in the horizontal direction (row direction) and the vertical direction (column direction) in the pixel unit 401. In FIG. 4, only a total of 6 unit pixels 200 in 2 rows and 3 columns are shown, but in reality, millions and tens of millions of unit pixels 200 are arranged. The drive circuit unit 402 includes a power supply circuit for driving the pixel unit 401, a timing generator (TG), a scanning circuit, and the like. By driving the pixel unit 401 by the drive circuit unit 402, the pixel signal in the entire imaging region of the pixel unit 401 is output from the pixel unit 401 to the readout unit 103. The drive circuit unit 402 is driven under the control of the control unit 104 in FIG. Analog-to-digital conversion (A / D conversion) is performed on the pixel signal from the pixel unit 401 input to the reading unit 103. The reading unit 103 includes a plurality of reading circuits, for example, one reading circuit in one row.

By the way, regarding the driving method of the pixel unit 401, if rows adjacent to each other are driven under different conditions (frame rate, accumulation time, etc.), noise such as crosstalk and blooming is likely to occur. However, in the present embodiment, since the drive circuit unit 402 drives the pixel unit 401 uniformly over the entire region under the same conditions, such a problem does not occur.

FIG. 5 is a diagram showing an example of a readout circuit 509 constituting the unit pixel 200 of the image pickup device 100 and the readout unit 103. In the unit pixel 200, the transfer switch 502A is connected to the photoelectric conversion unit 201A composed of the photodiode (PD), and the transfer switch 502B is connected to the photoelectric conversion unit 201B. The charges generated by the photoelectric conversion units 201A and 201B are transferred to the common floating diffusion unit (FD) 504 via the transfer switches 502A and 502B, respectively, and are temporarily stored. When the selection switch 506 is turned on, the electric charge transferred to the FD 504 is output to the column output line 507 as a voltage corresponding to the electric charge via the amplification MOS transistor (SF) 505 forming the source follower amplifier. A current source 508 is connected to the column output line 507.

The reset switch 503 resets the potential of the FD504 and the potentials of the photoelectric conversion units 201A and 201B to VDD via the transfer switches 502A and 502B. The transfer switches 502A and 502B, the reset switch 503, and the selection switch 506 are controlled by the control signals PTXA, PTXB, PRES, and PSEL via the signal lines connected to the peripheral drive circuit units 402, respectively.

Next, the circuit configuration of the readout circuit 509 will be described. The amplifier 510 amplifies the signal output to the column output line 507, and the capacitance 512 is used to hold the signal voltage. Writing to the capacitance 512 is controlled by a switch 511 that is turned on and off by the control signal PSH. The reference voltage Vslope supplied from the slope voltage generation circuit (not shown) is input to one input of the comparator 513, and the output of the amplifier 510 written in the capacitance 512 is input to the other input. The comparator 513 compares the output of the amplifier 510 with the reference voltage Vslope, and outputs either a low level or a high level depending on the magnitude relationship. Specifically, when the reference voltage Vslope is smaller than the output of the amplifier 510, a low level is output, and when it is large, a high level is output. The clock CLK starts to move at the same time as the transition of the reference voltage Vslope starts, and the counter 514 counts up corresponding to the clock CLK when the output of the comparator 513 is at a high level, and counts at the same time as the output of the comparator 513 reverses to a low level. Stop the signal. The count value at this time is held in either the memory 516 or the memory 517 as a digital signal.

The memory 516 holds a digital signal obtained by AD-converting the reset level signal of the FD 504 (hereinafter, “N signal”), and the memory 517 stores the signals of the photoelectric conversion unit 201A and the photoelectric conversion unit 201B as the N signal of the FD 504. A digital signal obtained by AD-converting a signal superimposed on the signal (hereinafter referred to as “S signal”) is held. Which of the memories 516 and 517 the count value of the counter 514 is written to is sorted by the switch 515. For the signals held in the memories 516 and 517, the difference obtained by subtracting the N signal from the S signal is calculated by the CDS circuit 518. Then, under the control of the drive circuit unit 402, the signal is output to the signal processing unit 105 via the digital signal output line 519.

One read circuit 509 is arranged for each column of pixels, and the pixel signal is read in row units. In this case, the selection switch 506 is controlled in row units, and the pixel signals of the selected row are collectively output to the column output line 507. The larger the number of the read circuits 509, the faster the pixel signal of the pixel unit 401 can be read out to the signal processing unit 105.

FIG. 6 is a timing chart showing an example of a charge reading operation from the unit pixel 200 of the image pickup device 100 having the circuit configuration shown in FIG. The timing of each drive pulse, the reference voltage Vslope, the clock CLK, and the horizontal scanning signal are schematically shown. In addition, the potential Vl of the column output line at each timing is also shown.

Prior to reading the signal from the photoelectric conversion unit 201A, the signal line PRESS of the reset switch 503 becomes Hi (t600). As a result, the gate of the SF (source follower amplifier) 505 is reset to the reset power supply voltage. At time t601, the control signal PSEL is set to Hi, and SF505 is set to the operating state. Then, the reset of the FD504 is released by setting the control signal PRESS to Lo at t602. The potential of the FD 504 at this time is output to the column output line 507 as a reset signal level (N signal) and input to the read circuit 509.

By turning the switch 511 on and off with the control signals PSH as Hi and Lo at time t603 and t604, the N signal output to the column output line 507 is amplified by the amplifier 510 with a desired gain and then held in the capacitance 512. NS. The potential of the N signal held in the capacitance 512 is input to one of the comparators 513. After the switch 511 is turned off at time t604, the reference voltage Vslope is decreased from the initial value with time by a slope voltage generation circuit (not shown) from time t605 to t607. The clock CLK is supplied to the counter 514 at the start of the transition of the reference voltage Vslope. The value of the counter 514 increases according to the number of CLKs. Then, when the reference voltage Vslope input to the comparator 513 becomes the same level as the N signal, the output COMP of the comparator 513 becomes low level, and at the same time, the operation of the counter 514 also stops (time t606). The value when the operation of the counter 514 is stopped becomes the value obtained by AD-converting the N signal, the counter 514 and the memory 516 are connected by the switch 515, and the digital value of the N signal is held in the N signal memory 516. Will be done.

Next, at the time t607 and t608 after the digitized N signal is held in the N signal memory 516, the control signal PTXA is sequentially set as Hi and Lo, and the optical charge accumulated in the photoelectric conversion unit 201A is transferred to the FD504. Then, the potential fluctuation of the FD 504 according to the amount of electric charge is output to the column output line 507 as a signal level (light component + reset noise component (N signal)) and input to the readout circuit 509. The input signal (S (A) + N) is amplified by the amplifier 510 with a desired gain, and then the control signal PSH is sequentially set to Hi and Lo at time t609 and t610, and the switch 511 is turned on and off at the timing of turning on and off the capacity 512. Be retained. The potential held in the capacitance 512 is input to one of the comparators 513. After the switch 511 is turned off at time t610, the reference voltage Vslope is decreased from the initial value with time by the slope voltage generation circuit from time t611 to t613. With the start of the transition of the reference voltage Vslope, CLK is supplied to the counter 514. The value of the counter 514 increases according to the number of CLKs. Then, when the reference voltage Vslope input to the comparator 513 becomes the same level as the S signal, the output COMP of the comparator 513 becomes low level, and at the same time, the operation of the counter 514 also stops (time t612). The value when the operation of the counter 514 is stopped is the value obtained by AD-converting the S (A) + N signal. Then, the counter 514 and the memory 517 are connected by the switch 515, and the digital value of the S (A) + N signal is held in the S signal memory 517. The differential signal level (optical component) is calculated by the CDS circuit 518 from the signals held in the memory 516 and the memory 517, and the S (A) signal from which the reset noise component is removed is obtained. The S (A) signal is sequentially sent to the signal processing unit 105 under the control of the control unit 104.

The above is the operation of reading the signal from the photoelectric conversion unit 201A of the unit pixel 200. Similarly, when reading the signal from the other photoelectric conversion unit 201B of the unit pixel 200, the signal may be driven according to the timing chart of FIG. However, in this case, at times t607 and t608, the control signals PTXB are sequentially set to Hi and Lo instead of the control signals PTXA. That is, the pixel signal S (A) is output with the control signal PTXA as Hi and Lo for the first drive from the time t600 in FIG. 6, and then the pixel signal S with the control signal PTXB as Hi and Lo for the second time. By outputting (B), the output of the pixel signal for one line is completed. By repeating this for all lines, the output of the pixel signals S (A) and S (B) of all the pixels is completed.

FIG. 7 is a diagram showing a configuration example of the signal processing unit 105 and the output unit 106 of the image pickup device 100. The signal processing unit 105 has a signal processing unit 701 for captured images and a signal processing unit 702 for focus detection. The pixel signal output from the reading unit 103 is input to the signal processing unit 105 via the digital signal output line 519. The input signal is processed according to the control from the control unit 104. The captured image signal processing unit 701 and the focus detection signal processing unit 702 are each provided with a memory (not shown).

The captured image signal processing unit 701 calculates the captured image signal from the signal output from the readout unit 103. That is, the pixel signals S (A) and S (B) of the photoelectric conversion unit 201A of the unit pixel 200 and the photoelectric conversion unit 201B are received and mixed, and the S (A + B) signal is calculated. Then, the pixel signal S (A + B) is sent to the output unit 106 via the captured image signal output line 703. In the captured image signal processing unit 701, the pixel signals S (A) and S (B) of the two photoelectric conversion units are mixed and processed and output from the output unit 106 to the outside of the image pickup element 100, whereby the image pickup element 100 The amount of signal transmission to the outside can be reduced. The calculation of the pixel signal S (A) and the pixel signal S (B) becomes possible when both the signals of the unit pixels are prepared. The previously read pixel signal S (A) is stored in the memory, and when the pixel signal S (B) is read and input to the captured image signal processing unit 701, S (A) + S are sequentially obtained. The calculation of (B) is performed, and the output is output from the output unit 106.

The captured image signal processing unit 701 may further mix the signals of the unit pixels 200 or perform averaging processing. For example, in a pixel portion having a general configuration in which a color filter having a Bayer array of red (R), green (G), and blue (B) is provided, signals of adjacent pixels of the same color are mixed and averaged and output. If it is sent to the unit 106, the signal transmission amount can be further reduced. Further, instead of outputting all the signals of the pixel unit to the output unit 106, the signal of only the necessary region may be output. These processes are controlled by the control unit 104.

Subsequently, the processing of the focus detection signal processing unit 702 will be described. The focus detection signal processing unit 702 calculates and outputs a focus detection signal from the signal output from the read unit 103. As described above, the pixel signal S (A) and the pixel signal S (B) are required in order to detect the phase difference. However, if the pixel signals S (A) and the pixel signals S (B) of all the pixels of the pixel unit 401 are output from the output unit 106 to the outside of the image pickup element 100, the amount of signal transmission becomes enormous, which hinders high-speed reading. Become.

Therefore, the focus detection signal processing unit 702 performs arithmetic processing, reduces the signal amount, and outputs the signal from the output unit 106. For example, the pixel signals S (A) and S (B) are added to the bayer to calculate the luminance value Y, and the luminance signals Y (A) and Y (B) are output. The calculation for focus detection may be performed after converting the signal into a Y value, and by converting the signal into a Y value before outputting from the image sensor 100, the signal transmission amount can be reduced to 1/4. .. Since a Bayer-based signal is required to calculate the Y value, the pixel signal input to the focus detection signal processing unit 702 is held in the memory until the signals required for the calculation are prepared. That is, since the signal of the RG line is output after the signal of the RG line is output, the signal of the RG line is output, so that the pixel signal S (A) and the pixel signal S (B) of the RG line are held in the memory. When the signals of the GB and G lines are output, the luminance signals Y (A) and Y (B) are sequentially calculated and output from the output unit 106 via the signal line 704.

Further, the focus detection signal processing unit 702 may perform a correlation calculation, and the calculated value may be output from the signal line 704 to the output unit 106. The phase difference detection using the correlation calculation can be carried out by a known method. If only the correlation calculation value is output, the amount of output signal can be significantly reduced, although it depends on the number of region divisions at the time of the calculation.

As described above, in the image sensor 100 including the image sensor signal processing unit 701 and the focus detection signal processing unit 702, signal processing is performed to output only necessary signals to the outside of the image sensor 100. As a result, the amount of signal transmission can be reduced, and both the captured image data and the focus detection information can be obtained at high speed.

In this embodiment, the captured image signal processing unit 701 and the focus detection signal processing unit 702 are each provided with a memory. However, even in a configuration in which a memory is provided in front of these stages and signals are sent to the captured image signal processing unit 701 and the focus detection signal processing unit 702 when the signals required for calculation in each processing unit are prepared. good.

Further, in the description of the present embodiment, the luminance signals Y (A) and Y (B) are output as the focus detection signals, but only the luminance signal Y (A) may be output from the output unit 106. .. Specifically, the captured image signal processing unit 701 outputs a captured image signal, that is, a pixel signal S (A + B). Therefore, after being output to the outside of the image pickup device 100, the luminance signal Y (A + B) is calculated from the pixel signal S (A + B) by the phase difference detection unit 108 or the like, and the luminance signal Y (A) is subtracted to obtain the luminance. A focus detection signal may be obtained by calculating the signal Y (B). As described above, by outputting only the luminance signal Y (A) from the focus detection signal processing unit 702, the signal transmission amount can be further reduced.

For example, when the number of pixels is 20 million pixels, it is necessary to output the pixel signals S (A) and S (B) for all the pixels, that is, 40 million pieces of data when the signal processing unit is not provided. On the other hand, when the Y value is calculated and output as a focus detection signal by the image sensor provided with the signal processing unit of the present embodiment, 20 million pieces of data for the captured image and 20 million/4 = 5 million for focus detection. It can be seen that the amount of signal transmission is reduced because individual data are output. As a result, high-speed reading becomes possible. Further, when the focus detection signal is a correlation calculation value, it is clear that the signal transmission amount is further reduced.

Further, in the timing chart of FIG. 6, it is also possible to simultaneously control the control signals PTXA and PTXB at times t607 and t608 to obtain a signal of the unit pixel 200 in which the charges of the photoelectric conversion unit 201A and the photoelectric conversion unit 201B are mixed. be. Specifically, after reading the signal of the photoelectric conversion unit 201A according to the timing chart of FIG. 6, if the control signals PTXA and PTXB are controlled to be Hi and Lo at the same time and the signal is read, the pixel signal S (A + B) is read. ) Can be obtained. In this case, since the reset signal is read out once, higher speed reading is possible.

When the pixel signals S (A) and S (A + B) are read from the pixels, the focus detection signal processing unit 702 performs a process of subtracting the pixel signals S (A) from the pixel signals S (A + B). For example, the pixel signal S (B) can be obtained. Alternatively, the focus detection processing unit 702 may process and output only the pixel signal S (A), and the phase difference detection unit 108 may calculate the pixel signal S (B) or the luminance signal Y (B).

(Second embodiment)
Next, a second embodiment of the present invention will be described. In the first embodiment, the focus detection signal is output in the entire area of the pixel portion, but if the focus detection signal is selected and output only in the necessary area, the speed can be further increased. can do.

FIG. 8 is a diagram showing an example of a focus detection signal output region in the pixel region. The area shown by the diagonal line outputs the focus detection signal and the captured image signal, and the other area outputs only the captured image signal. For example, as in the example shown in FIG. 8A, the focus detection signal of only the target region in a wide range of pixels is discretely (selectively) output. This makes it possible to obtain focus detection information for the entire pixel region, while suppressing the amount of signal output to the outside of the image sensor 100. Further, in the case of the example shown in FIG. 8B, detailed focus detection information can be obtained for a part of the region, and the amount of signal output to the outside of the image sensor 100 can be suppressed. .. The selection of these output areas is controlled by the control unit 104. In the focus detection signal processing unit 702, only the signal in the output target area is read from the memory and calculated.

The output of the focus detection signal in a part of the region as shown in FIG. 8 may be a Y value signal or a correlation calculation result, but may be a pixel signal S (A). Although the amount of signal to be transmitted is larger than that of the Y value signal and the result of the correlation calculation, since only the necessary area is output, the suppression of the amount of signal transmission is achieved. It can also be realized with a relatively small-scale signal processing circuit.

The focus detection signal processing unit 702 may also mix and average the focus detection signals. In this case, the pixel signals S (A) and the pixel signals S (B) are mixed and averaged.

As described above, in the image sensor 100 including the image sensor signal processing unit 701 and the focus detection signal processing unit 702, signal processing is performed to output only necessary signals to the outside of the image sensor 100. As a result, the transmission amount of the focus detection signal can be reduced, and both the captured image data and the focus detection information can be obtained at high speed and efficiently.

By limiting the pixels used for focus detection in this way, there is a method of shortening the read time of one frame. Normally, only the row used for the focus detection process outputs the signals of the two photoelectric conversion units in the unit pixel, and the row not used for the focus detection process mixes the signals of the two photoelectric conversion units for image generation. By outputting only the signal, the increase in read time is suppressed. In this case, the individual output signals of the two photoelectric conversion units output for focus detection can be mixed and used as pixel signals for captured images. However, the method of reading the signal and the method of mixing the output signals of the two photoelectric conversion units differ between the line used for the focus detection process and the line not used, resulting in a difference in noise level and the like, resulting in an image pickup. The problem of image deterioration arises. However, by providing the focus detection signal processing unit as in the present embodiment, all the signals from the pixel unit can be read out at the same read timing, and the pixels to be output by the focus detection signal processing unit 702 can be selected. Therefore, the noise amount of the pixel signal S (A + B) used for the captured image does not differ depending on the region, and a high-quality captured image can be obtained.

(Third embodiment)
Next, a third embodiment of the present invention will be described. In the second embodiment described above, an example has been described in which the focus detection signal processing unit 702 selects and outputs only the signal in the required output target region from the focus detection signals. In the present embodiment, this will be further advanced, and an example of setting a necessary area of the focus detection signal based on the user's input and the area of the subject detected by the subject detection unit will be described.

As shown in FIG. 9, the configuration of the image pickup apparatus of the present embodiment is the configuration of the image pickup apparatus of the first and second embodiments shown in FIG. 1 with the subject detection unit 105a added. Since the other configurations are the same as the configurations of FIG. 1, the description of the same parts will be omitted, and only the different parts will be described.

In FIG. 9, the subject detection unit 105a receives a digital signal output for image generation from the readout unit 103, detects a subject using a known pattern recognition processing circuit, and performs focus detection processing. To decide. Examples of the subject to be detected here include the face and eyes of a person or an animal. Further, the subject detection unit 105a may be provided with a memory for temporarily storing a signal for image generation in the subject detection unit 105a in order to perform the subject detection process.

The signal processing unit 105 includes a captured image signal processing unit 701 and a focus detection signal processing unit 702, as already described with reference to FIG. 7 in the first embodiment. The operation of the captured image signal processing unit 701 is the same as that of the first embodiment.

On the other hand, the focus detection signal processing unit 702 selects a necessary region from the focus detection digital signal output and outputs the output to the output unit 106, as in the second embodiment. However, when the focus detection area is set manually, the focus detection signal processing unit 702 selectively outputs the focus detection signal of the focus detection area arbitrarily selected by the user. Alternatively, when the focus detection area is set automatically, the focus detection signal processing unit 702 receives the subject detection result of the subject detection unit 105a and selectively outputs the focus detection signal in the area where the subject is detected. do. The output unit 106 outputs the image generation digital signal received from the captured image signal processing unit 701 and the focus detection digital signal received from the focus detection signal processing unit 702 to the outside of the image pickup element 100.

The phase difference detection unit 108 receives a digital signal for focus detection from the output unit 106, and calculates a phase difference evaluation value for performing focus detection by the phase difference detection method. In the present embodiment, the focus detection signal input to the phase difference detection unit 108 is a signal output by the focus detection signal processing unit 702 in the signal processing unit 105 inside the image sensor 100 by selecting a region. be. Therefore, since the focus detection signal transmitted by the output unit 106 of the image pickup device 100 to the phase difference detection unit 108 is only the signal necessary for the focus detection control, the transmission band is efficiently used. Further, also in the internal processing of the phase difference detection unit 108, there is no need for arithmetic processing for calculating the phase difference evaluation value in a region unnecessary for focus detection control and processing for extracting a signal finally required for focus detection control. Therefore, the processing speed at which the phase difference detection unit 108 calculates the phase difference evaluation value can be increased. Further, the scale of the processing circuit of the phase difference detection unit 108 can be reduced.

The display unit 110 in FIG. 9 is used not only to display the image signal received from the overall control / calculation unit 109, but also to display the focus detection area that can be arbitrarily selected by the user of the image pickup apparatus. NS. Alternatively, it is also used to display a subject area, which is an area in which the subject detected by the subject detection unit 105a exists. Further, although the operation unit 113 is used for various inputs, it is also used for the user of the image pickup apparatus to set an arbitrary focus detection area. However, if the display unit 110 is a touch panel, the input operation of the operation unit 113 may be replaced by a touch operation on the display unit 110.

Next, FIG. 10 is a flowchart showing the flow of processing of the imaging surface phase difference AF in the present embodiment. When the image pickup surface phase difference AF process is started, first, in step S401, the pixel unit 401 of the image sensor 100 is driven, and the signals (focus) of a plurality of PDs included in the unit pixel 200 in the entire area of the pixel unit 401. The detection signal) is read out respectively. At this time, the drive circuit unit 402 sets the pixel unit 401 by a read scanning method such as row thinning read, row addition read, row partial read, column thinning read, column addition read, and column partial read, depending on the required frame rate. It may be driven. However, as described above, since the drive circuit unit 402 drives the pixel unit 401 uniformly under the same conditions, there is no problem that crosstalk or blooming is likely to occur.

After that, the signal of each PD is AD-converted by the reading unit 103 to obtain a digital signal for focus detection. Further, the reading unit 103 can also generate a signal for image generation by mixing the digital signals of a plurality of PDs for each unit pixel 200.

The focus detection signal and the image generation signal in the entire region of the pixel unit 401 are output from the readout unit 103 to the signal processing unit 105. Then, the image generation signal is output from the output unit 106 to the outside of the image pickup device 100 via the signal processing unit 105, and is processed by the image processing unit 107. After that, the generated image is displayed on the display unit 110 by the overall control / calculation unit 109. The image sensor 100 continuously outputs an image generation signal at a predetermined frame rate, so that a moving image is continuously displayed on the display unit 110.

Next, in step S402, the selection mode of the focus detection region is confirmed. Here, if the user has manually set the focus detection area in advance in order to focus the photographing lens 101 on an arbitrary area, the process proceeds to step S403. In this case, FIG. 11 shows an example of an operation screen displayed on the display unit 110 when the user selects the focus detection area. As shown in FIG. 11, a plurality of selectable focus detection areas 1101 are displayed on the operation screen. Here, it is shown by a detection frame divided into 7 horizontally and 5 vertically, but it may be further divided into finer parts or coarsely divided. Further, the user may specify an arbitrary position from the entire shooting screen instead of selecting from a predetermined detection frame. The user selects, for example, the area 1102 including the face of a person as the area to be focused on. Then, the position information of the focus detection area 1102 selected by the user is input from the operation unit 113 to the signal processing unit 105 via the overall control / calculation unit 109.

Next, in step S403, among the focus detection signals of the entire area of the pixel unit 401 output from the read unit 103 to the signal processing unit 105 in step S401, the focus detection signal of the area specified by the user is used as the signal processing unit. The focus detection signal processing unit 702 in 105 selects. Then, the focus detection signal selected by the focus detection signal processing unit 702 is input to the phase difference detection unit 108 via the output unit 106. At this time, in the present embodiment, the focus detection signal transmitted by the output unit 106 to the phase difference detection unit 108 is only the signal required for focus detection control, that is, only the focus detection signal in the region specified by the user. Therefore, high-speed transmission is possible.

Next, in step S404, the phase difference detection unit 108 receives the digital signal for focus detection from the output unit 106, and calculates the phase difference evaluation value for performing the focus detection of the phase difference detection method. At this time, the phase difference detection unit 108 does not need to perform arithmetic processing for calculating the phase difference evaluation value in a region unnecessary for focus detection control and processing for extracting a signal finally required for focus detection control. Therefore, the focus detection control can be performed at high speed.

Next, in step S405, the lens control unit 114 calculates the optical system drive information and controls the focus lens position of the photographing lens 101 based on the phase difference evaluation value calculated by the phase difference detection unit 108.

Next, in step S406, it is confirmed whether the imaging device should finish the photographing. If the user inputs an operation to end shooting from the operation unit 113, shooting ends as it is. If the user does not input an operation to end shooting from the operation unit 113, the process proceeds to step S401 to continue shooting and phase difference AF processing on the imaging surface.

On the other hand, if the image pickup device is set to automatically determine the area to be focused on the photographing lens 101 in step S402, the process proceeds to step S407. In step S407, the subject detection unit 105a receives the image generation signal from the readout unit 103 and performs subject detection processing for automatically determining the region to be focused on the photographing lens 101. The subject to be detected is, for example, the face or eyes of a person or animal. Various known pattern recognition processes can be applied to the method of subject detection processing. As a typical pattern recognition method, for example, a method called template matching or deep learning can be mentioned.

FIG. 12A shows an example of a screen displayed on the display unit 110 for showing the subject area detected by the subject detection unit 105a. As shown in FIG. 12A, the subject area 1201 detected by the subject detection unit 105a is displayed as an area including the face of a person. The subject detection unit 105a outputs the horizontal / vertical address information in the image of the detected subject area 1201 to the signal processing unit 105. The signal processing unit 105 outputs the horizontal / vertical address information in the image of the subject area 1201 to the overall control / calculation unit 109 via the output unit 106 in order to display it on the display unit 110 as the subject area. Then, the overall control / calculation unit 109 synthesizes the subject area information with the generated image processed by the image processing unit 107 and displays it on the display unit 110. Further, although the details will be described later, the signal processing unit 105 uses the horizontal / vertical address information in the image of the subject area 1201 to select the focus detection signal.

Further, as another example of the subject detection result, the case where the face of the person is framed up is shown in FIG. 12 (b). In the example shown in FIG. 12B, since the area including the face of a person is wide, if the focus detection signal in a wide area including the entire face is output to the phase difference detection unit 108 in the subsequent stage, the communication time is assumed. Becomes long and interferes with high-speed focus detection control. Therefore, as shown in FIG. 12B, when the face of a person is detected in close-up, the subject detection unit 105a further extracts a characteristic portion of the face (for example, eyes) and sets the subject detection area 1202. It is preferable to do so. This is realized, for example, by detecting a case where the subject detection unit 105a detects a case where the number of pixels in the region where the face is detected exceeds a predetermined number of pixels and controlling the subject detection target to be switched.

Next, in step S408, the detection result of the subject detection unit 105a in step S407 is confirmed. If the subject can be detected in step S407, the process proceeds to step S409.

In step S409, the signal processing unit 105 selects the focus detection signal in the region detected by the subject detection unit 105a among the focus detection signals read from the entire area of the pixel unit 401. Then, the focus detection signal selectively output from the signal processing unit 105 is input to the phase difference detection unit 108 via the output unit 106. In this case, in the present embodiment, the focus detection signal transmitted by the output unit 106 to the phase difference detection unit 108 is only the signal in the region required for focus detection control, so that the signal can be transmitted at high speed.

Next, in step S404, the phase difference detection unit 108 receives the digital signal for focus detection from the output unit 106, and calculates the phase difference evaluation value for performing the focus detection of the phase difference detection method. In this case, the phase difference detection unit 108 does not need to perform arithmetic processing for calculating the phase difference evaluation value in a region unnecessary for focus detection control and processing for extracting a signal finally required for focus detection control. Therefore, the focus detection control can be performed at high speed. After that, the processes already described in step S405 and step S406 are executed.

On the other hand, if it is confirmed in step S408 that the subject could not be detected in step S407, the process proceeds to step S410.

In step S410, the lens control unit 114 controls the focus lens of the photographing lens 101 so as to drive the focus lens by a predetermined amount. Further, in step S411, the focus lens position of the photographing lens 101 is confirmed, and it is determined whether the search drive is completed. If the search drive is in the middle, the process proceeds to step S401. Therefore, the setting is such that the focus detection area is automatically determined in step S402, and if the state in which the subject detection unit 105a cannot detect the subject continues in step S407, the search drive of the focus lens is continued. However, when the search drive from the infinite end to the nearest end of the focus lens position is completed in step S411, the process proceeds to step S412.

In step S412, even if the lens control unit 114 repeats the operation of searching and driving the focus lens in step S410, the subject detection unit 105a cannot detect the subject in step S407. Here, in order to provisionally determine the focus lens position of the photographing lens 101, the signal processing unit 105 selects the focus detection signal in the provisional region among the focus detection signals read from the entire region of the pixel unit 401. Then, the focus detection signal selectively output from the signal processing unit 105 is input to the phase difference detection unit 108 via the output unit 106.

Next, in step S404, the phase difference detection unit 108 receives the digital signal for focus detection from the output unit 106, and calculates the phase difference evaluation value for performing the focus detection of the phase difference detection method. After that, the processes already described in step S405 and step S406 are executed.

As described above, in the present embodiment, the focus detection signal transmitted by the output unit 106 of the image sensor 100 to the phase difference detection unit 108 is only the signal required for the focus detection control, so that high-speed transmission is possible. It is possible. Further, the phase difference detection unit 108 does not require arithmetic processing for calculating the phase difference evaluation value in a region unnecessary for focus detection control and processing for extracting a signal finally required for focus detection control. Therefore, the processing speed at which the phase difference detection unit 108 calculates the phase difference evaluation value can be increased. Therefore, the focus detection control by the imaging surface phase difference AF can be performed at high speed.

(Fourth Embodiment)
Next, a fourth embodiment of the present invention will be described. In the fourth embodiment, the configuration of the unit pixel 200 of the pixel unit 402 is different. FIG. 13 is a view of the light receiving unit 102 of the image pickup device 100 and the microlens array observed from the optical axis direction (Z direction). Four photoelectric conversion units 901A, 901B, 901C, and 901D are arranged for one microlens 202. As described above, by having a total of four photoelectric conversion units having two in each of the X-axis direction and the Y-axis direction, it is possible to receive the light in the exit pupil region divided into four. The signal readout method and the processing of the signal processing unit 105 in the image pickup device 100 including the pixel unit 401 in which each such unit pixel has four photoelectric conversion units will be described.

FIG. 14 is a schematic diagram showing an example of the configuration of the unit pixel 900 and the reading unit 103. In the configuration of FIG. 14, a readout circuit corresponding to each photoelectric conversion unit is provided. That is, the pixel signal of the photoelectric conversion unit 901A is output to the readout circuit 1001A. Similarly, the photoelectric conversion unit 901B is output to the readout circuit 1001B, the photoelectric conversion unit 901C is output to the readout circuit 1001C, and the photoelectric conversion unit 901D is output to the readout circuit 1001D. Since the signal reading operation from the photoelectric conversion unit can be carried out by substantially the same method as the driving method described with reference to FIGS. 5 and 6, the description thereof will be omitted.

The processing of the signal processing unit 105 for the signal read from each photoelectric conversion unit will be described. The captured image signal processing unit 701 calculates a captured image signal from the read signal. That is, it receives the pixel signals S (A), S (B), S (C), S (D) of the plurality of photoelectric conversion units 901A, 901B, 901C, and 901D of the unit pixel 900, performs mixing processing, and pixels. The signal S (A + B + C + D) is calculated. Then, the pixel signal S (A + B + C + D) is sent to the output unit 106 via the captured image signal output line 703. In the captured image signal processing unit 701, the signal transmission amount to the outside of the image pickup element 100 can be reduced by mixing and processing the signals of the plurality of photoelectric conversion units and outputting the signals from the output unit 106 to the outside of the image pickup element 100. can. The effect is even greater in an image pickup device that includes four photoelectric conversion units, such as the unit pixel 900.

Subsequently, the processing of the focus detection signal processing unit 702 will be described. In the case of an image sensor having four photoelectric conversion units per unit pixel 900 as shown in FIG. 13, it is necessary to individually output the signals of the four photoelectric conversion units for focus detection, and the amount of signal transmission is enormous. This is not desirable for high-speed reading. As described in the first embodiment, it is preferable to calculate and output the Y value, or to output only the correlation calculation result. If the signal is output only in the required area, the amount of signal transmission can be further reduced.

Further, in the case of a configuration in which the unit pixel 900 is provided with a 2x2 photoelectric conversion unit as shown in FIG. 13, it is possible to detect the phase difference in the vertical direction in addition to the horizontal direction. For example, the pixel signals S (A) and S (C) are mixed and processed, and the pixel signals S (B) and S (D) are mixed and output. In this case, from the obtained focus detection signal, it is possible to perform focus detection by a phase difference method in which the pupil is divided in the left-right direction. Further, when the pixel signals S (A) and S (B) are mixed and processed, and the pixel signals S (C) and S (D) are mixed and output, the obtained focus detection signals are up and down. It is possible to detect the focal point by the phase difference method in which the pupil is divided in the direction. By switching and outputting these according to the subject, it is possible to accurately detect the focus on each subject of vertical stripes and horizontal stripes. Further, the output pattern may be changed depending on the pixel area.

Further, the focus detection signal processing unit 702 may perform a correlation calculation using the obtained mixed signals (for example, S (A + C) and S (B + D)) and output only the result. If the focus detection signal processing unit 702 performs correlation calculation processing, the amount of signal transmission output from the image sensor 100 can be reduced. Further, the signal transmission amount can be suppressed even if the correlation calculation in both the left-right direction and the up-down direction is performed and output in the same region.

As described above, in the image sensor provided with the multi-divided pixels, by providing the focus detection signal processing unit 702 in the image sensor, an increase in the amount of signal output from the image sensor is suppressed, and the captured image data and the focus are performed at high speed. Detection information can be obtained. Further, since the phase difference information in the left-right direction and the up-down direction can be acquired, the focus can be detected with high accuracy. In the present embodiment, the image sensor having four photoelectric conversion units per unit pixel has been described as an example, but a configuration having more photoelectric conversion units may be used. In order to obtain more parallax, only the necessary PD signal may be output, or the signal may be mixed and output in the diagonal direction.

(Fifth Embodiment)
Since the signal processing unit 105 of the image pickup device 100 as described in the first to third embodiments is a large-scale circuit, the image pickup device 100 is likely to have a large area as a whole. Therefore, in the present embodiment, the configuration of the image pickup device 100 that suppresses the increase in the area will be described.

15 and 16 are block diagrams of the image pickup device 100 according to the fifth embodiment. The image pickup device of this embodiment has a configuration (multilayer structure) in which a pixel region chip 1501 and a signal processing chip 1502 are laminated. The wiring between the semiconductor chips is electrically connected by using a micro bump or the like by a known substrate lamination technique.

The pixel region chip 1501 includes a pixel unit 401 in which unit pixels 200 having a plurality of photoelectric conversion units are arranged in a matrix, a drive circuit unit 402, and a readout unit 103. The drive circuit unit 402 sends a drive signal to the pixels of the pixel unit 401. In FIG. 15, the unit pixel 200 has two photoelectric conversion units, but the number of photoelectric conversion units is not limited to this.

The reading unit 103 is configured to include a large number of reading circuits 509, for example, one reading circuit 509 for each pixel row, and reads out the pixel signal of the pixel unit 401. The read pixel signal is selected vertically or horizontally under the control of the drive circuit unit 402, and is sequentially transferred to the signal processing unit 105.

The signal processing chip 1502 includes a control unit 104, a signal processing unit 105, and an output unit 106. The signal processing unit 105 has a signal processing unit 701 for captured images and a signal processing unit 702 for focus detection, processes the pixel signal read from the reading unit 103, and processes the pixel signal read from the reading unit 103, and the image sensor 100 passes through the output unit 106. Output to the outside. Since the signal processing in the signal processing unit 105 is the same as the processing described in the first to third embodiments, the description thereof will be omitted. As shown in FIG. 16, the image pickup device 100 has a configuration in which the pixel region chip 1501 and the signal processing chip 1502 are laminated and integrated with each other.

As described above, by forming the image pickup element in a laminated structure, a sufficient area can be taken in the signal processing unit 105, and a large-scale circuit can be mounted. The signal processing unit 105 can reduce the amount of signal transmission by performing signal processing that outputs only the necessary signals to the outside of the image pickup element 100, and can obtain both the captured image data and the focus detection information at high speed. Become.

(Sixth Embodiment)
In an image pickup device composed of a pixel unit including a plurality of photoelectric conversion units per unit pixel as described in the first to third embodiments, it is preferable that the number of readout circuits 509 is large. For example, in the case of a configuration in which one readout circuit is provided for each unit pixel or a configuration in which one readout circuit is provided for one photoelectric conversion unit, a pixel signal is output simultaneously for all pixels and A for each pixel. Since / D conversion can be performed, higher-speed reading becomes possible. In this case, an area is required for arranging the readout circuit, and it is desirable that the image sensor has a laminated structure.

FIG. 17 is a diagram showing the configuration of the image pickup device 100 according to the sixth embodiment. The image pickup device of this embodiment has a configuration in which a pixel region chip 1301, a readout circuit chip 1302, and a signal processing chip 1303 are laminated. The wiring between the semiconductor chips is electrically connected by using a micro bump or the like by a known substrate lamination technique.

The pixel region chip 1301 includes a pixel unit 401 and a drive circuit unit 402 in which unit pixels 200 having a plurality of photoelectric conversion units are arranged in a matrix. The drive circuit unit 402 sends a drive signal to the pixels of the pixel unit 401. In FIG. 17, the unit pixel 200 includes two photoelectric conversion units, but the number of photoelectric conversion units is not limited to two.

The readout circuit chip 1302 includes a readout unit 103, a vertical selection circuit 1304, and a horizontal selection circuit 1305. The reading unit 103 has a large number of reading circuits 509 corresponding to each of the unit pixel or the photoelectric conversion unit, and the pixel signal of the pixel unit 401 is output. The pixel signal output to the readout circuit 509 is sequentially transferred to the signal processing unit 105 under the control of the vertical selection circuit 1304 and the horizontal selection circuit 1305.

The signal processing chip 1303 includes a control unit 104, a signal processing unit 105, and an output unit 106. The signal processing unit 105 has a signal processing unit 701 for captured images and a signal processing unit 702 for focus detection, and the pixel signal read by the reading unit 103 is processed by the signal processing unit 105 and passed through the output unit 106. Is output to the outside of the image pickup element 100. Since the signal processing in the signal processing unit 105 is the same as the processing described in the first to third embodiments, the description thereof will be omitted.

The image pickup device 100 has a configuration in which a pixel region chip 1301, a readout circuit chip 1302, and a signal processing chip 1303 are laminated and integrated. By the way, in the case of a configuration including, for example, one readout circuit for one photoelectric conversion unit as in the configuration of the present embodiment, the time for outputting the pixel signal to the signal processing unit 105 is dramatically increased. For example, assuming that the time required for signal output of one photoelectric conversion unit is α, in the case of an image sensor equipped with one read circuit per column, it takes time for αx rows to read a pixel signal for one frame. It takes. On the other hand, when one readout circuit is provided for one photoelectric conversion unit, the pixel signal for one frame can be read out in the time of α. However, in this case, there is a concern that the signal processing unit 105 receives the pixel signal and the signal processing is rate-determining. However, as in the present embodiment, since the signal processing unit 105 of the image pickup device 100 having a laminated structure can have a large area, it is possible to provide a large number of transmission lines for signals from the readout circuit chip 1302, and the speed is high. The pixel signal can be sent to the signal processing unit 105. Further, since a large number of signal processing circuits can be mounted on the signal processing unit 105, parallel processing becomes possible and the signal processing time is shortened.

As described above, the laminated structure of the image pickup device makes it possible to secure a sufficient area for the readout unit and the signal processing unit. The image pickup device of the present embodiment also speeds up signal reading from the pixel section, and further reduces the signal transmission amount by performing signal processing in the signal processing section 105 to output only necessary signals to the outside of the image pickup device. This makes it possible to obtain both captured image data and focus detection information at high speed.

(7th Embodiment)
FIG. 18 is a block diagram showing a configuration of an image pickup apparatus according to a seventh embodiment of the present invention. In FIG. 18, the same components as those in the third embodiment are designated by the same symbols, and detailed description thereof will be omitted.

The seventh embodiment has the same components as those of the third embodiment when viewed as a whole of the image pickup device, but the components included inside the image pickup device 800 are different from those of the third embodiment. The image pickup device 800 of the third embodiment includes a light receiving unit 801, a readout unit 802, a signal processing unit 803, an image processing unit 804, a subject detection unit 805, a phase difference detection unit 806, an output unit 807, and a control unit 104. It is configured.

The light receiving unit 801 receives the optical image formed by the photographing lens 101. The light receiving unit 801 is arranged with focus detection pixels having a plurality of PDs under one microlens so as to receive the light flux passing through the divided exit pupil region of the photographing lens 101. The readout unit 802 performs analog-digital signal processing by the A / D conversion circuit and adjustment of the reference level (clamp processing).

The signal processing unit 803 receives the digital signal output from the reading unit 103, and outputs a signal to the phase difference detecting unit 806 and the image processing unit 804, which will be described later. At this time, the signal processing unit 803 receives the subject detection result of the subject detection unit 805 and selectively outputs the focus detection signal in the subject detection area.

The image processing unit 804 performs image processing such as defect pixel correction, noise reduction, color conversion, white balance correction, gamma correction, resolution conversion processing, and image compression for the image generation signal received from the signal processing unit 803. Perform processing etc. The subject detection unit 805 receives the digital signal output from the image processing unit 804 and determines a signal region for performing the focus detection process.

The phase difference detection unit 806 calculates a phase difference evaluation value for performing focus detection by the phase difference detection method with respect to the focus detection signal received from the signal processing unit 803. The output unit 807 outputs the phase difference evaluation value and the digital signal for image generation received from the phase difference detection unit 806 and the image processing unit 804 to the outside of the image pickup device 800. As described above, the image sensor of the seventh embodiment of the present invention also has the same effect as that of the third embodiment.

By the way, in the imaging surface phase difference AF, one phase difference evaluation value is calculated by correlating the focus detection signal obtained from a plurality of pixels × a plurality of PDs. In the third embodiment, the output unit 106 outputs a plurality of focus detection digital signals necessary for calculating the phase difference evaluation value to the outside of the image pickup device 100. On the other hand, in the case of the seventh embodiment, the output unit 807 outputs the phase difference evaluation value calculated by the phase difference detection unit 806 to the outside of the image sensor 800. That is, the amount of the signal output to the outside of the image sensor 800 is further smaller in the seventh embodiment than in the third embodiment. Therefore, the output unit 807 of the image pickup device 800 can transmit the signal required for the focus detection control in a shorter time than in the third embodiment, and the focus detection control by the image pickup surface phase difference AF is performed at high speed. be able to.

Further, in the image pickup device 800 of the seventh embodiment, the subject detection unit 805 detects the subject by using the digital signal processed by the image processing unit 804. Therefore, it is possible to perform subject detection processing using a signal to which defect pixel correction and noise reduction have been performed, and further using color information. Therefore, the subject detection of the seventh embodiment can be performed with higher accuracy than that of the third embodiment.

In the image sensor 800 of the seventh embodiment, more components than the image sensor 100 of the third embodiment are configured in the image sensor 800. Therefore, a stacked image sensor as shown in FIG. 16 is preferable.

(8th Embodiment)
In the first to third embodiments, it has been described that the image pickup device selectively outputs / transmits a signal for focus detection, whereby high-speed image pickup surface phase-difference AF becomes possible. However, the scope of application of the present invention is not limited to the image pickup surface phase difference AF, but can also be applied to automatic exposure control (imaging surface AE) using the signal of the image pickup element. Specifically, on the image pickup surface AE, the image pickup device automatically determines and controls the aperture of the photographing lens, the accumulation time of the image pickup element, the sensitivity (gain) of the image pickup element, and the like. A third embodiment of the present invention aims to increase the speed of the image pickup surface AE by selectively outputting / transmitting the signal used for the image pickup surface AE by the image pickup element.

FIG. 19 is a block diagram showing a configuration of an image pickup apparatus according to an eighth embodiment of the present invention. In FIG. 19, the same components as those in the third embodiment are designated by the same symbols, and detailed description thereof will be omitted. This embodiment is different from the third embodiment in that the image pickup apparatus includes a photometric unit 820.

The signal processing unit 105 receives a digital signal output for image generation from the reading unit 103, and outputs a signal to the output unit 106. The signal processing unit 105 selects a digital signal for image generation to be used for the image pickup surface AE according to the photometric method set in the image pickup apparatus. At this time, for example, in the case of the so-called "spot metering method" or "partial metering method", the signal processing unit 105 selects a signal in a part of the center of the screen. Further, in the case of the so-called "evaluation metering method", the signal processing unit 105 selects the signal of the entire screen. However, instead of selecting the signals of all the pixels of the entire screen, it is possible to select the signals by thinning out the rows or columns within the range where the signal amount required for calculating the photometric evaluation value can be obtained. Is preferable for speeding up. Further, as in the third embodiment, the image generation signal of the focus detection region arbitrarily selected by the user or the subject region detected by the subject detection unit 105a may be selected.

The photometric unit 820 receives a digital signal for image generation from the output unit 106 and calculates a photometric evaluation value for performing image pickup surface AE. The digital signal for image generation input to the photometric unit 820 is a signal in a region selected by the signal processing unit 105 inside the image sensor 100. Therefore, since the image generation signal for the image pickup surface AE transmitted by the output unit 106 of the image sensor 100 to the photometric unit 820 is only a signal necessary for exposure control, the communication band is efficiently used. .. Further, also in the internal processing of the photometric unit 820, there is no need for arithmetic processing for calculating the photometric evaluation value in a region finally unnecessary for the imaging surface AE and processing for extracting the finally necessary signal. Therefore, the processing speed at which the photometric unit 820 calculates the photometric evaluation value can be increased.

The lens control unit 114 receives the output of the photometric evaluation value from the photometric unit 820 and drives the aperture of the photographing lens 101 based on the photometric evaluation value. Further, the overall control / calculation unit 109 drives the image pickup device 100 based on the photometric evaluation value to control the accumulation time and sensitivity (gain) of the image pickup device 100.

As described above, according to the eighth embodiment of the present invention, the signal for image generation transmitted by the output unit 106 of the image sensor 100 to the photometric unit 820 is only the signal required for the image pickup surface AE. , High-speed transmission is possible. Then, in the photometric unit 820, there is no need for arithmetic processing for calculating the photometric evaluation value that is finally unnecessary for the image pickup surface AE, and processing for extracting the finally necessary signal. Therefore, the speed of the image pickup surface AE can be increased.

In the pixels of the image pickup device 100 of the eighth embodiment, it is not always necessary to arrange a plurality of PDs for each pixel as described with reference to FIG. In order to calculate the photometric evaluation value, only a signal for image generation (that is, a signal obtained by mixing the signals of a plurality of PDs under one microlens for each pixel) is required, so one under one microlens. It may be configured to have two PDs.

Further, in the eighth embodiment, the configuration in which the imaging surface AE is applied based on the configuration of the third embodiment has been described, but similarly, the imaging surface AE is applied based on the configuration of the seventh embodiment. You can also do it. In this case, the phase difference detection unit 806 in FIG. 18 is replaced with the photometric unit to form the image sensor 800.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes can be made within the scope of the gist thereof.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100: Image sensor, 101: Photograph lens, 102: Light receiving unit, 103: Read unit, 104: Control unit, 105: Signal processing unit, 106: Output unit, 107: Image processing unit, 108: Phase difference detection unit, 109 : Overall control / calculation unit

Claims (14)

A pixel section in which multiple pixels that photoelectrically convert light from the subject are arranged,
A reading unit that reads a signal from the pixel unit and
Among the signals read by the reading unit, the signal of the pixels in the entire region of the pixel unit is output to the outside of the image pickup element as a signal for generating an image, and the drive control of the device including the image pickup element is performed. As a signal for calculating the evaluation value to be used, an output unit that outputs the signal of the pixel of the subject region, which is the region where the subject exists in the pixel portion, to the outside of the image sensor, and an output unit.
An image pickup device characterized by being provided with. The image pickup device according to claim 1, wherein the signal for calculating the evaluation value used for the drive control of the device including the image pickup device is a signal for performing focus detection control. The output unit is characterized in that a signal from a pixel in the subject region is output to the outside of the image pickup device as a signal that is a basis for calculating a phase difference evaluation value used for focus detection control of a phase difference detection method. The image pickup device according to claim 2. A pixel section in which multiple pixels that photoelectrically convert light from the subject are arranged,
A reading unit that reads a signal from the pixel unit and
Among the signals read by the reading unit, the signal of the pixels in the entire region of the pixel unit is output to the outside of the image pickup element as a signal for generating an image, and the drive control of the device including the image pickup element is performed. As a signal for calculating the evaluation value to be used, an output unit that outputs a signal of a pixel in a part of the pixel unit to the outside of the image sensor, and an output unit.
An image pickup device characterized by being provided with. The image pickup device according to claim 4, further comprising a selection means for selecting the partial region. The image pickup device according to claim 5, further comprising a setting means for setting either a mode in which the partial area is designated by the user or a mode in which the partial area is automatically selected. The image pickup device according to any one of claims 4 to 6, wherein the evaluation value includes a phase difference evaluation value. The image pickup device according to any one of claims 4 to 6, wherein the evaluation value includes a photometric evaluation value. A pixel section in which multiple pixels that photoelectrically convert light from the subject are arranged,
A reading unit that reads a signal from the pixel unit and
A phase difference detection unit for calculating the phase difference evaluation value used for the focus detection control of the phase difference detection method, and
Of the signals read by the reading unit, the signals of the pixels in the entire area of the pixel unit are output to the outside of the image sensor as signals for generating an image, and in the region where the subject exists in the pixel unit. An output unit that outputs the phase difference evaluation value calculated by the phase difference detection unit using the signal of a pixel in a certain subject area to the outside of the image sensor, and an output unit.
An image pickup device characterized by being provided with. The image pickup device according to any one of claims 1 to 9, further comprising an AD conversion unit that converts a signal output from the pixel into a digital signal. The image pickup device according to any one of claims 1 to 10, wherein each of the pixels has one microlens and a plurality of photoelectric conversion units. The image pickup device according to claim 11, wherein the reading unit reads out all the signals of the plurality of photoelectric conversion units from each of the pixels as a signal for forming an image. The image pickup device according to any one of claims 1 to 12, wherein the image pickup device has a laminated structure in which a plurality of substrates are laminated. The image sensor according to any one of claims 1 to 13, and the image sensor.
A display unit that displays an image captured by the image sensor, and a display unit.
A control means for controlling the entire device including the image sensor and the display unit, and
An imaging device characterized by being provided with.

Images