JP2020178163A - Imaging apparatus and control method of imaging apparatus - Google Patents

Abstract

To prevent the displacement of a moving subject and expand the dynamic range.SOLUTION: When a signal based on a charge obtained by transferring a charge accumulated in a photoelectric conversion unit by a first transfer unit and a second transfer unit to a third holding unit is smaller than a reference signal, an analog-to-digital conversion unit performs analog-to-digital conversion on a first signal based on a charge obtained by transferring the charge accumulated in the photoelectric conversion unit by the first transfer unit and the second transfer unit to the third holding unit, and when the signal based on the charge obtained by transferring the charge accumulated in a photoelectric conversion unit by the first transfer unit and the second transfer unit to the third holding unit is larger than the reference signal, the analog-to-digital conversion unit performs analog-to-digital conversion on a second signal based on the charge obtained by transferring the charge accumulated in the photoelectric conversion unit by the first transfer unit and the second transfer unit to the third holding unit and the charge obtained by transferring the charge held in a first holding unit by a third transfer unit to the third transfer unit.SELECTED DRAWING: Figure 7

Description

The present invention relates to an image pickup apparatus and a control method for the image pickup apparatus.

In recent years, there are known imaging devices capable of capturing images using an imaging element such as a digital still camera or a digital video camera and storing the captured images as digital data. As the image sensor, there is a CMOS (Complementary Metal Oxide Semiconductor) type image sensor (hereinafter referred to as a CMOS sensor) that reads out each pixel signal by the XY address method. In the image sensor, the photoelectric conversion element of the pixel accumulates electric charges according to the amount of incident light, and performs photoelectric conversion to output an electric signal corresponding to the accumulated charges.

Further, the image sensor has an electronic shutter function. The electronic shutter function starts the exposure by resetting the photoelectric conversion element of the pixel, and ends the exposure by reading out the electric charge accumulated in the photoelectric conversion element. Since the image sensor controls the start and end of exposure, it is possible to realize accurate control of the exposure time from a low-speed shutter to a high-speed shutter.

Further, the CMOS sensor has a rolling shutter operation (also called a focal plane shutter operation). In the rolling shutter operation of the CMOS sensor, unlike the CCD sensor, a plurality of pixels arranged in two dimensions are sequentially scanned for each line to reset the charge of the pixels. Then, after the elapse of a predetermined exposure time, the CMOS sensor sequentially scans each line to read out the accumulated charges and output a signal. As described above, the rolling shutter operation is an operation having a time difference for charge reading and signal output for each line. As a result, the timing of exposure for each line shifts in one shooting operation.

To solve this problem, Patent Document 1 discloses a CMOS sensor having a global shutter function. In the CMOS sensor, each pixel is provided with a charge holding unit, and exposure is started by simultaneously performing an electronic shutter operation on all pixels. Further, at the end of the exposure period, the CMOS sensor ends the exposure by transferring the charge of the photoelectric conversion element to the charge holding unit at the same time for all the pixels. As a result, the CMOS sensor can simultaneously perform the exposure operation of all the pixels.

The imaging device has a problem that the dynamic range tends to be insufficient when capturing an image of a subject in which bright and dark areas are mixed. For example, in an imaging apparatus, if the exposure time is controlled to be short in accordance with a bright place, a sufficient exposure time cannot be obtained in a dark part, so that the image quality is deteriorated due to blackout and deterioration of S / N. On the contrary, if the exposure time of the imaging device is controlled to be long according to a dark place, the accumulated charge amount of the photoelectric conversion element reaches the saturation level, and the subject region having a certain brightness or more is set to the saturated brightness level. Overexposure occurs.

Patent Document 2 discloses a dynamic range expansion process (hereinafter referred to as HDR process) as a method for accurately reproducing the gradation of a bright portion and a dark portion. In HDR processing, the image sensor uses a signal with a long exposure time for pixels with a small amount of incident light on the image sensor to achieve a high S / N ratio, and uses a signal with a short exposure time for pixels with a large amount of incident light. , Avoid saturation. The image pickup apparatus synthesizes the image pickup signal exposed for a long time and the image pickup signal exposed for a short time.

Japanese Unexamined Patent Publication No. 2014-060519 Japanese Unexamined Patent Publication No. 2001-346096

However, in Patent Document 2, since the long exposure period and the short exposure period are different timings, in the case of a moving subject, the images are taken out of alignment. Therefore, there is a problem that it is difficult to realize the global shutter function and the HDR processing at the same time in one shooting. Further, in the short-time exposure of a pixel having a large amount of incident light, the charge that can be handled is limited to the saturated charge amount of the photoelectric conversion element, so that there is a problem that the dynamic range expansion process is limited.

An object of the present invention is to prevent a moving subject from shifting and to expand the dynamic range.

According to one aspect of the present invention, it has a plurality of pixels and an analog-digital conversion unit that converts the signal of the pixels from analog to digital, and each of the plurality of pixels converts light into electric charges and described above. A photoelectric conversion unit that stores the converted charge, a first holding unit that holds the charge overflowing from the photoelectric conversion unit, a second holding unit that holds the charge, and the photoelectric conversion unit are stored. A first transfer unit that transfers an electric charge to the second holding unit, a third holding unit for holding the electric charge, and a third holding unit that holds the electric charge held in the second holding unit. The analog-digital conversion unit includes a second transfer unit that transfers the electric charge to the unit and a third transfer unit that transfers the electric charge held in the first holding unit to the third holding unit. When the charge accumulated in the photoelectric conversion unit by the first transfer unit and the second transfer unit is smaller than the reference signal, the signal based on the charge transferred to the third holding unit is the first. The electric charge accumulated in the photoelectric conversion unit by the transfer unit 1 and the second transfer unit converts the first signal based on the electric charge transferred to the third holding unit from analog to digital, and the first signal is converted. When the charge accumulated in the photoelectric conversion unit by the transfer unit 1 and the second transfer unit is larger than the reference signal, the signal based on the charge transferred to the third holding unit is the first. The electric charge accumulated in the photoelectric conversion unit by the transfer unit and the second transfer unit is held in the first holding unit by the charge transferred to the third holding unit and the third transfer unit. Provided is an imaging device that converts a second signal based on the electric charge transferred to the third transfer unit from analog to digital.

According to the present invention, it is possible to prevent the displacement of a moving subject and expand the dynamic range.

It is a figure which shows the configuration example of the image pickup apparatus. It is a figure which shows the structural example of the image sensor. It is a circuit diagram which shows the structural example of a pixel. It is a figure which shows the structural example of the column signal processing part. It is a timing chart which shows the operation of shooting. It is a timing chart which shows the operation of a pixel. It is a timing chart which shows the control method of an image pickup apparatus.

The embodiment described below is an example as a means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions, and the present invention is the following embodiment. It is not limited to.

FIG. 1 is a diagram showing a configuration of an image pickup apparatus 100 according to the present embodiment. The image pickup device 100 is, for example, a digital still camera or a digital video camera. The image pickup device 100 includes an optical system 111, an image pickup element 112, a signal processing unit 113, a compression / expansion unit 114, a synchronization control unit 115, an operation unit 116, an image display unit 117, and an image recording unit 118.

The optical system 111 includes a lens for forming an image of a subject, a lens driving mechanism for zooming and focusing, a mechanical shutter mechanism, an aperture mechanism, and the like. The optical system 111 is driven based on a control signal from the synchronization control unit 115.

The image sensor 112 is an XY address type CMOS sensor, which converts an optical image formed by the optical system 111 into an electric signal in response to a control signal from the synchronous control unit 115, and changes the electric signal from analog to digital. Converts and outputs a digital image signal.

The synchronization control unit 115 controls the signal processing unit 113. The signal processing unit 113 receives signal processing such as white balance adjustment, color correction, and gamma correction for the image signal input from the image sensor 112, and control information such as AF (Auto Focus) and AE (Auto Exposure). Is detected.

Under the control of the synchronization control unit 115, the compression / decompression unit 114 performs compression coding processing on the image signal signal-processed by the signal processing unit 113, or encodes data of a still image supplied from the synchronization control unit 115. Is decompressed and decoded. Further, the compression / decompression unit 114 may execute the compression coding / decompression / decoding process of the moving image.

The synchronization control unit 115 is, for example, a microcontroller composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The synchronization control unit 115 controls each component of the image pickup apparatus 100 by executing a program stored in a ROM or the like.

The operation unit 116 has various operation keys such as a shutter release button, a lever, a dial, and the like, and outputs a control signal corresponding to an input operation by the user to the synchronization control unit 115.

The image display unit 117 includes a display device such as a liquid crystal display and an interface circuit for the display device. The image display unit 117 generates an image signal for display from the image signal supplied from the synchronization control unit 115, supplies the image signal to the display device, and displays the image.

The image recording unit 118 inputs and records an image file compressed and encoded by the compression / decompression unit 114 from the synchronization control unit 115 on a recording medium such as a portable semiconductor memory. Further, the image recording unit 118 reads a designated image file from the recording medium based on the control signal from the synchronization control unit 115, and outputs the image file to the synchronization control unit 115.

Next, the operation of the image pickup apparatus 100 will be described. Before capturing a still image, the image sensor 112 sequentially outputs an image signal to the signal processing unit 113 in frame units. The signal processing unit 113 performs signal processing on the image signal from the image sensor 112 and outputs it as a camera-through image signal to the image display unit 117 via the synchronization control unit 115. The image display unit 117 displays a camera-through image, and the user can adjust the angle of view by looking at the displayed image.

When the shutter release button of the operation unit 116 is pressed, the image sensor 112 outputs an image signal for one frame as a still image to the signal processing unit 113 under the control of the synchronization control unit 115. The signal processing unit 113 performs signal processing on the image signal for one frame, and outputs the processed image signal to the compression / decompression unit 114 via the synchronization control unit 115.

The compression / decompression unit 114 compresses and encodes the input image signal, generates encoded data, and outputs the encoded data to the synchronization control unit 115. The synchronization control unit 115 generates an image file including the coded data, and outputs the image file to the image recording unit 118. The image recording unit 118 records an image file of a still image on a recording medium.

Next, the still image reproduction process will be described. The synchronization control unit 115 reads the selected image file from the image recording unit 118 and outputs the image file to the compression / decompression unit 114 in response to the operation of the still image reproduction of the operation unit 116. The compression / decompression unit 114 executes decompression / decoding processing on the coded data in the image file, and outputs the decompression / decoding processed image signal to the image display unit 117 via the synchronization control unit 115. The image display unit 117 displays the input image signal as a still image.

Next, the moving image recording process will be described. By operating the operation unit 116, recording of a moving image is instructed. The image sensor 112 sequentially outputs an image signal to the signal processing unit 113 in frame units. The signal processing unit 113 performs signal processing on the image signal from the image sensor 112 and outputs it to the compression / decompression unit 114 via the synchronization control unit 115. The compression / decompression unit 114 performs compression coding processing on the image signal after signal processing, sequentially generates coded data of moving images, and outputs the coded data to the synchronization control unit 115. The synchronization control unit 115 generates a moving image file including the encoded data of the moving image and outputs the moving image file to the image recording unit 118. The image recording unit 118 records the moving image file on the recording medium.

Next, the video reproduction process will be described. Playback of the moving image is instructed by the operation of the operation unit 116. The synchronization control unit 115 reads a moving image file from the image recording unit 118 and outputs it to the compression / decompression unit 114. The compression / decompression unit 114 executes decompression / decoding processing on the coded data in the moving image file, and outputs the decompression / decoding processed image signal to the image display unit 117 via the synchronization control unit 115. The image display unit 117 displays the input image signal as a moving image.

FIG. 2 is a diagram showing a configuration example of the image pickup device 112 of FIG. The image sensor 112 is, for example, a CMOS sensor. The image pickup device 112 includes a pixel region 201, a vertical scanning unit 202, a plurality of column signal processing units 203, a horizontal scanning unit 207, an output unit 209, and a timing unit 211.

The pixel region 201 has a plurality of pixels 200 arranged in a matrix. The plurality of pixels 200 are represented by matrix pixels P11 to P86. The pixel 200 in the first row is represented by pixels P11 to P16. The pixel 200 on the eighth row is represented by pixels P81 to P86. In FIG. 2, a pixel 200 having 8 rows and 6 columns is shown as an example. The number of matrices of the pixel 200 is not limited to this number.

In the pixels 200 of the odd-numbered rows, the pixels provided with the R (red) filter and the pixels provided with the G (green) filter are alternately arranged. In the even-numbered row of pixels 200, pixels provided with a G (green) filter and pixels provided with a B (blue) filter are alternately arranged.

The vertical scanning unit 202 selects the pixels 200 in the pixel area 201 line by line, and controls the reset operation and the read operation of the selected pixel lines. The vertical scanning unit 202 is connected to the pixels 200 of each row via the control line 221 of each row. The vertical scanning unit 202 controls the pixels 200 on a line-by-line basis.

The signal lines 231 in each row are commonly connected to the pixels 200 in each row. When the pixel 200 in each row is selected by the control line 221, the pixel signal is output to the row signal processing unit 203 in each row via the signal line 231 in each row.

The column signal processing unit 203 of each column converts the pixel signal of the signal line 231 of each column from analog to digital, respectively. The horizontal scanning unit 207 selects the column signal processing unit 203 for each column via the column selection line 251 of each column, and transmits the digital pixel signal stored in the column signal processing unit 203 via the output line 261. Transfer to the output unit 209. The output unit 209 outputs a digital pixel signal to the signal processing unit 113 in units of lines.

The timing unit 211 outputs various clock signals, control signals, and the like to each component of the image pickup device 112 based on the control signal from the synchronization control unit 115. The timing 211 outputs a clock signal, a control signal, or the like to the vertical scanning unit 202, the column signal processing unit 203, and the horizontal scanning unit 207 via the control lines 271, 281 and 285.

FIG. 3 is a circuit diagram showing a configuration example of the pixel 200. The pixel 200 includes a photoelectric conversion unit PD, a transfer transistor GST, a holding transistor GSH, a read transistor GTX, a control transistor OFC, a holding transistor OFS, and a read transistor OTX. Further, the pixel 200 has a floating diffusion (FD) 300, a reset transistor RST, an amplification transistor TD, and a selection transistor SEL. Each of the pixels 200 in each row is connected to the signal line 231 in each row. A load transistor 231 is connected to each of the signal lines 231 in each row.

The transfer unit 301 includes a transfer transistor GST, a holding transistor GSH, and a read transistor GTX. The transfer unit 302 includes a control transistor OFC, a holding transistor OFS, and a read transistor OTX.

The vertical scanning unit 202 controls the gate signals of the transfer transistor GST, the holding transistor GSH, the reading transistor GTX, the control transistor OFC, the holding transistor OFS, and the reading transistor OTX via the control line 221. Further, the vertical scanning unit 202 controls the gate signals of the reset transistor RST and the selection transistor SEL via the control line 221.

The photoelectric conversion unit PD is, for example, a photodiode, which converts light into electric charges and stores the converted electric charges. In the photoelectric conversion unit PD, the anode is connected to the ground potential node, and the cathode is connected to the transfer transistor GST and the control transistor OFC.

The cathode of the photoelectric conversion unit PD is connected to the FD 300 via the transfer transistor GST, the holding transistor GSH, and the readout transistor GTX. The transfer transistor GST, the holding transistor GSH, and the read transistor GTX are MOS field effect transistors.

The cathode of the photoelectric conversion unit PD is connected to the FD 300 via the control transistor OFC, the holding transistor OFS, and the readout transistor OTX. The control transistor OFC, the holding transistor OFS, and the readout transistor OTX are MOS field effect transistors. The FD 300 is a charge holding unit that holds charges.

In the reset transistor RST, the drain is connected to the power supply voltage node Vdd, the source is connected to the FD300, and the voltage of the FD300 is reset to the voltage of the power supply voltage node Vdd.

The amplification transistor TD has a gate connected to the FD300, a drain connected to the power supply voltage node Vdd, a source connected to the drain of the selection transistor SEL, and outputs a voltage corresponding to the amount of charge held in the FD300. In the selection transistor SEL, the source is connected to the signal line 231 and the output signal of the amplification transistor TD is output to the signal line 231 as the output signal of the pixel 200.

In the load transistor TL, the source and the gate are connected to the ground potential node, and the drain is connected to the signal line 231. The load transistor TL and the amplification transistor TD form a source follower circuit. When the pixel 200 outputs a signal, the load transistor TL operates as a constant current source for grounding the gate.

Transistors other than the control transistor OFC, amplification transistor TD and load transistor TL act as switches, conducting (turning on) when the gate signal is high level and shutting off (turning off) when the gate signal is low level.

The FD 300 converts the charge into a voltage. It is desirable that the FD 300 be operated within a range in which linearity can be maintained in relation to the voltage that becomes the pixel signal and the electric charge that is transferred. Therefore, the amount of charge immediately before the photoelectric conversion unit PD overflows is set so that the amount of charge that can maintain the linearity of the charge-voltage conversion in the FD 300 is substantially equal. In this setting, the ratio of the capacity of the photoelectric conversion unit PD to the capacity of the FD 300 is 1: 1.

Further, the saturation capacitance that can be held by the holding transistor GSH is called a GSH capacitance. Further, the saturation capacitance in which the holding transistor OFS can hold a charge is called an OFS capacitance. The ratio of the capacity of the photoelectric conversion unit PD, the capacity of the FD 300, the GSH capacity, and the OFS capacity is set to be approximately 1: 1: 1: 3. The conversion gain of the charge-voltage conversion of the FD300 is 1 times.

Here, the ratio of the capacity of the photoelectric conversion unit PD, the capacity of the FD 300, and the GSH capacity may be 1: 1: 1. The ratio of the capacitance of the photoelectric conversion unit PD to the OFS capacitance is not limited to 1: 3, and if it is set to 1 or more, there is an effect described later.

FIG. 4 is a diagram showing a configuration example of the column signal processing unit 203 of FIG. The column signal processing unit 203 includes a switch circuit 401, a comparator 402, a counter circuit 403, a latch circuit 404, an arithmetic circuit 405, an AND circuit 406, and a sample hold capacitance 407. The control line 281 includes a control line pSwS, a signal line Vrmp, a control line pCNT, a control line pLTC, and a control line pCAL.

The AND circuit 406 outputs a logical product signal of the signals of the control lines pSwS and pSwH to the switch circuit 401. The switch circuit 401 writes the signal of the signal line 231 to the sample hold capacitance 407 according to the output signal of the AND circuit 406. The signal line Vsig is connected to the sample hold capacitance 407. The switch circuit 401 conducts when the output signal of the AND circuit 406 is high level, and cuts off when the output signal of the AND circuit 406 is low level.

The timing unit 211 outputs the lamp wave G1 or the determination signal Vjd of FIG. 7 to the signal line Vrmp. As shown in FIG. 7, the ramp wave G1 is a triangular wave that gradually increases from the initial voltage. The amplitude of the lamp wave G1 may be sufficiently larger than the amplitude of the signal of the signal line Vsig connected to the sample hold capacitance 407.

The comparator 402 compares the signal of the signal line Vsig with the lamp wave G1 of the signal line Vrmp, and outputs the comparison result to the counter circuit 403. For example, when the magnitude relationship between the signal of the signal line Vsig and the lamp wave G1 of the signal line Vrmp is reversed, the output signal output by the comparator 402 to the counter circuit 403 changes from a high level to a low level.

Further, the comparator 402 compares the signal of the signal line Vsig with the reference signal Vjd of the signal line Vrmp, and outputs the comparison result to the control line pSwH. For example, when the signal of the signal line Vsig is larger than the reference signal Vjd of the signal line Vrmp, the control line pSwH becomes a high level. When the signal of the signal line Vsig is smaller than the reference signal Vjd of the signal line Vrmp, the control line pSwH becomes low level. Hereinafter, the case where the control line pSwH is at a high level is referred to as a case of high illuminance, and the case where the control line pSwL is at a low level is referred to as a case of low illuminance.

The timing unit 211 starts outputting a clock signal to the control line pCNT when the level change of the lamp wave G1 of the signal line Vrmp starts. The counter circuit 403 performs a counter based on the clock signal of the control line pCNT, and when the output signal of the comparator 402 changes from a high level to a low level, the count value at that time is held and output to the latch circuit 404. This count value is a digital value obtained by converting the signal of the signal line Vsig from analog to digital. The comparator 402, the counter circuit 403, and the latch circuit 404 are analog-to-digital converters.

The latch circuit 404 holds the count value output by the counter circuit 403, and outputs the held count value to the arithmetic circuit 405 in response to the signal of the control line pLTC.

The calculation circuit 405 is a calculation unit, stores a count value output by the latch circuit 404 in response to a signal on the control line pCAL, performs signal processing, and holds a pixel signal. Further, the arithmetic circuit 405 outputs the stored pixel signal to the output line 261 in response to the signal of the column selection line 251.

FIG. 5 is a timing chart showing the photographing operation of the image pickup apparatus 100. The frame synchronization signal FS is a frame-by-frame synchronization signal, and is effective at the falling edge. The frame synchronization signal FS falls at equal intervals of times s01, s03, s06, s08, and s11. The vertical direction in FIG. 5 indicates line numbers from the first line to the last line when the pixel area 201 of the image sensor 112 is controlled line by line.

The frames starting with the times s01 and s06 are the frames for the exposure in the global shutter operation. At time s02 and s07, the electric charge of the photoelectric conversion unit PD is reset at the same time in all the pixels 200, and the exposure starts.

The frames starting with the times s03 and s08 are the frames for performing the read operation in the global shutter operation. At times s03 and s08, at all pixels 200, the transfer transistor GST simultaneously transfers the electric charge accumulated in the photoelectric conversion unit PD to the holding transistor GSH to end the exposure. The holding transistor GSH holds an electric charge. As a result, the global shutter operation by simultaneous exposure can be realized.

At times s04 to s05 and times s09 to s10, the read transistor GTX transfers the electric charge held by the holding transistor GSH to the FD300 in the row order of all pixels 200. The amplification transistor TD outputs a voltage corresponding to the amount of electric charge held in the FD 300 to the signal line 231 via the selection transistor SEL.

In this way, the image pickup apparatus 100 realizes the global shutter operation by combining the exposure frame and the readout frame. The image pickup apparatus 100 can also shoot a moving image by alternately performing exposure frames and readout frames.

FIG. 6 is a timing chart showing a control method of the image sensor 112. The line synchronization signal LS is a line-by-line synchronization signal, and is effective at the falling edge. The line synchronization signal LS is a synchronization signal from the first line to the last line when the pixel area 201 is controlled line by line within the period of the frame synchronization signal FS.

The control line 221 includes control lines pSEL, pRST, pOFC, pOFS, pOTX, pGST, pGSH and pGTX. The control line pSEL is connected to the gate of the selection transistor SEL. The control line pRST is connected to the gate of the reset transistor RST. The control line pOFC is connected to the gate of the control transistor OFC. The control line pOFS is connected to the gate of the holding transistor OFS. The control line pOTX is connected to the gate of the readout transistor OTX. The control line pGST is connected to the gate of the transfer transistor GST. The control line pGSH is connected to the gate of the holding transistor GSH. The control line pGTX is connected to the gate of the readout transistor GTX.

Times t1 to t8 indicate lines for performing simultaneous reset of all pixels at times s02 and s07 in FIG. At time t1, the line synchronization signal LS falls.

At time t2, the control lines pRST, pOFC, pOFS, pOTX, pGST, pGSH and pGTX are at high levels. Then, the reset transistor RST, the control transistor OFC, the holding transistor OFS, the reading transistor OTX, the transfer transistor GST, the holding transistor GSH, and the reading transistor GTX are turned on. As a result, the voltages of the photoelectric conversion unit PD, the transfer unit 301, the transfer unit 302, and the FD 300 are reset to the voltage of the power supply voltage node Vdd.

At time t3, the control line pGST goes low and the transfer transistor GST goes off. At time t4, the control line pGSH goes to a low level and the holding transistor GSH goes off. At time t5, the control line pGTX becomes low level and the read transistor GTX is turned off. As a result, the reset of the transfer unit 301 is released.

Further, at time t4, the control line pOFC becomes low level and the control transistor OFC is turned off. At time t5, the control line pOFS becomes low level and the holding transistor OFS is turned off. At time t6, the control line pOTX goes low and the read transistor GTX goes off. The readout transistor GTX is in a state where the holding transistor OFS and the FD300 are disconnected. As a result, the reset of the transfer unit 302 and the photoelectric conversion unit PD is released.

At time t7, the control line pOFS becomes high level and the holding transistor OFS is turned on. At time t8, the control line pOFC becomes the third potential. The third potential is higher than the low level and lower than the high level. Further, at time t8, the control line pRST becomes a low level, the reset transistor RST is turned off, and the reset of the FD300 is released.

When the control line pOFC reaches the third potential, the control transistor OFC guides the charge overflowed by the photoelectric conversion unit PD to the holding transistor OFS instead of the holding transistor GSH. The photoelectric conversion unit PD can accumulate a predetermined amount of electric charge. The electric charge exceeding a predetermined amount in the photoelectric conversion unit PD flows into the holding transistor OFS via the control transistor OFC. The holding transistor OFS is a holding unit that holds the electric charge overflowing from the photoelectric conversion unit PD via the control transistor OFC.

At time t2, the transfer transistor GST is turned off. After that, at time t3, when the control transistor OFC is turned off, the exposure starts. That is, the photoelectric conversion unit PD is released from the reset, and the charge accumulation period starts.

Times t9 to t12 indicate lines at which all pixels are simultaneously transferred at times s03 and s08 in FIG. At time t9, the line synchronization signal LS becomes low level.

At time t10, the control lines pGST and pGSH are at a high level, and the transfer transistor GST and the holding transistor GSH are turned on. The transfer transistor GST transfers the electric charge stored in the photoelectric conversion unit PD to the holding transistor GSH. The holding transistor GSH is a holding portion and holds the transferred charge.

At time t11, the control line pOFC becomes low level and the control transistor OFC is turned off. The control transistor OFC ends the transfer of the overflowed charge.

At time t12, the control line pGST goes to a low level and the transfer transistor GST goes off. The transfer transistor GST ends the transfer of the electric charge stored in the photoelectric conversion unit PD. As a result, the exposure is completed, and the charge accumulation period of the photoelectric conversion unit PD is completed.

FIG. 7 is a waveform diagram showing an operation of reading a signal of pixel 200 for each row at times s04 to s05 and times s09 to s10 of FIG. The signal line Vsig is connected to the input terminal of the comparator 402 from the sample hold capacitance 407. The signal line Vrmp is connected to the input terminal from the timing unit 211 to the comparator 402. The vertical direction of FIG. 7 indicates the voltage V, and the horizontal direction of FIG. 7 indicates the time t. The voltage V is a voltage based on the initial voltage of the lamp wave G1.

In the period tr1, the column signal processing unit 203 initially sets. The initial setting is, for example, clamping the input signal of the comparator 402. The control lines pSwH and pSwS are at high levels. The AND circuit 406 outputs a high level, and the switch circuit 401 connects the signal line 231 to the signal line Vsig. The sample hold capacitance 407 holds the signal of the signal line 231. The comparator 402 internally clamps the signals of the signal lines Vsig and Vrmp as reference levels.

At time t13 in FIG. 6, the line synchronization signal LS becomes low level. At time t14, the control line pSEL becomes high level and the selection transistor SEL is turned on.

Further, at time t14, the control line pRES becomes high level and the reset transistor SEL is turned on. The reset transistor SEL resets the FD 300 to the voltage of the power supply voltage node Vdd.

At time t15, the control line pRES becomes high level and the reset transistor SEL is turned on. The reset transistor SEL resets the FD 300 to the voltage of the power supply voltage node Vdd. The holding transistors GSH and OFS each maintain the above-mentioned charge-holding state.

The period tt1 in FIG. 7 is the period after the time t15 in FIG. In the period tt1, the amplification transistor TD outputs the N signal Vn_H based on the reset release of the FD 300 to the signal line Vsig via the selection transistor SEL and the switch circuit 401. The N signal Vn_H is held in the sample hold capacitance 407.

After the period tt1, the timing unit 211 may lower the control line pSwS and turn off the switch circuit 401. The N signal Vn_H is held in the sample hold capacitance 407.

In the period tr2, the timing unit 211 starts the level change of the lamp wave G1 of the signal line Vrmpn and starts the output of the clock signal of the control line pCNT. The counter circuit 403 starts counting the count value based on the clock signal. The comparator 402 compares the N signal Vn_H of the signal line Vsig with the lamp wave G1 of the signal line Vrmp.

The period tngs is the counting period of the N signal Vn_H of the counting circuit 403. At the end time of the period tngs, the magnitude relationship between the ramp wave G1 and the N signal Vn_H is reversed, and the output signal of the comparator 402 changes from high level to low level. Then, the counter circuit 403 outputs the count value at that time to the latch circuit 403. This count value is a digital value cngs obtained by converting the N signal Vn_H from analog to digital. The latch circuit 403 and the arithmetic circuit 405 hold the digital value cngs of the N signal Vn_H.

Next, the processing at times t16 to t18 in FIG. 6 is performed. At time t16, the control line pGTX goes to a high level and the read transistor GTX is turned on. The read transistor GTX transfers the electric charge held by the holding transistor GSH to the FD300. The FD 300 is a holding unit and holds the transferred charge.

At time t17, the control line pGSH goes to a low level and the holding transistor GSH goes off. At time t18, the control line pGTX goes low and the read transistor GTX goes off. The read transistor GTX ends the transfer of charges held in the holding transistor GSH. The holding transistor OFS maintains a state in which the overflow charge is held. The timing unit 211 raises the control line pSwS to a high level and turns on the switch circuit 401.

In the period tt2 of FIG. 7, the amplification transistor TD outputs the S signal VgsL or VgsH based on the photoelectric conversion held in the FD 300 to the signal line Vsig via the selection transistor SEL and the switch circuit 401. The S signal VgsL is a signal when the photoelectric conversion unit PD is incident with low-illuminance light. The S signal VgsH is a signal when the photoelectric conversion unit PD injects high-intensity light.

In the case of low illuminance, the pixel 200 outputs the S signal VgsL, the charge of the photoelectric conversion unit PD does not overflow, and the charge is not held by the holding transistor OFS. In the case of high illuminance, the pixel 200 outputs the S signal VgsH, the charge of the photoelectric conversion unit PD overflows, and the charge is held in the holding transistor OFS.

After the period tt2, the timing unit 211 may lower the control line pSwS and turn off the switch circuit 401. The sample hold capacitance 407 holds the S signal VgsL or VgsH.

First, a case where the pixel 200 outputs the S signal VgsH will be described. In the period tr3, the timing unit 211 outputs a fixed level reference signal Vjd with respect to the signal line Vrmp. The comparator 402 compares the S signal VgsH of the signal line Vsig with the reference signal Vjd of the signal line Vrmp. Since the S signal VgsH is larger than the reference signal Vjd, the comparator 402 outputs a high level to the control line pSwH. Since the control line pSwH is at a high level, the AND circuit 406 outputs a high level and the switch circuit 401 is turned on.

Next, a case where the pixel 200 outputs the S signal VgsL will be described. In the period tr3, the comparator 402 outputs a low level to the control line pSwH because the S signal VgsL is smaller than the reference signal Vjd. The AND circuit 406 outputs a low level, and the switch circuit 401 is turned off. The sample hold capacitance 407 holds the S signal VgsL.

The period tt3 corresponds to the period after the time t19 in FIG. At time t19, the control line pOTX goes to a high level and the read transistor OTX is turned on. The readout transistor OTX is in a state in which the holding transistor OFS and the FD300 are connected. The readout transistor OTX transfers the overflow charge of the holding transistor OFS to the FD300. The FD 300 adds the charge transferred by the read transistor OTX to the charge transferred by the read transistor GTX, and holds the added charge.

The readout transistor OTX connects the capacitance of the holding transistor OFS and the capacitance of the FD300. The ratio of the capacitance of the FD300 to the capacitance of the holding transistor OFS is approximately 1: 3. The charge-to-voltage conversion gain of the capacitances of the holding transistors OFS and FD300 is 1/4 times the conversion gain of the FD300 capacitance alone.

In the case of low illuminance, the pixel 200 outputs the S signal VgsL, the charge of the photoelectric conversion unit PD does not overflow, and the charge is not held by the holding transistor OFS.

As a result, in the period tt3 of FIG. 7, in the case of low illuminance, the S signal VgsL decreases to the S signal VgoL in the signal line Vsig. The S signal VgoL is a signal that is 1/4 times the S signal VgsL. The S signal VgoL is a signal corresponding to the electric charge held in the capacitance of the holding transistor GSH.

Similarly, in the case of high illuminance, the S signal VgsH decreases to the S signal VgoH in the signal line Vsig. The S signal VgoH is a signal corresponding to a charge obtained by adding the electric charge held in the capacitance of the holding transistor GSH and the electric charge held in the capacitance of the holding transistor OFS.

In the case of high illuminance, the S signal VgoH is output to the signal line Vsig, and the switch circuit 401 is turned on as described above. The sample hold capacitance 407 holds the S signal VgoH. After the period tt3, the timing unit 211 may lower the control line pSwS and turn off the switch circuit 401. The sample hold capacitance 407 holds the S signal VgoH.

In the case of low illuminance, the S signal VgoL is output to the signal line Vsig, and the switch circuit 401 is turned off as described above. The sample hold capacitance 407 still holds the S signal VgsL.

In the period tr4, the timing unit 211 starts the level change of the lamp wave G1 of the signal line Vrmpn and starts the output of the clock signal of the control line pCNT. The counter circuit 403 starts counting the count value based on the clock signal.

In the case of high illuminance, the comparator 402 compares the S signal VgoH of the signal line Vsig with the lamp wave G1 of the signal line Vrmp. The period tsgoH is the counting period of the S signal VgoH of the counting circuit 403. At the end time of the period tsgoH, the magnitude relationship between the ramp wave G1 and the S signal VgoH is reversed, and the output signal of the comparator 402 changes from a high level to a low level. Then, the counter circuit 403 outputs the count value at that time to the latch circuit 403. This count value is a digital value csgoH obtained by converting the S signal VgoH from analog to digital. The latch circuit 403 and the arithmetic circuit 405 hold the digital value csgoH of the S signal VgoH.

In the case of low illuminance, the comparator 402 compares the S signal VgsL of the signal line Vsig with the lamp wave G1 of the signal line Vrmp. The period tsgsL is the counting period of the S signal VgsL of the counting circuit 403. At the end time of the period tsgsL, the magnitude relationship between the ramp wave G1 and the S signal VgsL is reversed, and the output signal of the comparator 402 changes from high level to low level. Then, the counter circuit 403 outputs the count value at that time to the latch circuit 403. This count value is a digital value csgsL obtained by converting the S signal VgsL from analog to digital. The latch circuit 403 and the arithmetic circuit 405 hold the digital value csgsL of the S signal VgsL.

The rate of change of the lamp wave G1 in the period tr4 is the same as the rate of change of the lamp wave G1 in the period tr2. The maximum level of the ramp wave G1 of the period tr4 may be the level of the reference signal Vjd.

The period tt4 corresponds to the period from time t20 to t21 in FIG. At time t20, the control signal pRST becomes high level and the reset transistor RST is turned on. The readout transistor OTX remains on. The reset transistor RST resets the capacitance of the holding transistor OFS and the capacitance of the FD300 to the voltage of the power supply voltage node Vdd.

At time t21, the control signal pRST goes low and the reset transistor RST goes off. The reset transistor RST releases the reset of the capacitance of the holding transistor OFS and the capacitance of the FD300.

As a result, in the period tt4, the pixel 200 outputs the N signal Vn_L based on the reset release of the capacitance of the holding transistor OFS and the capacitance of the FD300. In the case of high illuminance, the timing unit 211 sets the control line pSwS to a high level, turns on the switch circuit 401, and the sample hold capacitance 407 holds the N signal Vn_L. The N signal Vn_L is an N signal when the read transistor OTX connects the capacitance of the holding transistor OFS and the capacitance of the FD300.

In the case of low illuminance, the switch circuit 401 maintains the off state, and the sample hold capacitance 407 maintains the state in which the S signal VgsL is held.

In the period tr5, the timing unit 211 starts the level change of the lamp wave G1 of the signal line Vrmpn and starts the output of the clock signal of the control line pCNT. The counter circuit 403 starts counting the count value based on the clock signal. In the case of high illuminance, the comparator 402 compares the N signal Vn_L of the signal line Vsig with the lamp wave G1 of the signal line Vrmp.

The period tngo is the counting period of the N signal Vn_L of the counting circuit 403. At the end time of the period tngo, the magnitude relationship between the ramp wave G1 and the N signal Vn_L is reversed, and the output signal of the comparator 402 changes from high level to low level. Then, the counter circuit 403 outputs the count value at that time to the latch circuit 403. This count value is a digital value cngo obtained by converting the N signal Vn_L from analog to digital. The latch circuit 403 and the arithmetic circuit 405 hold the digital value cngo of the N signal Vn_L.

The calculation circuit 405 calculates the difference between the digital value csgsL of the S signal VgsL and the digital value cngs of the N signal Vn_H when the control line pSwH is at a low level and the illuminance is low, and the difference is calculated as the pixel value of the S signal VgsL. Remember as.

Further, the arithmetic circuit 405 calculates the difference between the digital value csgoH of the S signal VgoH and the digital value cngo of the N signal Vn_L when the control line pSwH is at a high level and the light is high, and the value is four times the difference. Is stored as the pixel value of the S signal VgoH. This quadruple is a correction process for reducing the conversion gain between the capacitance of the holding transistor OFS and the capacitance of the FD300 to 1/4. As a result, the arithmetic circuit 405 can correct the conversion gain of the pixel value in the case of high illuminance with respect to the pixel value in the case of low illuminance.

In order to correct the difference in the conversion gain of the charge-voltage conversion of the pixel value in the case of low illuminance and the pixel value in the case of low illuminance, the arithmetic circuit 405 sets the digital value csgoH of the S signal VgoH and the digital value cngo of the N signal Vn_L. Correct the difference of. Instead of the correction, the arithmetic circuit 405 may correct the difference between the digital value csgsL of the S signal VgsL and the digital value cngs of the N signal Vn_H.

The period tt5 corresponds to the period from time t22 to t24 in FIG. At time t22, the control line pOFS becomes low level and the holding transistor OFS is turned off. At time t23, the control line pOTX becomes low level and the read transistor OTX is turned off. The readout transistor OTX cuts the capacitance of the FD300 from the capacitance of the holding transistor OFS. The conversion gain of the charge-voltage conversion of the capacitance of the FD300 returns to 1 time.

At time t24, the control signal pSEL becomes low level and the selection transistor SEL is turned off. The selection transistor SEL cuts the amplification transistor TD from the signal line 231. After that, the next line synchronization signal LS becomes a robel, and the processing of the next line starts.

The horizontal scanning unit 207 sequentially selects the arithmetic circuit 405 of the column signal processing unit 203 of each row via the column selection line 251 of each row. The arithmetic circuit 405 of each column sequentially outputs the pixel values of each column to the output unit 209 via the output line 261.

As described above, the image pickup apparatus 100 can simultaneously realize the global shutter function and the dynamic range expansion process in one shooting, and can increase the amount of electric charge that the pixel 200 can handle. The image sensor 112 realizes a global shutter operation by simultaneous exposure of all 200 pixels by using a holding transistor OFS for holding the overflow charge of the photoelectric conversion unit PD. In the global shutter operation, all the pixels 200 are simultaneously exposed, so that it is possible to prevent a moving subject from shifting.

Further, the image pickup device 112 can generate a pixel value when the charge of the photoelectric conversion unit PD overflows by adding the charges held by the holding transistors OFS and GSH. Further, the image sensor 112 increases the conversion gain of charge-voltage conversion by 1/4 in the case of the high-illumination S signal VgsH, and increases the conversion gain of charge-voltage conversion by 1 in the case of the low-illumination S signal VgsL. By doing so, it is possible to realize HDR processing with an expanded dynamic range. Since HDR processing can be realized only by performing analog-to-digital conversion of the S signal once in the period tr4, the frame rate can be improved.

The image pickup device 100 can be applied to smartphones, tablets, industrial cameras, medical cameras, in-vehicle cameras, etc., in addition to digital cameras and video cameras.

It should be noted that all of the above embodiments merely show examples of embodiment in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from the technical idea or its main features.

100 image sensor, 112 image sensor, 200 pixels, 203 column signal processing unit, 300 FD, PD photoelectric conversion unit, GST transfer transistor, GSH holding transistor, GTX read transistor, OFC control transistor, OFS holding transistor, OTX read transistor

Claims (7)

With multiple pixels
It has an analog-to-digital conversion unit that converts the signal of the pixel from analog to digital.
Each of the plurality of pixels
A photoelectric conversion unit that converts light into electric charges and stores the converted electric charges.
A first holding unit that holds the electric charge overflowing from the photoelectric conversion unit, and
A second holding part for holding the electric charge,
A first transfer unit that transfers the electric charge accumulated in the photoelectric conversion unit to the second holding unit, and
A third holding part for holding the electric charge,
A second transfer unit that transfers the electric charge held in the second holding unit to the third holding unit, and
It has a third transfer unit that transfers the electric charge held in the first holding unit to the third holding unit.
The analog-to-digital converter
When the signal based on the charge accumulated in the photoelectric conversion unit by the first transfer unit and the second transfer unit and transferred to the third holding unit is smaller than the reference signal, the first transfer unit is used. The charge accumulated in the photoelectric conversion unit by the transfer unit 1 and the second transfer unit converts the first signal based on the charge transferred to the third holding unit from analog to digital.
When the signal based on the charge accumulated in the photoelectric conversion unit by the first transfer unit and the second transfer unit and transferred to the third holding unit is larger than the reference signal, the first transfer unit is used. The charge accumulated in the photoelectric conversion unit by the transfer unit 1 and the second transfer unit is held in the first holding unit by the charge transferred to the third holding unit and the third transfer unit. An image pickup apparatus characterized in that a second signal based on the charge transferred to the third transfer unit is converted from analog to digital. The photoelectric conversion unit of the plurality of pixels is reset at the same time.
The imaging device according to claim 1, wherein the first transfer unit simultaneously transfers the electric charge accumulated in the photoelectric conversion unit to the second holding unit in the plurality of pixels. The analog-to-digital converter
When the charge accumulated in the photoelectric conversion unit by the first transfer unit and the second transfer unit is smaller than the reference signal, the signal based on the charge transferred to the third holding unit is the first. The first signal based on the amount of electric charge held in the third holding unit in a state where the transfer unit 3 cuts the first holding unit and the third holding unit is converted from analog to digital. ,
When the charge accumulated in the photoelectric conversion unit by the first transfer unit and the second transfer unit is larger than the reference signal, the signal based on the charge transferred to the third holding unit is the first. A second signal based on the amount of electric charge held in the third holding unit in a state where the transfer unit 3 connects the first holding unit and the third holding unit is converted from analog to digital. The imaging device according to claim 1 or 2, characterized in that. The analog-to-digital converter
A third signal based on the reset release of the third holding unit in a state where the third transferring unit disconnects the first holding unit and the third holding unit is converted from analog to digital.
An analog fourth signal based on the reset release of the first holding unit and the third holding unit in a state where the third transfer unit connects the first holding unit and the third holding unit. The imaging apparatus according to any one of claims 1 to 3, further comprising converting from to digital. It also has a calculation unit,
The calculation unit
When the signal based on the charge accumulated in the photoelectric conversion unit by the first transfer unit and the second transfer unit is transferred to the third holding unit is smaller than the reference signal, the first transfer unit is used. Calculate the difference between the digital value of the signal 1 and the digital value of the third signal,
When the charge accumulated in the photoelectric conversion unit by the first transfer unit and the second transfer unit is larger than the reference signal, the signal based on the charge transferred to the third holding unit is the first. The imaging device according to claim 4, wherein the difference between the digital value of the signal 2 and the digital value of the fourth signal is calculated. The calculation unit corrects the difference between the digital value of the first signal and the digital value of the third signal, or the difference between the digital value of the second signal and the digital value of the fourth signal. The imaging device according to claim 5, wherein the image pickup apparatus is characterized by the above. With multiple pixels
A control method for an imaging device having an analog-to-digital conversion unit that converts a pixel signal from analog to digital.
Each of the plurality of pixels
A photoelectric conversion unit that converts light into electric charges and stores the converted electric charges.
A first holding unit that holds the electric charge overflowing from the photoelectric conversion unit, and
A second holding part for holding the electric charge,
A first transfer unit that transfers the electric charge accumulated in the photoelectric conversion unit to the second holding unit, and
A third holding part for holding the electric charge,
A second transfer unit that transfers the electric charge held in the second holding unit to the third holding unit, and
It has a third transfer unit that transfers the electric charge held in the first holding unit to the third holding unit.
The analog-digital conversion unit uses the reference signal as a signal based on the charge accumulated in the photoelectric conversion unit by the first transfer unit and the second transfer unit and transferred to the third holding unit. When it is small, the charge accumulated in the photoelectric conversion unit by the first transfer unit and the second transfer unit transfers a first signal based on the charge transferred to the third holding unit from analog. Convert to digital
The analog-digital conversion unit uses the reference signal as a signal based on the charge accumulated in the photoelectric conversion unit by the first transfer unit and the second transfer unit and transferred to the third holding unit. When it is large, the charge accumulated in the photoelectric conversion unit by the first transfer unit and the second transfer unit is transferred to the third holding unit, and the charge is transferred by the third transfer unit. A control method for an image pickup apparatus, characterized in that a charge held in a first holding unit converts a second signal based on the charge transferred to the third transfer unit from analog to digital.

Images