JP2017103603A - Image pickup device, imaging apparatus, and imaging method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image pickup device which has a high degree of freedom of read timing of electric charges from a photoelectric conversion part.SOLUTION: An image pickup device (1) comprises: a first photoelectric conversion part (3) and a second photoelectric conversion part (4) which are arranged in a first pixel and convert light to electric charges; a first transfer part (21) for transferring the electric charges from the first photoelectric conversion part; a first transfer control line (22) which sends a control signal to the first transfer part; a second transfer part (41) for transferring electric charges from the second photoelectric conversion part; and a second transfer control line (42) which is provided separately from the first transfer control line and sends a control signal to the second transfer part.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an imaging element, an imaging apparatus, and an imaging method.

  A CMOS image sensor having a plurality of photoelectric conversion units in one pixel is known. (For example, refer to Patent Document 1). It has been difficult to perform imaging by changing the accumulation times of the two photoelectric conversion units in one pixel.

JP 2001-250931 A

  According to the first aspect of the present invention, the first photoelectric conversion unit and the second photoelectric conversion unit which are arranged in the first pixel and convert light into electric charge, and the first photoelectric conversion unit for transferring electric charge from the first photoelectric conversion unit. 1 transfer unit, a first transfer control line for transmitting a control signal to the first transfer unit, a second transfer unit for transferring charges from the second photoelectric conversion unit, and a first transfer control line. There is provided an imaging device including a second transfer control line that transmits a control signal to the two transfer units.

  According to a second aspect of the present invention, there is provided an imaging apparatus including the imaging element of the first aspect and an imaging control unit that controls the imaging element.

  According to the third aspect of the present invention, of the first photoelectric conversion unit and the second photoelectric conversion unit that are arranged in the first pixel and convert light into electric charge, from the first photoelectric conversion unit to the period of one frame. There is provided an imaging method including transferring charges in a first period and transferring charges from a second photoelectric conversion unit to a second period different from the first period in one frame period. .

It is a figure showing an image sensor concerning an embodiment. It is a figure which shows the circuit structure of the image pick-up element which concerns on embodiment. 6 is a timing chart illustrating an example of the operation of the image sensor according to the embodiment. It is a timing chart which shows the other example of operation of the image sensor concerning an embodiment. It is a figure which shows the other example of the some photoelectric conversion part of the image pick-up element which concerns on embodiment. It is a figure which shows the circuit structure of the image pick-up element which concerns on embodiment. 6 is a timing chart illustrating an example of the operation of the image sensor according to the embodiment. It is a block diagram which shows the imaging device which concerns on embodiment. It is a flowchart which shows an example of operation | movement of the imaging device which concerns on embodiment.

[First Embodiment]
1A is a plan view showing the image sensor 1 according to this embodiment, FIG. 1B is an enlarged plan view showing pixels, and FIG. 1C is A- of FIG. 1B. It is A sectional view. As shown in FIG. 1A, the image sensor 1 has a plurality of pixels P arranged in the pixel region 1a. The plurality of pixels P are two-dimensionally arranged in a lattice shape. One of the arrangement directions of the pixels P is called a horizontal scanning direction, and the other is called a vertical scanning direction. In the present embodiment, a direction perpendicular to each of the horizontal scanning direction and the vertical scanning direction is appropriately referred to as a normal direction of the pixel region 1a (imaging device 1). The image sensor 1 is generally plate-shaped, and the normal direction of the pixel region 1 a corresponds to the thickness direction of the image sensor 1.

  As shown in FIG. 1B, the pixel region 1a is provided with a light blocking portion 2 that blocks light. The light shielding unit 2 is, for example, a black matrix. The light shielding portion 2 is formed in a lattice shape and extends in each of the horizontal scanning direction and the vertical scanning direction. A region surrounded by the light shielding portion 2 is a pixel opening Pa through which light passes. In the present embodiment, the pixel P is a region surrounded by the center line of the light shielding unit 2 when viewed from the normal direction of the pixel region 1a in FIG. 1 and includes the pixel opening Pa and the surrounding light shielding unit 2.

  Each of the plurality of pixels P is provided with a photoelectric conversion unit 3 and a photoelectric conversion unit 4. When viewed from the normal direction of the pixel region 1a, at least a part of the photoelectric conversion unit 3 and at least a part of the photoelectric conversion unit 4 are arranged inside one pixel opening Pa. The photoelectric conversion unit 3 and the photoelectric conversion unit 4 each convert light that has passed through the pixel opening Pa into electric charges.

  In the present embodiment, the image sensor 1 can capture a full color image. The plurality of pixels P include a red pixel R, a green pixel Gr, a green pixel Gb, and a blue pixel B. In FIG. 1B, the array of pixels P is a Bayer array, and in the horizontal scanning direction, red pixels R and green pixels Gr are alternately arranged, and green pixels Gb and blue pixels B are alternately arranged. In the vertical scanning direction, red pixels R and green pixels Gb are alternately arranged, and green pixels Gr and blue pixels B are alternately arranged. In the present embodiment, the photoelectric conversion unit 3 and the photoelectric conversion unit 4 are provided in each color pixel.

  The imaging device 1 includes an imaging unit 5, a signal processing unit 6, and a storage unit 7 that are stacked on each other. The signal processing unit 6 is stacked on the storage unit 7 and is electrically connected to the storage unit 7 by a conductive bump or the like (not shown). The imaging unit 5 is stacked on the signal processing unit 6 and is electrically connected to the signal processing unit 6 by a conductive bump or the like (not shown).

  The imaging unit 5 is, for example, a backside illumination type CMOS image sensor. The imaging unit 5 includes an element layer 11, a light receiving layer 12, a color filter layer 13, and a lens layer 14. A passivation film, a planarizing film, an antireflection film, etc. (not shown) may be provided between these layers. The imaging unit 5 is arranged with the lens layer 14 facing the imaging target when imaging. Light from the imaging target enters the light receiving layer 12 via the lens layer 14 and the color filter layer 13.

  The lens layer 14 is laminated on the color filter layer 13. The lens layer 14 is a microlens array, for example, and includes a plurality of lens elements 14a. For example, the lens element 14a has a one-to-one correspondence with the pixel P, and is provided for each pixel P. The optical axis 14b of the lens element 14a is set so as to pass through the center of the pixel aperture Pa (pixel P), for example. Each of the plurality of lens elements 14a condenses light incident from the outside on the light receiving layer 12 of the pixel P provided with the lens element 14a.

  The color filter layer 13 is laminated on the light receiving layer 12. The color filter layer 13 includes a first filter 15, a second filter 16, and the light shielding unit 2. The first filter 15 is provided in the pixel opening Pa of the red pixel R. The first filter 15 transmits light in the red wavelength band and absorbs light outside the red wavelength band. The red wavelength band includes, for example, 700 nm and is a wavelength band of 620 nm to 750 nm. The second filter 16 is provided in the pixel opening Pa of the green pixel Gr. The second filter 16 transmits light in the green wavelength band and absorbs light outside the green wavelength band. The green wavelength band includes, for example, 546.1 nm and is a wavelength band of 495 nm to 570 nm.

  Although not shown in FIG. 1C, the second filter 16 is provided in the pixel opening Pa of the green pixel Gb, and the third filter is provided in the pixel opening Pa of the blue pixel B. The third filter transmits light in the blue wavelength band and absorbs light outside the blue wavelength band. The blue wavelength band includes, for example, 435.8 nm and is a wavelength band of 450 nm to 495 nm.

  The light shielding unit 2 absorbs light in a wavelength band in which the photoelectric conversion unit 3 and the photoelectric conversion unit 4 have sensitivity. For example, in the case of an imaging device used for a visible light camera or the like, the light shielding portion 2 is formed of a material that absorbs light in a wavelength band of 380 nm to 780 nm. The light shielding unit 2 is provided, for example, so as to suppress crosstalk between two adjacent pixels. For example, the thickness or the like of the light shielding unit 2 is set so as to shield the light that passes through the pixel opening Pa of the pixel P and that travels toward the light receiving layer 12 of another pixel P.

  The light receiving layer 12 is stacked on the element layer 11. The light receiving layer 12 includes a photoelectric conversion unit 3 and a photoelectric conversion unit 4. The photoelectric conversion unit 3 and the photoelectric conversion unit 4 include, for example, a photodiode. However, the photoelectric conversion unit 3 and the photoelectric conversion unit 4 may be any unit that converts light into electric charge, and may include, for example, a phototransistor. In FIG. 1C, the photoelectric conversion unit 4 is provided symmetrically with the photoelectric conversion unit 3 with respect to a plane parallel to the optical axis and the vertical scanning direction of the lens element 14a (a plane perpendicular to the horizontal scanning direction). .

  The photoelectric conversion unit 4 is provided in the vicinity of the photoelectric conversion unit 3 of the same pixel P. For example, the interval d1 between the photoelectric conversion unit 3 and the photoelectric conversion unit 4 in the same pixel P is set narrower than the interval d2 between the photoelectric conversion unit 3 and the photoelectric conversion unit 4 of the adjacent pixel P, but the interval d2 Or may be set longer than the interval d2.

  The photoelectric conversion unit 3 and the photoelectric conversion unit 4 are insulated from each other. For example, the movement of charges is suppressed between the photoelectric conversion unit 3 and the photoelectric conversion unit 4 in the same pixel P by, for example, a PN junction. Further, between the photoelectric conversion unit 3 and the photoelectric conversion unit 4 of the adjacent pixel P, for example, an insulating unit such as an oxide film formed by a LOCOS method or a trench structure is provided. Note that the structure between the photoelectric conversion unit 3 and the photoelectric conversion unit 4 in the same pixel P and the structure between the photoelectric conversion unit 3 and the photoelectric conversion unit 4 of the adjacent pixel P may be the same. .

  The element layer 11 is provided on the side opposite to the light incident side with respect to the light receiving layer 12. The element layer 11 is provided with a circuit (shown later in FIG. 2 and the like) for reading out charges from each of the photoelectric conversion unit 3 and the photoelectric conversion unit 4. The element layer 11 may be provided with a circuit that generates a signal based on charges read from at least one of the photoelectric conversion unit 3 and the photoelectric conversion unit 4. These circuits include, for example, at least one of switching elements such as transistors, various wirings, capacitors, and terminal electronic components. These circuits are formed in a multilayer structure using, for example, an interlayer insulating film and vias. Note that at least a part of the electronic components constituting these circuits may be provided in the light receiving layer 12 or may be provided between the light receiving layer 12 and the lens layer 14.

  The signal processing unit 6 has a first surface 6 a facing the image capturing unit 5 and a second surface 6 b facing the storage unit 7. At least one of the first surface 6a and the second surface 6b is provided with a circuit (not shown) for processing a signal output from the imaging unit 5. Further, at least one of the first surface 6a and the second surface 6b may be provided with a routing wiring used for rewiring and the like. The signal processing unit 6 is provided with a conductive unit that electrically connects the first surface 6a side and the second surface 6b side. This conductive portion includes, for example, TSV (Through-Silicon Via), and can transmit a signal between the first surface 6a and the second surface 6b. The storage unit 7 includes, for example, at least one of a nonvolatile memory and a volatile memory, and stores a signal processed by the signal processing unit 6.

  Note that the signal processing unit 6 may be provided around the pixel region 1 a illustrated in FIG. 1, for example, instead of being stacked with the imaging unit 5. In addition, the imaging element 1 may not include at least a part of the signal processing unit 6 and the storage unit 7. At least a part of the signal processing unit 6 and the storage unit 7 may be provided in a device outside the image sensor 1. The external device may be provided in a device such as a camera or a measurement device on which the image sensor 1 is mounted, for example.

  FIG. 2 is a diagram illustrating a circuit configuration of the image sensor 1 according to the present embodiment. FIG. 2 shows an equivalent circuit diagram for one pixel. The imaging device 1 has a first circuit system that reads a first output signal based on charges accumulated in the photoelectric conversion unit 3 of the pixel P, and a second output signal based on charges accumulated in the photoelectric conversion unit 4 of the pixel P. A second circuit system for reading is included.

  First, the first circuit system will be described. The first circuit system includes a transfer unit 21, a transfer control line 22, a charge voltage conversion unit 23, a reset unit 24, a reset control line 25, a selection unit 26, a selection control line 27, an amplification unit 28, a power supply line 29, and a vertical signal. Includes line 30. The transfer unit 21, the reset unit 24, the selection unit 26, and the amplification unit 28 each include a switching element such as an n-type MOS transistor.

  The photoelectric conversion unit 3 has its anode grounded and its cathode connected to the source of the transfer unit 21. The transfer unit 21 is used to transfer charges from the photoelectric conversion unit 3. The drain of the transfer unit 21 is electrically connected to the node 31. The gate of the transfer unit 21 is connected to the transfer control line 22. The transfer control line 22 can transmit a control signal TX 1 to the gate of the transfer unit 21.

  The charge voltage conversion unit 23 is, for example, a floating diffusion, and converts the charge from the transfer unit 21 into a voltage. The charge-voltage conversion unit 23 includes a capacitor 32. The capacitor 32 has a first electrode grounded and a second electrode connected to the node 31. Note that the capacitor 32 may include at least a part of the capacitor connected to the node 31. The capacitance connected to the node 31 includes, for example, the capacitance of the drain of the transfer unit 21, the coupling capacitance between the wiring connected to the node 31 and another wiring, and the capacitance of the source of the reset unit 24.

  The reset unit 24 resets the voltage converted by the charge-voltage conversion unit 23. The source of the reset unit 24 is connected to the node 31. The drain of the reset unit 24 is connected to a power supply line 29 to which a predetermined potential is supplied with respect to a reference potential (ground potential). The gate of the reset unit 24 is connected to the reset control line 25. The reset control line 25 can transmit a control signal RST1 to the gate of the reset unit 24.

  The selection unit 26 is used to select a pixel P from which a signal is read out among a plurality of pixels P arranged in the vertical scanning line direction shown in FIG. The source of the selection unit 26 is connected to the power supply line 29. The drain of the selection unit 26 is connected to the source of the amplification unit 28. The gate of the selection unit 26 is connected to the selection control line 27. The selection control line 27 can transmit a control signal SEL to the gate of the selection unit 26.

  The amplifying unit 28 amplifies the voltage converted by the charge / voltage converting unit 23. The source of the amplifying unit 28 is connected to the power supply line 29 via the selection unit 26. The power supply voltage VDD can be supplied to the source of the amplification unit 28 via the selection unit 26 and the power supply line 29. The drain of the amplifying unit 28 is connected to the vertical signal line 30. The gate of the amplifying unit 28 is connected to the node 31.

  Next, a second circuit system that reads a second output signal based on the electric charge accumulated in the photoelectric conversion unit 4 of the pixel P will be described. The second circuit system includes a transfer unit 41, a transfer control line 42, a charge voltage conversion unit 43, a reset unit 44, a reset control line 45, a selection unit 46, an amplification unit 47, a selection control line 27, a power supply line 29, and a vertical signal. Includes line 48. The transfer unit 41, the reset unit 44, the selection unit 46, and the amplification unit 47 each include a switching element such as an n-type MOS transistor.

  The photoelectric conversion unit 4 has its anode grounded and its cathode connected to the source of the transfer unit 41. The transfer unit 41 is used to transfer charges from the photoelectric conversion unit 4. The drain of the transfer unit 41 is electrically connected to the node 50. The gate of the transfer unit 41 is connected to the transfer control line 42. The transfer control line 42 is provided separately from the transfer control line 22 and is insulated from the transfer control line 22. The transfer control line 42 can transmit a control signal TX <b> 2 to the gate of the transfer unit 41.

  The charge voltage conversion unit 43 is, for example, a floating diffusion, and converts the charge from the transfer unit 41 into a voltage. The charge voltage conversion unit 43 includes a capacitor 49. The capacitor 49 has a first electrode grounded and a second electrode connected to the node 50. Note that the capacitor 49 may include at least a part of the capacitor connected to the node 50. The capacitance connected to the node 50 includes, for example, the capacitance of the drain of the transfer unit 41, the coupling capacitance between the wiring connected to the node 50 and another wiring, and the capacitance of the source of the reset unit 44.

  The reset unit 44 resets the voltage converted by the charge-voltage conversion unit 43. The source of the reset unit 44 is connected to the node 50. The drain of the reset unit 44 is connected to the power supply line 29. The gate of the reset unit 44 is connected to the reset control line 45. The reset control line 45 can transmit a control signal RST <b> 2 to the gate of the reset unit 44.

  The source of the selection unit 46 is connected to the power supply line 29. The drain of the selection unit 46 is connected to the source of the amplification unit 47. The gate of the selection unit 46 is connected to the selection control line 27. The selection control line 27 can transmit a control signal SEL to the gate of the selection unit 46.

  The amplification unit 47 amplifies the voltage converted by the charge voltage conversion unit 43. The source of the amplifying unit 47 is connected to the power supply line 29 via the selecting unit 46. The power supply voltage VDD can be supplied to the source of the amplifying unit 47 via the selection unit 46 and the power supply line 29. The drain of the amplifying unit 47 is connected to the vertical signal line 30. The gate of the amplifying unit 47 is connected to the node 50.

  Here, the circuit configuration of one pixel among the plurality of pixels P illustrated in FIG. 1 has been described, but the plurality of pixels P have the same circuit configuration. For example, the transfer control lines 22 generally extend in the horizontal scanning direction and are periodically arranged in the vertical scanning direction. The plurality of transfer control lines 22 are respectively connected to the gates of the transfer units 21 of two or more pixels P arranged in the horizontal scanning direction. The transfer control lines 42 extend substantially in the horizontal scanning direction and are periodically arranged in the vertical scanning direction. The plurality of transfer control lines 42 are respectively connected to the gates of the transfer units 41 of two or more pixels P arranged in the horizontal scanning direction.

  The reset control lines 25 extend substantially in the horizontal scanning direction and are periodically arranged in the vertical scanning direction. The plurality of reset control lines 25 are respectively connected to the gates of the reset units 24 of two or more pixels P arranged in the horizontal scanning direction. The reset control lines 45 extend substantially in the horizontal scanning direction and are periodically arranged in the vertical scanning direction. The plurality of reset control lines 45 are respectively connected to the gates of the reset units 44 of two or more pixels P arranged in the horizontal scanning direction.

  The selection control lines 27 extend substantially in the horizontal scanning direction and are periodically arranged in the vertical scanning direction. The plurality of selection control lines 27 are respectively connected to the gates of the selection unit 26 and the selection unit 46 of two or more pixels P arranged in the horizontal scanning direction. The power supply line 29 is connected to the source of the selection unit 26, the source of the selection unit 46, the source of the reset unit 24, and the source of the reset unit 44 of each of the two or more pixels P. Further, the vertical signal lines 30 extend substantially in the vertical scanning direction and are periodically arranged in the horizontal scanning direction. The plurality of vertical signal lines 30 are respectively connected to the drains of the amplification units 47 of two or more pixels P arranged in the vertical scanning direction.

  In the present embodiment, the transfer control line 22 and the transfer control line 42 are provided in the element layer 11 illustrated in FIG. That is, the transfer control line 22 and the transfer control line 42 are disposed on the opposite side of the light incident side with respect to the photoelectric conversion unit 3 and the photoelectric conversion unit 4. Similarly, the reset control line 25, the reset control line 45, the selection control line 27, the power supply line 29, and the vertical signal line 30 are opposite to the light incident side with respect to the photoelectric conversion unit 3 and the photoelectric conversion unit 4. Has been placed.

  Note that at least a part of the transfer control line 22, the transfer control line 42, the reset control line 25, the reset control line 45, the selection control line 27, the power supply line 29, and the vertical signal line 30 are the photoelectric conversion unit 3 and the photoelectric conversion unit. 4 may be disposed on the same side as the light incident side. At least a part of these wirings may be provided in the light receiving layer 12 shown in FIG. 1C or may be provided between the light receiving layer 12 and the lens layer 14.

  The imaging device 1 is provided with a vertical scanning circuit 51, a horizontal scanning circuit 52, a constant current source 53, and a constant current source 54. The vertical scanning circuit 51 supplies a control signal for reading a signal based on the charge from each of the plurality of pixels P.

  First, an example of an operation for reading the first output signal based on the electric charge accumulated in the photoelectric conversion unit 3 will be described. The vertical scanning circuit 51 supplies a control signal SEL to the selection control line 27. When the vertical scanning circuit 51 sets the control signal SEL for the selection control line 27 to H level (high level), between the source and drain of the selection unit 26 and the source of the selection unit 46 connected to the selection control line 27. A state in which current can be passed between the drains (hereinafter referred to as an on state) is established. As a result, the power supply voltage VDD is supplied to the source of the amplification unit 28 via the power supply line 29 and the selection unit 26.

  When the selection unit 26 reads charges from the pixels P in other rows, the control signal SEL for the selection control line 27 is set to the L level (low level), so that the source of the selection unit 26 A state where current cannot be passed between the drains (hereinafter referred to as an off state) is established. In addition, the vertical scanning circuit 51 can turn off the selection unit 26 during operation of a circuit (for example, an amplifier 55 described later) arranged on the downstream side of signal transmission from the vertical signal line 30. In this case, transmission of noise from the vertical signal line 30 to the amplification unit 28 can be suppressed.

  Further, the vertical scanning circuit 51 supplies a control signal TX1 to the transfer control line 22. When the vertical scanning circuit 51 sets the control signal TX1 for the transfer control line 22 to the H level, the transfer unit 21 connected to the transfer control line 22 is turned on, and the charge accumulated in the photoelectric conversion unit 3 is charged. It is transferred to the voltage converter 23. The capacitor 32 is charged by the charge transferred from the photoelectric conversion unit 3 via the transfer unit 21, and the voltage between the first electrode and the second electrode becomes a voltage corresponding to the charge. This voltage is applied to the gate of the amplifying unit 28 via the node 31, and the resistance value between the source and drain of the amplifying unit 28 becomes a value corresponding to the voltage applied to the gate of the amplifying unit 28. Further, between the source and the drain of the amplifying unit 28, a predetermined voltage applied to the source via the power line 29 and the selection unit 26 and a resistance value between the source and the drain of the amplifying unit 28 are determined. Current (signal) flows. This signal is output to the vertical scanning circuit 51 via the vertical signal line 30.

  Further, the vertical scanning circuit 51 supplies a control signal RST1 to the reset control line 25. When the vertical scanning circuit 51 sets the control signal RST1 for the reset control line 25 to the H level, the reset unit 24 connected to the reset control line 25 is turned on and stored in the capacitor 32 of the charge-voltage conversion unit 23. The charges are discharged through the reset unit 24 and the power supply line 29.

  Next, an example of an operation for reading the second output signal based on the electric charge accumulated in the photoelectric conversion unit 4 will be described. When the vertical scanning circuit 51 sets the control signal TX1 for the transfer control line 22 to the H level, the power supply voltage VDD is supplied to the source of the amplification unit 47 via the power supply line 29 and the selection unit 46. The vertical scanning circuit 51 supplies a control signal TX2 to the transfer control line 42. When the vertical scanning circuit 51 sets the control signal TX2 for the transfer control line 42 to the H level, the transfer unit 41 connected to the transfer control line 42 is turned on, and the charge accumulated in the photoelectric conversion unit 4 is charged. It is transferred to the voltage converter 43.

  The capacitor 49 is charged by the charge transferred from the photoelectric conversion unit 4 via the transfer unit 41, and the voltage between the first electrode and the second electrode becomes a voltage corresponding to the charge. This voltage is applied to the gate of the amplifying unit 47 via the node 50, and the resistance value between the source and drain of the amplifying unit 47 becomes a value corresponding to the voltage applied to the gate of the amplifying unit 47. Further, between the source and the drain of the amplifying unit 47, a predetermined voltage applied to the source via the power supply line 29 and the selecting unit 46 and a resistance value between the source and the drain of the amplifying unit 47 are determined. Current (signal) flows. This signal is output to the vertical scanning circuit 51 via the vertical signal line 48.

  Further, the vertical scanning circuit 51 supplies a control signal RST2 to the reset control line 45. When the vertical scanning circuit 51 sets the control signal RST2 for the reset control line 45 to the H level, the reset unit 44 connected to the reset control line 45 is turned on and stored in the capacitor 49 of the charge voltage conversion unit 43. The electric charges are discharged through the reset unit 44 and the power supply line 29.

  In the present embodiment, a constant current source 53 is connected to the vertical signal line 30, and a constant current source 54 is connected to the vertical signal line 48. When the selection unit 26 is in the ON state, the amplification unit 28, the selection unit 26, and the constant current source 53 connected to the vertical signal line 30 constitute a source follower circuit. In this case, the signal of the pixel P belonging to the row selected by the selection unit 26 is output to the vertical signal line 30. Similarly, when the selection unit 46 is in the on state, the amplification unit 47, the selection unit 46, and the constant current source 54 connected to the vertical signal line 48 constitute a source follower circuit.

  The horizontal scanning circuit 52 converts a signal output from each of the plurality of pixels P into image data of a predetermined format. The horizontal scanning circuit 52 reduces the noise of the signal output from the pixel P by correlated double sampling.

  The horizontal scanning circuit 52 includes an amplifier 55, an amplifier 56, and an analog-digital converter (not shown). The amplifier 55 is an amplifying unit that amplifies a signal corresponding to the output from the photoelectric conversion unit 3. The amplifier 55 is a column amplifier, for example, and is provided for each vertical signal line 30. The amplifier 55 amplifies a signal output via the vertical signal line 30. The amplifier 56 is an amplifying unit that amplifies a signal corresponding to the output from the photoelectric conversion unit 4. The amplifier 56 is a column amplifier, for example, and is provided for each vertical signal line 48. The amplifier 56 amplifies a signal output via the vertical signal line 48.

  In the present embodiment, the amplification factor (gain) of the amplifier 55 and the amplification factor of the amplifier 56 are variable. The amplification factor of the amplifier 55 can be set to the same value as or different from the amplification factor of the amplifier 56. The amplification factor of each of the amplifier 55 and the amplifier 56 can be set to, for example, about several tenths to several tens of times. When adjusting the amplification factors of the amplifier 55 and the amplifier 56, the amplification factor of the amplifier 55 and the amplification factor of the amplifier 56 may be adjusted in conjunction with each other, or the amplification factor of the amplifier 55 and the amplification factor of the amplifier 56 may be adjusted. May be adjusted independently. For example, the ratio between the amplification factor of the amplifier 55 and the amplification factor of the amplifier 56 may be constant or may be fixed.

  The analog-digital converter of the horizontal scanning circuit 52 converts the analog signal output from each of the amplifier 55 and the amplifier 56 into a digital signal. This digital signal includes, for example, a gradation value (pixel value) in which the output of each pixel P is represented by 8 bits. The horizontal scanning circuit 52 arranges pixel values representing the output of each pixel P in accordance with the pixel arrangement and outputs it in a predetermined data format. The horizontal scanning circuit 52 may include at least a part of the constant current source 53 and the constant current source 54.

  Next, the timing for reading out charges from the photoelectric conversion unit will be described. FIG. 3 is a timing chart showing an example of the operation of the image sensor 1.

  The image sensor 1 performs dummy reading from the start up to the first imaging. As shown in FIG. 3, the vertical scanning circuit 51 sets the control signal RST1 for the reset control line 25 to the H level at the time t1 before the first frame period, and the control signal RST2 for the reset control line 45. Is set to H level. As a result, the voltage between the electrodes in the capacitor 32 of the charge-voltage converter 23 and the voltage between the electrodes in the capacitor 49 of the charge-voltage converter 43 are reset. The vertical scanning circuit 51 sets the control signal RST1 and the control signal RST2 to L level (low level) at time t2. The vertical scanning circuit 51 sets the control signal TX1 for the transfer control line 22 to H level and sets the control signal TX2 for the transfer control line 42 to H level at time t3. Thereby, when charges are accumulated in the photoelectric conversion unit 3, the charges are transferred to the charge-voltage conversion unit 23. Further, when charges are accumulated in the photoelectric conversion unit 4, the charges are transferred to the charge / voltage conversion unit 43. The vertical scanning circuit 51 sets the control signal TX1 and the control signal TX2 to L level at time t4.

  Here, dummy reading from the photoelectric conversion unit 3 is performed in parallel with dummy reading from the photoelectric conversion unit 4, but the dummy reading from the photoelectric conversion unit 3 and the dummy reading from the photoelectric conversion unit 4 do not overlap. You may go to For example, the vertical scanning circuit 51 may set the period during which the control signal RST1 is at the H level and the period during which the control signal RST2 is at the H level so that at least a part thereof overlaps or does not overlap. May be. Further, the vertical scanning circuit 51 may set at least a part of the period during which the control signal TX1 is at the H level and the period at which the control signal TX2 is at the H level, or may be set so as not to overlap. May be.

  The imaging device 1 starts imaging processing of the first frame at time t5. The vertical scanning circuit 51 maintains the control signal SEL for the selection unit 26 at the H level. Further, the vertical scanning circuit 51 performs a reset operation on the photoelectric conversion unit 3 during a period from time t6 to time t7. During this period, the vertical scanning circuit 51 sets the control signal RST1 for the reset control line 25 to the H level after setting it to the H level. Next, the vertical scanning circuit 51 sets the control signal TX1 for the transfer unit 21 to H level and then to L level. Here, the vertical scanning circuit 51 sets each of the control signal RST1 and the control signal TX1 into three pulses, but the number of pulses may be one time, two times, or four times or more. After the control signal TX1 is set to the L level at time t7, charges corresponding to the exposure amount are accumulated in the photoelectric conversion unit 3.

  The vertical scanning circuit 51 reads out charges from the photoelectric conversion unit 3 during a period from time t8 to time t10. The vertical scanning circuit 51 sets the control signal RST1 to H level at time t8 and then sets it to L level. The vertical scanning circuit 51 sets the control signal TX1 to H level at time t9 after time t8, and sets the control signal TX1 to L level at time t10. Exposure to the photoelectric conversion unit 3 in the period of the first frame ends at time t9. The photoelectric conversion unit 3 outputs the charge accumulated during the exposure period from time t7 to time t9 to the charge-voltage conversion unit 23 via the transfer unit 21. The amplifying unit 28 outputs a signal corresponding to the voltage converted by the charge / voltage converting unit 23 to the vertical signal line 30.

  In the present embodiment, the image sensor 1 exposes the photoelectric conversion unit 4 during a period that overlaps at least a part of the exposure period for the photoelectric conversion unit 3. The vertical scanning circuit 51 performs a reset operation on the photoelectric conversion unit 4 during a period from time t11 to time t12 after the time t7 in the period of the first frame. During this period, the vertical scanning circuit 51 sets the control signal RST2 for the reset control line 45 to the H level after setting it to the H level. Next, the vertical scanning circuit 51 sets the control signal TX2 for the transfer unit 41 to H level and then to L level. Here, the vertical scanning circuit 51 sets each of the control signal RST2 and the control signal TX2 in three pulses, but the number of pulses may be one time, two times, or four times or more. After the control signal TX2 is set to the L level at time t12, charges corresponding to the exposure amount are accumulated in the photoelectric conversion unit 4.

  The vertical scanning circuit 51 reads out charges from the photoelectric conversion unit 4 when the exposure period of the photoelectric conversion unit 4 ends. Here, the vertical scanning circuit 51 reads out the charge from the photoelectric conversion unit 4 in parallel with the reading of the charge from the photoelectric conversion unit 3. The vertical scanning circuit 51 sets the control signal RST2 to H level at time t8 and then sets it to L level. The vertical scanning circuit 51 sets the control signal TX2 to H level at time t9, and sets the control signal TX2 to L level at time t10. The photoelectric conversion unit 4 outputs the charge accumulated during the exposure period from time t12 to time t9 to the charge-voltage conversion unit 43 via the transfer unit 41. The amplifier 47 outputs a signal corresponding to the voltage converted by the charge-voltage converter 43 to the vertical signal line 48.

  As described above, at the time t10, the period of the first frame ends and the period of the next second frame starts. In FIG. 3, the same operation as in the first frame period is repeated in the second frame period.

  In the example of FIG. 3, charges are accumulated in the photoelectric conversion unit 4 during a period in which charges are accumulated in the photoelectric conversion unit 3 (exposure period from time t7 to time t9) in the period of the first frame. It is set longer than the period (exposure period from time t12 to time t9).

  As described above, the imaging device 1 includes the first output signal based on the charge accumulated in the photoelectric conversion unit 3 and the photoelectric conversion unit, with the exposure period length being different between the photoelectric conversion unit 3 and the photoelectric conversion unit 4. 4 can generate a second output signal based on the electric charge accumulated in 4. By using such first output signal and second output signal, two images having different exposure periods can be generated. Thereby, for example, an image having a wide dynamic range can be generated. For example, when the exposure period of the first output signal is longer than the exposure period of the second output signal, the first output signal is used to generate an image of a relatively dark part of the imaging target, and the second output signal is used. Thus, an image of a brighter bright part of the imaging target is generated. By synthesizing these images, the dark part can be brightly expressed while suppressing the saturation of the bright part.

  By the way, in order to widen the dynamic range, there is a method of alternately repeating a frame having a long exposure time and a frame having a short exposure time by using an image sensor having one photoelectric conversion unit arranged in one pixel. . In this method, two frames are used to obtain an image of one frame, and the frame rate becomes slow. As another method, there is a method using a signal from an image with a short exposure time and a signal with a long exposure time set. In this method, since one pixel of an image is formed, a signal of two pixels is used, and the resolution is lowered.

  In the present embodiment, since a plurality of photoelectric conversion units (photoelectric conversion unit 3 and photoelectric conversion unit 4) are arranged in one pixel, the dynamic range can be expanded without reducing the resolution of the captured image. Further, the transfer control line 22 that transmits the control signal TX1 to the transfer unit 21 for transferring the charge from the photoelectric conversion unit 3 transfers the control signal TX2 to the transfer unit 41 for transferring the charge from the photoelectric conversion unit 4. Since it is provided separately from the line 42, exposure to the photoelectric conversion unit 3 and exposure to the photoelectric conversion unit 4 can be performed in parallel, and a decrease in frame rate can be suppressed. In such a case, when the transfer control line 22 and the transfer control line 42 are arranged on the opposite side of the light incident side with respect to the photoelectric conversion unit 3 and the photoelectric conversion unit 4, the light reception of the photoelectric conversion unit 3 and the photoelectric conversion unit 4. It can avoid that an area becomes narrow. The dynamic range can be expanded by adjusting the ratio between the amplification factor of the amplifier 55 and the amplification factor of the amplifier 56.

  Incidentally, in FIG. 3, the time t9 at which the exposure to the photoelectric conversion unit 4 ends is set to be substantially the same as the time t9 at which the exposure to the photoelectric conversion unit 3 ends. For this reason, it is possible to avoid the complicated control of reading signals from the pixels P.

  FIG. 4 is a timing chart showing another example of the operation of the image sensor 1. FIGS. 4A to 4D each show a timing chart for a period of one frame. In each of FIGS. 4A to 4D, the operation for reading out the charges from the photoelectric conversion unit 3 is the same as the example shown in FIG. Turn into.

  In FIG. 4A, the time at which the exposure period for the photoelectric conversion unit 4 ends is set to a time different from the time at which the exposure period for the photoelectric conversion unit 3 ends. The vertical scanning circuit 51 performs resetting of the photoelectric conversion unit 3 and resetting of the photoelectric conversion unit 4 in parallel during a period from time t6 to time t7. At time t7, exposure to the photoelectric conversion unit 3 starts and exposure to the photoelectric conversion unit 4 starts.

  At time t15, the vertical scanning circuit 51 sets the control signal RST2 for the reset unit 44 to H level and then to L level. Further, at time t16, the vertical scanning circuit 51 sets the control signal TX2 for the transfer unit 41 to H level and then sets it to L level. The exposure period (charge accumulation period) for the photoelectric conversion unit 4 in the period of the first frame is a period from time t7 to time t16.

  4B, the time when the exposure period for the photoelectric conversion unit 4 starts is set to a time different from the time when the exposure period for the photoelectric conversion unit 3 starts. The time when the exposure period for the photoelectric conversion unit 4 ends is set to a time different from the time when the exposure period for the photoelectric conversion unit 3 ends. The vertical scanning circuit 51 resets the photoelectric conversion unit 4 during a period from time t17 to time t18 after time t6 when resetting the photoelectric conversion unit 3 is started. The exposure period for the photoelectric conversion unit 4 starts at time t18.

  At time t19, the vertical scanning circuit 51 sets the control signal RST2 for the reset unit 44 to H level and then to L level. Further, at time t20, the vertical scanning circuit 51 sets the control signal TX2 for the transfer unit 41 to H level and then sets it to L level. The exposure period (charge accumulation period) for the photoelectric conversion unit 4 in the first frame period is a period from time t18 to time t20. Here, the exposure period for the photoelectric conversion unit 4 is set so that the center time thereof coincides with the center time of the exposure period for the photoelectric conversion unit 3.

  In FIG. 4C, a plurality of exposure periods for the photoelectric conversion unit 4 are set. The vertical scanning circuit 51 resets the photoelectric conversion unit 4 during the period from time t6 to time t7. The first exposure period for the photoelectric conversion unit 4 starts at time t7. At time t21, the vertical scanning circuit 51 sets the control signal RST2 for the reset unit 44 to the H level after setting it to the H level. Further, at time t22, the vertical scanning circuit 51 sets the control signal TX2 for the transfer unit 41 to H level and then sets it to L level. The first exposure period for the photoelectric conversion unit 4 in the period of the first frame is a period from time t7 to time t22. Next, the vertical scanning circuit 51 resets the photoelectric conversion unit 4 during a period from time t23 to time t24. The second exposure period for the photoelectric conversion unit 4 starts at time t24. At time t8, the vertical scanning circuit 51 sets the control signal RST2 for the reset unit 44 to H level and then to L level. Further, at time t9, the vertical scanning circuit 51 sets the control signal TX2 for the transfer unit 41 to H level and then sets it to L level. The second exposure period for the photoelectric conversion unit 4 in the period of the first frame is a period from time t24 to time t9.

  The length of the first exposure period for the photoelectric conversion unit 4 in one frame may be set to the same length as the second exposure period, or set to a different length from the second exposure period. It may be. As described above, when a plurality of exposures are performed during one frame period, the signal based on the charge accumulated in the photoelectric conversion unit 4 in the first exposure period and the photoelectric conversion unit 4 accumulated in the second exposure period. For example, an arithmetic operation such as averaging of these signals may be performed using signals based on the charges. Further, when the length of the first exposure period and the length of the second exposure period for the photoelectric conversion unit 4 are different, a signal based on the charge accumulated in the photoelectric conversion unit 4 in the first exposure period, The dynamic range can be further expanded by using a signal based on the charge accumulated in the photoelectric conversion unit 4 during the exposure period and a signal based on the charge accumulated in the photoelectric conversion unit 3 during the frame period.

  In FIG. 4D, the exposure period for the photoelectric conversion unit 4 is set to the same length as the exposure period for the photoelectric conversion unit 3. Here, the time t7 at which the exposure period for the photoelectric conversion unit 4 starts is set to be the same as the time t7 at which the exposure period for the photoelectric conversion unit 3 starts. The time t9 when the exposure period for the photoelectric conversion unit 4 ends is set to be the same as the time t9 when the exposure period for the photoelectric conversion unit 3 ends.

  When the photoelectric conversion unit 3 and the photoelectric conversion unit 4 have the same exposure period, the first output signal based on the charge accumulated in the photoelectric conversion unit 3 during the first frame period; Information related to focusing in the image sensor 1 may be detected using the second output signal based on the charge accumulated in the photoelectric conversion unit 4 during the period of one frame. For example, when the levels of the first output signal and the second output signal are equal, it may be determined that the light receiving layer 12 of the image sensor 1 is in focus. Further, when one level of the first output signal and the second output signal is higher than the other level, it may be determined that the focal point is on the object plane side with respect to the light receiving layer 12 (front pin). . When one level of the first output signal and the second output signal is lower than the other level, it may be determined that the light receiving layer 12 is on the object plane side (rear pin) with respect to the focal point. .

  When the exposure periods of the photoelectric conversion unit 3 and the photoelectric conversion unit 4 are set to different lengths, information relating to focusing on the image sensor 1 is detected using the first output signal and the second output signal. May be. In this case, using the estimated value of the signal level when the lengths of the exposure periods are aligned, the exposure period is set to the same length in the photoelectric conversion unit 3 and the photoelectric conversion unit 4. Thus, it is possible to detect information related to focusing. The estimated value of the level of the signal when the length of the exposure period is aligned is, for example, the level of the first output signal when the length of the exposure period for the photoelectric conversion unit 4 is half of the exposure period for the photoelectric conversion unit 3 Is halved, or the level of the second output signal is doubled. In order to adjust the level of the first output signal, the amplification factor of the amplifier 55 shown in FIG. 2 may be adjusted. To adjust the level of the second output signal, the amplification factor of the amplifier 56 is adjusted. May be. In order to adjust at least one of the level of the first output signal and the level of the second output signal, arithmetic processing may be performed by the horizontal scanning circuit 52 shown in FIG. 2 or an external device.

  By the way, in this embodiment, when the some photoelectric conversion part (the photoelectric conversion part 3 and the photoelectric conversion part 4) is arrange | positioned in 1 pixel, it has 1 light receiving area equal to the total light receiving area of several photoelectric conversion parts. Compared with the case of using two photoelectric conversion units, the output per photoelectric conversion unit is small. Therefore, when the level of the first output signal or the second output is insufficient, the level of the first output signal or the second output signal may be adjusted by adjusting the amplification factor of the amplifier 55 or the amplifier 56. . For example, when the amplification factor of the amplifier 55 is set to double, when one photoelectric conversion unit having a light receiving area equal to the total light receiving area of the photoelectric conversion unit 3 and the photoelectric conversion unit 4 is used as the level of the first output signal Can be at the same level.

  Note that the example shown in FIG. 3 and the examples shown in FIGS. 4A to 4D can be combined. For example, the imaging device 1 includes a first mode (see FIG. 4D) in which the exposure time of the photoelectric conversion unit 3 and the exposure time of the photoelectric conversion unit 4 are the same length, the exposure time of the photoelectric conversion unit 3 and the photoelectric conversion unit 3. The second mode can be switched to a different length for the exposure time of the conversion unit 4. In the first mode, for example, the vertical scanning circuit 51 sets the waveform of the control signal TX1 for the transfer unit 21 and the waveform of the control signal TX2 for the transfer unit 41 to be the same. In the second mode, for example, the vertical scanning circuit 51 sets the waveform of the control signal TX1 for the transfer unit 21 and the waveform of the control signal TX2 for the transfer unit 41 to different waveforms. The second mode may include one of the examples illustrated in FIG. 3 and the examples illustrated in FIGS. 4A to 4C, or may include a combination of two or more. .

  In the above example, the length of the exposure period for the photoelectric conversion unit 3 is set to be equal to or longer than the length of the exposure period for the photoelectric conversion unit 4, but less than the length of the exposure period for the photoelectric conversion unit 4. It may be set. Of the exposure time of the photoelectric conversion unit 3 and the exposure time of the photoelectric conversion unit 4, one exposure time may be set to 1/10 times or more and 1 time or less of the other exposure time, or 1/10 times It may be set to less than.

  Next, another example of a plurality of photoelectric conversion units arranged in one pixel will be described. FIG. 5 is a diagram illustrating another example of a plurality of photoelectric conversion units.

  5A, the photoelectric conversion unit 3 is provided asymmetrically with the photoelectric conversion unit 4 when viewed from the normal direction of the pixel region 1a. The light receiving area of the photoelectric conversion unit 3 is set larger than the light receiving area of the photoelectric conversion unit 4. When the light receiving areas are different between the photoelectric conversion unit 3 and the photoelectric conversion unit 4, the dynamic range can be further expanded. Thus, the photoelectric conversion unit 3 may be different from the photoelectric conversion unit 4 in at least one of shape and size.

  5B, the photoelectric conversion unit 3, the photoelectric conversion unit 4, and the photoelectric conversion unit 60 are provided in one pixel. The light receiving area of the photoelectric conversion unit 3 is set larger than the light receiving area of the photoelectric conversion unit 4. The light receiving area of the photoelectric conversion unit 60 is set to be substantially the same as the light receiving area of the photoelectric conversion unit 4. The photoelectric conversion unit 60 is provided symmetrically with the photoelectric conversion unit 4 with respect to the center line of the pixel P. As described above, the number of photoelectric conversion units arranged in one pixel may be three, or four or more. Further, among the plurality of photoelectric conversion units, at least one of the shape and size of any one of the photoelectric conversion units may be the same as or different from the other photoelectric conversion units. .

  When such an image sensor 1 is used, for example, imaging is performed using the second output signal based on the charge accumulated in the photoelectric conversion unit 4 and the third output signal based on the charge accumulated in the photoelectric conversion unit 60. Information on focusing in the element 1 can also be detected. In addition, the dynamic range can be expanded by using at least two of the first output signal, the second output signal, and the third output signal based on the charges accumulated in the photoelectric conversion unit 3.

[Second Embodiment]
Next, a second embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are denoted by the same reference numerals, and the description thereof is simplified or omitted.

  FIG. 6 is a diagram illustrating a circuit configuration of the image sensor 1 according to the present embodiment. FIG. 6 shows an equivalent circuit diagram for two pixels. The circuit configuration of the pixel P1 is the same as that in the above-described embodiment (see FIG. 2). The pixel P2 is a pixel arranged next to the pixel P1. For example, the pixel P is the red pixel R in FIG. 1B, and the pixel P2 is the green pixel Gb.

  In the pixel P2, the photoelectric conversion unit 61 and the photoelectric conversion unit 62 are provided in the pixel P2. The photoelectric conversion unit 61 and the photoelectric conversion unit 62 include, for example, a photodiode. However, the photoelectric conversion unit 61 and the photoelectric conversion unit 62 only need to convert light into electric charge, and may include a phototransistor, for example. The imaging device 1 has a third circuit system that reads an output signal based on the charge accumulated in the photoelectric conversion unit 61 of the pixel P2, and a fourth circuit that reads an output signal based on the charge accumulated in the photoelectric conversion unit 62 of the pixel P2. Includes systems.

  First, the third circuit system will be described. The third circuit system includes a transfer unit 63, a transfer control line 64, a charge voltage conversion unit 23, a reset unit 24, a reset control line 25, a selection unit 26, a selection control line 27, an amplification unit 28, a power supply line 29, and a vertical signal. Includes line 30. That is, the third circuit system is shared with the first circuit system except for the transfer unit 63 and the transfer control line 64. The transfer unit 63 includes a switching element such as an n-type MOS transistor, for example.

  The photoelectric conversion unit 61 has its anode grounded and its cathode connected to the source of the transfer unit 63. The transfer unit 63 is used to transfer charges from the photoelectric conversion unit 61. The drain of the transfer unit 63 is electrically connected to the node 31. The gate of the transfer unit 63 is connected to the transfer control line 64. The transfer control line 64 can transmit a control signal TX3 to the gate of the transfer unit 63. The charge-voltage conversion unit 23 converts the charge from the transfer unit 21 into a voltage, and converts the charge from the transfer unit 63 into a voltage.

  Next, the fourth circuit system will be described. The fourth circuit system includes a transfer unit 65, a transfer control line 66, a charge voltage conversion unit 43, a reset unit 44, a reset control line 45, a selection unit 46, a selection control line 27, an amplification unit 47, a power supply line 29, and a vertical signal. Includes line 30. That is, the fourth circuit system is shared with the second circuit system except for the transfer unit 65 and the transfer control line 66. The transfer unit 65 includes a switching element such as an n-type MOS transistor, for example.

  The photoelectric conversion unit 62 has its anode grounded and its cathode connected to the source of the transfer unit 65. The transfer unit 65 is used to transfer charges from the photoelectric conversion unit 62. The drain of the transfer unit 65 is electrically connected to the node 50. The gate of the transfer unit 65 is connected to the transfer control line 66. The transfer control line 66 can transmit a control signal TX4 to the gate of the transfer unit 65. The charge-voltage converter 43 converts the charge from the transfer unit 41 into a voltage, and converts the charge from the transfer unit 65 into a voltage.

  In this embodiment, the transfer control line 64 and the transfer control line 66 are provided in the element layer 11 shown in FIG. That is, the transfer control line 64 and the transfer control line 66 are disposed on the opposite side of the light incident side with respect to the photoelectric conversion unit 61 and the photoelectric conversion unit 62. Note that at least part of the transfer control line 64 and the transfer control line 66 may be disposed on the same side as the light incident side with respect to the photoelectric conversion unit 61 and the photoelectric conversion unit 62. At least a part of these wirings may be provided in the light receiving layer 12 shown in FIG. 1C or may be provided between the light receiving layer 12 and the lens layer 14.

  Next, the timing for reading out charges from the photoelectric conversion unit will be described. FIG. 7 is a timing chart showing an example of the operation of the image sensor 1.

  The image sensor 1 performs dummy readout in the same manner as in the first embodiment after the activation until the first imaging. Further, the image sensor 1 reads out signals from the pixels P in the period of the first frame as in the first embodiment. The image sensor 1 reads out a signal from the pixel P2 in a second frame period following the first frame period.

  The imaging element 1 starts the imaging process of the second frame at time t10. The vertical scanning circuit 51 maintains the control signal SEL for the selection unit 26 at the H level. Further, the vertical scanning circuit 51 performs a reset operation on the photoelectric conversion unit 61 during a period from time t30 to time t31. During this period, the vertical scanning circuit 51 sets the control signal RST1 for the reset control line 25 to the H level after setting it to the H level. Next, the vertical scanning circuit 51 sets the control signal TX3 for the transfer unit 21 to H level and then to L level. After the control signal TX3 is set to L level at time t31, charges corresponding to the exposure amount are accumulated in the photoelectric conversion unit 61.

  The vertical scanning circuit 51 reads out charges from the photoelectric conversion unit 61 during a period from time t32 to time t33. The vertical scanning circuit 51 sets the control signal RST1 to H level at time t32 and then sets it to L level. The vertical scanning circuit 51 sets the control signal TX3 to H level at time t33 after time t32, and sets the control signal TX3 to L level at time t34. Exposure to the photoelectric conversion unit 3 in the period of the second frame ends at time t33. The photoelectric conversion unit 61 outputs the charge accumulated during the exposure period from time t31 to time t33 to the charge-voltage conversion unit 23 via the transfer unit 63. The amplifying unit 28 outputs a signal corresponding to the voltage converted by the charge / voltage converting unit 23 to the vertical signal line 30.

  In the present embodiment, the imaging device 1 can perform exposure on the photoelectric conversion unit 62 in a period that overlaps at least a part of the exposure period on the photoelectric conversion unit 61. The vertical scanning circuit 51 performs a reset operation on the photoelectric conversion unit 62 during a period from time t35 to time t36 after the time t31 in the period of the second frame. During this period, the vertical scanning circuit 51 sets the control signal RST2 for the reset control line 45 to the H level after setting it to the H level. Next, the vertical scanning circuit 51 sets the control signal TX4 for the transfer unit 65 to H level and then to L level. After the control signal TX4 is set to the L level at time t36, charges corresponding to the exposure amount are accumulated in the photoelectric conversion unit 62.

  The vertical scanning circuit 51 reads out charges from the photoelectric conversion unit 62 when the exposure period of the photoelectric conversion unit 62 ends. Here, the vertical scanning circuit 51 reads out charges from the photoelectric conversion unit 62 in parallel with reading out charges from the photoelectric conversion unit 61. The vertical scanning circuit 51 sets the control signal RST2 to H level at time t32 and then sets it to L level. Further, the vertical scanning circuit 51 sets the control signal TX4 to H level at time t33, and sets the control signal TX4 to L level at time t34. The photoelectric conversion unit 62 outputs the charge accumulated during the exposure period from time t36 to time t33 to the charge-voltage conversion unit 43 via the transfer unit 65. The amplifier 47 outputs a signal corresponding to the voltage converted by the charge-voltage converter 43 to the vertical signal line 48. In this way, at the time t34, the period of the second frame ends and the period of the next frame starts. In the next frame period after the second frame period, exposure to the pixel P1 and readout of the signal from the pixel P1 are performed as in the first frame period.

  In the above-described embodiment, the imaging device 1 can detect light in three wavelength bands, but may detect only light in one wavelength band (monochromatic light), or two Only light in the wavelength band or light in four or more wavelength bands may be detected. Further, the pixel array need not be a Bayer array and can be arbitrarily selected. Further, the imaging device 1 may have sensitivity in the invisible light region, and may detect, for example, at least one of infrared light and ultraviolet light.

  Note that the selection unit 26 and the selection control line 27 shown in FIGS. 2 and 6 can be omitted. When the selection unit 26 and the selection control line 727 are not provided, the timing for reading out charges from the pixel P can be controlled by the timing at which the transfer unit 21, the transfer unit 41, and the like are turned on.

[Third Embodiment]
Next, an image pickup apparatus using the image pickup device 1 described in the above embodiment will be described. FIG. 8 is a block diagram illustrating a configuration of the imaging device 70 according to the present embodiment. The imaging device 70 illustrated in FIG. 8 is at least a part of an electronic device such as a digital camera, a smartphone, a mobile phone, a personal computer, or a measurement device. The imaging device 70 includes a lens unit 71, a lens driving unit 72, an imaging device 1, a control unit 73, a work memory 74, a display unit 75, an operation unit 76, and a recording unit 77.

  The lens unit 71 includes, for example, an imaging optical system including a plurality of lenses and a lens barrel. The imaging optical system of the lens unit 71 collects light from the subject and guides it to the imaging device 1. The imaging optical system of the lens unit 71 forms an image of the subject on the light receiving layer 12 of the imaging device 1 (see FIG. 1C). The lens unit 71 may be integrated with a body that houses at least the control unit 73 in the imaging device 70, or may be an interchangeable lens that can be attached to and detached from the body. The lens unit 71 may include a focus lens or may include a zoom lens. The lens driving unit 72 drives the lens of the lens unit 71 and adjusts the focal point of the lens unit 71.

  The control unit 73 controls each unit of the imaging device 70. The control unit 73 includes an imaging control unit 80, an image processing unit 81, and a focus detection unit 82.

  The imaging control unit 80 causes the imaging device 1 to execute an imaging process by sending a command to the imaging device 1. For example, the imaging control unit 80 controls the timing at which exposure to each photoelectric conversion unit is started and the timing at which exposure is ended by sending a command to the vertical scanning circuit 51 of the imaging device 1 shown in FIG. Further, the imaging control unit 80 controls the timing of reading a signal from each pixel to the horizontal scanning circuit 52 by sending a command to the vertical scanning circuit 51. In addition, the imaging control unit 80 outputs (transmits) the imaging result to the control unit 73 by sending a command to the imaging element 1. For example, the imaging control unit 80 outputs an imaging result from the horizontal scanning circuit 52 to the control unit 73 by sending a command to the horizontal scanning circuit 52.

  The image processing unit 81 acquires the imaging result from the imaging device 1 and executes various image processing. The image processing unit 81 uses the work memory 74 as a work space to perform image processing on RAW data composed of pixel signals of each pixel to generate image data. The work memory 74 temporarily stores image data and the like when image processing by the image processing unit 81 is performed.

  In the present embodiment, the image processing unit 81 uses the first output signal based on the charge accumulated in the photoelectric conversion unit 3 and the second output signal based on the charge accumulated in the photoelectric conversion unit 4. Thus, the HDR process for expanding the dynamic range is performed. In this case, the imaging control unit 80 includes a period (exposure period) in which charges are accumulated in the photoelectric conversion unit 3 and a period (exposure period) in which charges are accumulated in the photoelectric conversion unit 4 in one frame period. Set different lengths.

  For example, as illustrated in FIG. 3, the imaging control unit 80 sets the exposure period for the photoelectric conversion unit 3 to be longer than the exposure period for the photoelectric conversion unit 4. Then, the image processing unit 81 uses the first output signal having a long exposure period (a large amount of exposure) to generate an image of a darker dark part of the imaging target. In addition, the image processing unit 81 generates an image of a brighter bright portion of the imaging target using the second output signal having a short exposure period (small exposure amount). The image processing unit 81 synthesizes the dark part image and the bright part image, thereby generating an image representing the dark part brightly while suppressing saturation of the bright part.

  Note that the imaging device 70 can switch between a mode in which image data is generated by performing HDR processing and a mode in which image data is generated without performing HDR processing. When HDR processing is not performed, the imaging control unit 80 sets the period during which charges are accumulated in the photoelectric conversion unit 3 and the period during which charges are accumulated in the photoelectric conversion unit 4 to different lengths. To do. For example, the image processing unit 81 adds the first output signal based on the charge accumulated in the photoelectric conversion unit 3 and the second output signal based on the charge accumulated in the photoelectric conversion unit 4 to generate image data. To do.

  The image processing unit 81 can execute various image processing in addition to the above-described HDR processing. For example, the image processing unit 81 generates an RGB image signal by performing color signal processing (color tone correction) on a signal obtained by the Bayer array. The image processing unit 81 performs image processing such as white balance adjustment, sharpness adjustment, gamma correction, and gradation adjustment on the RGB image signal. Further, the image processing unit 81 performs a process of compressing in a predetermined compression format (JPEG format, MPEG format, etc.) as necessary. The control unit 73 can output the image data generated by the image processing unit 81 to the recording unit 77. The control unit 73 can output the image data generated by the image processing unit 81 to the display unit 75.

  The image processing unit 81 performs processing for detecting a main subject from image data as described in, for example, Japanese Patent Application Laid-Open No. 2010-16621 (US2010 / 0002940). Here, the “main subject” refers to a subject that is an object to be imaged, a subject that is noticed by the user (photographer), or a subject that is estimated to be noticed by the user. The number of main subjects is not limited to one in the image data, and there may be a plurality of main subjects. The image processing unit 81 detects a human body included in the image data as a main subject as described in, for example, Japanese Patent Application Laid-Open No. 2010-16621 (US2010 / 0002940).

  Parameters referred to when the image processing unit 81 performs image processing are included in the control parameters (imaging conditions). For example, parameters such as color signal processing (tone correction), white balance adjustment, gradation adjustment, and compression rate are included in the control parameters. The signal read from the image sensor 1 changes according to the charge accumulation time, and the parameters referred to when performing image processing also change according to the change in the signal.

  The image processing unit 81 extracts frames at predetermined timings from a plurality of frames obtained in time series from the image sensor 1. Alternatively, the image processing unit 81 discards frames at a predetermined timing among a plurality of frames obtained in time series from the image sensor 1. Thereby, since the amount of data can be reduced, it is possible to reduce the load of subsequent processing. In addition, the image processing unit 81 calculates one or a plurality of frames to be interpolated between the frames based on a plurality of frames obtained in time series from the image sensor 1. Then, the image processing unit 81 adds one or more calculated frames between the frames. As a result, a moving image with smoother motion can be reproduced during moving image reproduction.

  The focus detection unit 82 detects focus information (information on focusing) of the imaging optical system of the lens unit 71 using the signal output from the imaging device 1. The focus detection unit 82 performs, for example, phase difference type focus detection. When the focus detection unit 82 detects focus information, the imaging control unit 80, for example, as shown in FIG. 4D, the period in which charges are accumulated in the photoelectric conversion unit 3 and the charge in the photoelectric conversion unit 4 are stored. Set the accumulated period to the same length. The focus detection unit 82 compares the first output signal based on the charge accumulated in the photoelectric conversion unit 3 and the second output signal based on the charge accumulated in the photoelectric conversion unit 4 to detect focus information.

  For example, when the difference between the level of the first output signal and the level of the second output signal is equal to or smaller than the threshold value, the focus detection unit 82 determines that the lens unit 71 is in focus on the light receiving layer 12 of the image sensor 1. . Further, the focus detection unit 82 has a case where the difference between the level of the first output signal and the level of the second output signal is larger than the threshold value, and one level of the first output signal and the second output signal is higher than the other level. The focus is determined to be on the object plane side of the light receiving layer 12 (front pin). Further, the focus detection unit 82 has a case where the difference between the level of the first output signal and the level of the second output signal is larger than the threshold value, and the other level of the first output signal and the second output signal is higher than one level. The focus is determined to be on the object plane side of the light receiving layer 12 (front pin). The focus detection unit 82 detects the amount of deviation between the focus of the lens unit 71 and the photoelectric conversion unit (light receiving layer 12) as focus information based on the difference between the level of the first output signal and the level of the second output signal. May be.

  The focus detection unit 82 supplies the detected focus information to the imaging control unit 80. For example, the imaging control unit 80 uses the focus information detected by the focus detection unit 82 to calculate a driving amount by the lens driving unit 72 necessary for focusing on the photoelectric conversion unit. The imaging control unit 80 controls the focal point of the lens unit 71 by sending a command to the lens driving unit 72 based on the calculated driving amount.

  When the focus detection unit 82 detects focus information, the imaging control unit 80 sets the period in which charges are accumulated in the photoelectric conversion unit 3 and the period in which charges are accumulated in the photoelectric conversion unit 4 to have different lengths. It may be set. In this case, the focus detection unit 82 may detect the focus information based on the estimated value of the level of the output signal when the length of the exposure period is aligned between the photoelectric conversion unit 3 and the photoelectric conversion unit 4.

  For example, the imaging control unit 80 controls at least one of the amplification factor of the amplifier 55 and the amplification factor of the amplifier 56 by sending a command to the horizontal scanning circuit 52 of the imaging element 1, and the photoelectric conversion unit 3 and the photoelectric conversion unit 4. And a signal corresponding to a state in which the length of the exposure period is uniform may be output. For example, when the length of the exposure period for the photoelectric conversion unit 3 is twice the length of the exposure period for the photoelectric conversion unit 4, the imaging control unit 80 sets the amplification factor of the amplifier 55 to half the amplification factor of the amplifier 56. You may adjust. In addition, the control unit 73 (for example, the image processing unit 81 and the focus detection unit 82) calculates a signal level corresponding to a state in which the lengths of exposure periods are the same in the photoelectric conversion unit 3 and the photoelectric conversion unit 4. May be.

  The display unit 75 includes, for example, a liquid crystal display panel, and displays an image according to image data supplied from the control unit 73. The image data supplied from the control unit 73 is an image image indicating settings such as image data based on an image (for example, a still image, a moving image, and a live view image) captured by the image sensor 1 and imaging conditions of the imaging device 70. Data. The live view image is an image displayed on the display unit 75 by sequentially outputting the image data generated by the image processing unit 81 to the display unit 75. The live view image is used for the user to confirm the image of the subject imaged by the image sensor 1. The live view image is also called a through image or a preview image. The display unit 75 may include an input unit such as a touch panel.

  For example, when the imaging device 70 is a camera, the operation unit 76 is a release switch, a moving image switch, various operation switches, and the like operated by a user. The operation unit 76 outputs a signal corresponding to the operation by the user to the control unit 73. For example, the control unit 73 causes the image sensor 1 to execute an imaging process in response to an operation of the operation unit 76. In addition, the control unit 73 can display an image indicating shooting conditions and the like on the display unit 75, and the operation unit 76 can accept input from the user.

  The recording unit 77 has a card slot into which a storage medium 83 such as a memory card can be mounted. The recording unit 77 stores the image data generated by the image processing unit 81 and various data in the storage medium 83 attached to the card slot. For example, the control unit 73 causes the storage medium 83 to store data based on an image captured by the image sensor 1 in response to an operation of the operation unit 76. For example, the control unit 73 may store the image data generated by the HDR process and the storage medium 83, or may store the image data generated without performing the HDR process and the storage medium 83. In addition, the control unit 73 receives the first image data generated based on the first output signal based on the charge accumulated in the photoelectric conversion unit 3 and the second output signal based on the charge accumulated in the photoelectric conversion unit 4. The originally generated second image data may be separately stored in the storage medium 83. The first image data may be image data generated without using the second output signal, and the second image data may be image data generated without using the first output signal. .

  Next, an operation when the imaging device 70 is a camera will be described. FIG. 9 is a flowchart illustrating an example of the operation of the imaging device 70. In step S1, the imaging control unit 80 determines whether the still image mode is set.

  First, a case where the still image mode is set will be described. When it is determined in step S1 that the still image mode is set (step S1; Yes), the imaging control unit 80 determines in step S2 whether or not the HDR mode is set. When it is determined that the HDR mode is set in step S2 (step S2; Yes), the imaging control unit 80 sets an exposure period in step S3. Here, in the initial state, it is assumed that the photoelectric conversion unit 3 and the photoelectric conversion unit 4 have the same exposure period. In step S <b> 3, the imaging control unit 80 sets the length of the exposure period for the photoelectric conversion unit 3 to be different from the exposure period for the photoelectric conversion unit 4.

  If the imaging control unit 80 determines in step S2 that the HDR mode is not set (step S2; No), or if the processing in step S3 ends, a release switch (hereinafter abbreviated as release SW) in step S4. ) Has been operated. In step S4, when it is determined that the release SW is half-pressed (step S4; Yes), the imaging control unit 80 performs an auto exposure process (AE process) in step S5. In Step S4, when it is determined that the release SW has not been operated (Step S4; No), the imaging control unit 80 continues to monitor the operation of the release SW. In step S4, when it is determined that the release SW is fully pressed, the imaging control unit 80 causes the imaging device 1 to execute the imaging process in step S8 described later.

  The imaging control unit 80 performs autofocus processing (AF processing) in step S6 after the processing in step S5 is completed. In step S5, the imaging control unit 80 causes the imaging device 1 to perform imaging processing. Further, the focus detection unit 82 detects focus information using the first output signal based on the charge accumulated in the photoelectric conversion unit 3 and the second output signal based on the charge accumulated in the photoelectric conversion unit 4. Also. The imaging control unit 80 controls the lens driving unit 72 based on the focus information detected by the focus detection unit 82 and adjusts the focus of the lens unit 71.

  When the HDR mode is set, the imaging control unit 80 may display the live view image on the display unit 75 after the process of step S5 is completed or after the process of step S6 is completed. . Further, the imaging control unit 80 may accept an input requesting to change at least one of the length of the exposure period for the photoelectric conversion unit 3 and the length of the exposure period for the photoelectric conversion unit 4. For example, the user can change the brightness setting in the HDR process while viewing the live view image. In addition, the imaging control unit 80 automatically changes at least one of the length of the exposure period for the photoelectric conversion unit 3 and the length of the exposure period for the photoelectric conversion unit 4 based on, for example, the distribution of pixel values in the live view image. May be. Further, the imaging control unit 80 may execute such an exposure period changing process while monitoring the operation of the release SW in step S4, for example.

  Further, the imaging control unit 80 may adjust at least one of the amplification factor of the amplifier 55 and the amplification factor of the amplifier 56 using the result of the AE process. For example, the imaging control unit 80 sets an aperture so that a dark dark part of the imaging target can be detected in accordance with the longer exposure time of the photoelectric conversion unit 3 and the photoelectric conversion unit 4, and the exposure time is set under these conditions. The amplification factor may be adjusted so that the output is not saturated at the shorter side.

  The imaging control unit 80 determines whether or not the release SW is operated in step S7 after the process of step S6 is completed. When it is determined in step S7 that the release SW has been fully pressed (step S7; Yes), the imaging control unit 80 causes the imaging device 1 to perform an imaging process in step S8. If it is determined in step S7 that the release SW is not operated (step S7; No), the imaging control unit 80 continues to monitor the operation of the release SW. In step S7, if the imaging control unit 80 determines that the release SW has been half-pressed, the imaging control unit 80 continues to monitor the operation of the release SW after executing the AE process in step S5 and the AF process in step S6 again.

  When the HDR mode is set, the imaging control unit 80 sets various control signals output from the vertical scanning circuit 51 to, for example, the waveforms shown in FIG. 3 in step S8. When the HDR mode is not set, the imaging control unit 80 sets various control signals output from the vertical scanning circuit 51 to, for example, the waveforms shown in FIG. 4D in step S8. The imaging control unit 80 causes the horizontal scanning circuit 52 to output an imaging result after the period of one frame ends.

  After the process of step S8 ends, the control unit 73 causes the image processing unit 81 to generate image data corresponding to the imaging result in step S9. When the HDR mode is set, the image processing unit 81 performs HDR processing in step S9 to generate image data. If the HDR mode is not set, the image processing unit 81 generates image data without performing HDR processing in step S9. After the process of step S9 is completed, the control unit 73 outputs the image data generated in step S8 to at least one of the storage unit 7 and the display unit 75 in step S10. In this way, a series of processing ends.

  Next, a case where the moving image mode is set (when the still image mode is not set) will be described. When the moving image mode is set, the imaging control unit 80 focuses on, for example, the center of the angle of view or a position designated by the user. If it is determined in step S1 that the still image mode is not set (step S1; No), the imaging control unit 80 determines whether or not the HDR mode is set in step S11. When it is determined that the HDR mode is set in step S11 (step S11; Yes), the imaging control unit 80 sets an exposure period in step S12. Here, in the initial state, it is assumed that the photoelectric conversion unit 3 and the photoelectric conversion unit 4 have the same exposure period. In step S <b> 12, the imaging control unit 80 sets the length of the exposure period for the photoelectric conversion unit 3 to a length different from the exposure period for the photoelectric conversion unit 4.

  If the imaging control unit 80 determines in step S11 that the HDR mode is not set (step S11; No), or if the process in step S12 ends, in step S13, a video switch (step S13) that requests the start of recording. Hereinafter, it is determined whether or not an operation of abbreviated as moving image SW has been performed. In step S13, when it is determined that the moving image SW has been operated (step S13; Yes), the imaging control unit 80 causes the imaging device 1 to perform an imaging process in step S14. In Step S13, when it is determined that the moving image SW is not operated (Step S13; No), the imaging control unit 80 continues monitoring the operation of the moving image SW.

  When the HDR mode is set, the imaging control unit 80 sets various control signals output from the vertical scanning circuit 51 to, for example, the waveforms shown in FIG. 3 in step S14. When the HDR mode is not set, the imaging control unit 80 sets various control signals output from the vertical scanning circuit 51 to, for example, the waveforms illustrated in FIG. 4D in step S14. The imaging control unit 80 causes the horizontal scanning circuit 52 to output an imaging result after the period of one frame ends.

  The control unit 73 causes the image processing unit 81 to generate image data corresponding to the imaging result after the process of step S14 is completed. When the HDR mode is set, the image processing unit 81 performs HDR processing to generate image data. When the HDR mode is not set, the image processing unit 81 generates image data without performing HDR processing. The control unit 73 outputs the image data generated by the image processing unit 81 to at least one of the storage unit 7 and the display unit 75.

  When the process of step S14 ends, the imaging control unit 80 determines whether the operation of the moving image SW that requests the end of the recording is performed in step S15. When it is determined in step S15 that the moving image SW has been operated (step S15; Yes), the imaging control unit 80 ends the series of processes. Further, when it is determined in step S15 that the moving image SW has not been operated (step S15; No), the imaging control unit 80 determines whether imaging of a predetermined number of frames has been completed in step S16. If the imaging control unit 80 determines in step S16 that imaging for a predetermined number of frames has not been completed (step S16; No), the imaging control unit 80 returns to step S14 and causes the imaging device 1 to perform imaging processing for the next frame.

  If it is determined in step S16 that imaging for a predetermined number of frames has been completed (step S15; Yes), the imaging control unit 80 performs AF processing in step S17. In step S <b> 16, the imaging control unit 80 causes the imaging device 1 to perform imaging processing. Further, the focus detection unit 82 detects focus information using the first output signal based on the charge accumulated in the photoelectric conversion unit 3 and the second output signal based on the charge accumulated in the photoelectric conversion unit 4. Also. The imaging control unit 80 controls the lens driving unit 72 based on the focus information detected by the focus detection unit 82 and adjusts the focus of the lens unit 71.

  In step S17, the imaging control unit 80 may cause the imaging process to be executed during a period between frames of the output moving image, and execute the AF process using the result. In step S <b> 17, the imaging control unit 80 may execute the AF process using the image data of the output moving image frame. In addition, the imaging control unit 80 may execute the AF process using the outputs of the plurality of pixels P, or may use the signals output from some of the pixels P to perform AF processing. Processing may be executed. When performing AF processing using signals output from some of the pixels P, some of the pixels P may be fixed among the plurality of pixels P or changed to different pixels for each AF processing. May be. When the HDR mode is set, the exposure period length may be set to be the same for the photoelectric conversion unit 3 and the photoelectric conversion unit 4 for some of the pixels P used for the AF processing.

  The predetermined number of frames is set according to the response speed of the lens driving unit 72 and the like. For example, when the time required for the lens driving unit 72 to achieve the driving amount specified by the imaging control unit 80 is 1 second and the frame rate is 30 FPS, the predetermined number of frames is set to the lower limit of 30 frames. Is set to an arbitrary value. For example, a default value is set as the predetermined number of frames, and can be changed to a value equal to or more than the lower limit value specified by the user.

  The imaging control unit 80 returns to step S13 after the processing of step S17 is completed, and causes the imaging device 1 to perform imaging processing of the next frame. As described above, the imaging control unit 80 repeatedly executes the imaging process in step S13 until it is determined in step S14 that the operation of the moving image SW requesting the stop of the recording has been performed. Further, the imaging control unit 80 executes the AF process in step S17 every time an imaging process for a predetermined number of frames is performed.

  The control unit 73 as described above may include a CPU (Central Processing Unit), for example, and the CPU may execute at least one process according to the imaging control program. This imaging control program is provided in the control unit 73 from the photoelectric conversion unit 3 among the photoelectric conversion unit 3 and the photoelectric conversion unit 4 that are arranged in the first pixel and converts light into electric charge. In addition, the charge may be transferred and the charge may be transferred from the photoelectric conversion unit 4 in a second period different from the first period in one frame period. The control unit 73 may include an arithmetic circuit such as an ASIC. For example, the image processing unit 81 may include an arithmetic circuit, and the arithmetic circuit may execute at least one process.

  In the imaging device 70 according to the present embodiment, since a plurality of photoelectric conversion units are arranged in one pixel of the imaging device 1, the dynamic range can be expanded without reducing the resolution of the captured image. In addition, since the transfer unit 21 for transferring charges from the photoelectric conversion unit 3 and the transfer unit 41 for transferring charges from the photoelectric conversion unit 4 can be controlled independently, a decrease in frame rate can be suppressed. In addition, the imaging device 70 may perform AF processing using the first output signal based on the charge accumulated in the photoelectric conversion unit 3 and the second output signal based on the charge accumulated in the photoelectric conversion unit 4. it can.

  The technical scope of the present invention is not limited to the above-described embodiment or modification. For example, one or more of the requirements described in the above embodiments or modifications may be omitted. In addition, the requirements described in the above embodiments or modifications can be combined as appropriate.

DESCRIPTION OF SYMBOLS 1 Image pick-up element, 3 photoelectric conversion part, 4 photoelectric conversion part, P pixel, 21 transfer part, 22 transfer control line, 23 charge voltage conversion part, 24 reset part, 25 reset control line, 41 transfer part, 42 transfer control line, 43 charge voltage conversion unit, 44 reset unit, 45 reset control line, 55 amplifier, 56 amplifier, 60 photoelectric conversion unit, 61 photoelectric conversion unit, 62 photoelectric conversion unit, 63 transfer unit, 64 transfer control line, 65 transfer unit, 66 Transfer control line, 70 imaging device, 80 imaging control unit, 81 image processing unit, 82 focus detection unit, P pixel, P1 pixel, P2 pixel, TX1 control signal, TX2 control signal, TX3 control signal, TX4 control signal, RST1 control Signal, RST2 control signal

Claims (12)

A first photoelectric conversion unit and a second photoelectric conversion unit which are arranged in the first pixel and convert light into electric charge;
A first transfer unit that transfers charges from the first photoelectric conversion unit;
A first transfer control line for transmitting a control signal to the first transfer unit;
A second transfer unit that transfers charges from the second photoelectric conversion unit;
An image sensor comprising: a second transfer control line provided separately from the first transfer control line and transmitting a control signal to the second transfer unit. A first charge-voltage converter that converts the charge from the first transfer unit into a voltage;
A first reset unit for resetting a voltage converted by the first charge-voltage conversion unit;
A first reset control line for transmitting a control signal to the first reset unit;
A second charge-voltage converter for converting the charge from the second transfer unit into a voltage;
A second reset unit for resetting the voltage converted by the second charge voltage conversion unit;
The imaging device according to claim 1, further comprising: a second reset control line that is provided separately from the first reset control line and transmits a control signal to the second reset unit. A first amplifying unit that amplifies a signal according to an output from the first photoelectric conversion unit;
A second amplifying unit for amplifying a signal according to the output from the second photoelectric conversion unit,
The imaging device according to claim 1, wherein the amplification factor of the first amplification unit and the amplification factor of the second amplification unit can be adjusted to different values. A third photoelectric conversion unit and a fourth photoelectric conversion unit which are arranged in a second pixel adjacent to the first pixel and convert light into electric charge;
A third transfer unit that transfers charges from the third photoelectric conversion unit;
A third transfer control line for transmitting a control signal to the third transfer unit;
A fourth transfer unit that transfers charges from the fourth photoelectric conversion unit;
A fourth transfer control line that is provided separately from the third transfer control line and transmits a control signal to the fourth transfer unit;
A third charge-voltage converter that converts the charge from the first transfer unit into a voltage and converts the charge from the third transfer unit into a voltage;
4. A fourth charge-voltage conversion unit that converts the charge from the second transfer unit into a voltage and converts the charge from the fourth transfer unit into a voltage. 5. The imaging device according to item. The imaging device according to any one of claims 1 to 4,
An imaging apparatus comprising: an imaging control unit that controls a first period in which charges are accumulated in the first photoelectric conversion unit and a second period in which charges are accumulated in the second photoelectric conversion unit.   The imaging apparatus according to claim 5, wherein the imaging control unit controls the first period and the second period to periods having different lengths.   The imaging apparatus according to claim 5, further comprising an image processing unit that performs image processing using a signal output from the imaging element.   The image processing unit uses a signal based on the charge accumulated in the first photoelectric conversion unit in the first period and a signal based on the charge accumulated in the second photoelectric conversion unit in the second period. The imaging apparatus according to claim 7, wherein the image processing is executed.   The imaging apparatus according to claim 8, wherein the imaging control unit sets the time point at which the first period ends and the time point at which the second period ends to be the same.   The focus detection part which detects the focus information of the imaging optical system which condenses light to the said image sensor using the signal output from the said image sensor is provided. Imaging device. The imaging control unit sets the third period in which charges are accumulated in the first photoelectric conversion unit and the fourth period in which charges are accumulated in the second photoelectric conversion unit to the same length,
The focus control unit uses a signal based on the charge accumulated in the first photoelectric conversion unit in the third period and a signal based on the charge accumulated in the second photoelectric conversion unit in the fourth period. The imaging apparatus according to claim 10, wherein focus information of the imaging optical system is detected. Of the first photoelectric conversion unit and the second photoelectric conversion unit that are arranged in the first pixel and converts light into electric charge, the electric charge is transferred from the first photoelectric conversion unit to the first period of one frame period. When,
An image pickup method comprising: transferring charges from the second photoelectric conversion unit to a second period different from the first period in the one frame period.

Images