JP2018207544A - Imaging apparatus and imaging device - Google Patents

Abstract

To obtain appropriate exposure for each image area.SOLUTION: An imaging apparatus includes: an imaging part 113 in which a plurality of photoelectric conversion parts 104 are arranged; a control part that divides the imaging part 113 into a plurality of area blocks containing the plurality of photoelectric conversion parts 104, controls storage time of the photoelectric conversion parts 104 by an area block unit, and is capable of reading a stored signal by an area block unit; and a first monitoring sensor and a second monitoring sensor, disposed at least at a first area block and a second area block, respectively, capable of reading charge storage amounts of the photoelectric conversion parts 104.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an imaging device and an imaging element.

  The image sensor includes a photoelectric conversion unit for image acquisition and a photoelectric conversion unit for luminance evaluation, and when the added value of signals repeatedly output from the luminance evaluation photoelectric conversion unit reaches a predetermined value, the photoelectric conversion for image acquisition An imaging device that outputs an image signal from a unit is known (see Patent Document 1).

JP 2012-204938 A

  In the conventional technique, it is difficult to apply the method to a case where an image is divided into blocks each having one or more of the above-described areas and a captured image is acquired for each block.

An imaging apparatus according to the present invention divides an imaging unit into an imaging unit in which a plurality of photoelectric conversion units are arranged and a plurality of area blocks including a plurality of photoelectric conversion units, and controls the accumulation time of the photoelectric conversion units in area blocks. And a control unit that can read out the accumulated signal in area block units, a first monitoring sensor that is arranged in each of at least the first area block and the second area block, and that can read out the charge accumulation amount by the photoelectric conversion unit, and the second And a monitoring sensor.
The imaging device according to the present invention is applied to an imaging device in which a plurality of photoelectric conversion units are arranged in an imaging unit. The accumulation time of the photoelectric conversion unit can be controlled in units of a plurality of area blocks obtained by dividing the imaging unit so as to include a plurality of photoelectric conversion units, and the accumulation signal of the photoelectric conversion unit can be read out in units of area blocks. The first monitoring sensor and the second monitoring sensor are arranged in each of the first area block and the second area block, and the charge accumulation amount obtained from the first monitoring sensor and the second monitoring sensor in the photoelectric conversion unit of the first area block The charge accumulation amounts obtained by the photoelectric conversion units of the second area block can be read out.

  According to the present invention, appropriate exposure can be obtained for each area of an image.

It is sectional drawing of a multilayer type image pick-up element. It is a figure explaining the pixel arrangement | sequence and unit block of an imaging chip. It is a circuit diagram explaining the block of an imaging chip. It is a block diagram which shows the functional structure of an image pick-up element. It is a figure explaining the flow of the pixel signal per pixel. It is a block diagram which illustrates the composition of the imaging device which has an image sensor. It is a figure explaining the several pixel arrangement | positioning in a block. It is a figure which shows the relationship between the pixel position in a block, and a pixel signal level. It is a figure explaining the pixel signal read from an image sensor via a read timing, accumulation | storage time, and an arithmetic circuit. It is a figure explaining a normalization process. It is a flowchart explaining the flow of the imaging | photography operation | movement which a control part performs.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.
<Description of Laminated Image Sensor>
First, a multilayer imaging device 100 mounted on an electronic apparatus (for example, the imaging device 1) according to an embodiment of the present invention will be described. The multilayer image sensor 100 is described in Japanese Patent Application No. 2012-139026 filed earlier by the applicant of the present application. FIG. 1 is a cross-sectional view of the multilayer image sensor 100. The imaging device 100 includes a backside illumination type imaging chip 113 that outputs a pixel signal corresponding to incident light, a signal processing chip 111 that processes the pixel signal, and a memory chip 112 that stores the pixel signal. The imaging chip 113, the signal processing chip 111, and the memory chip 112 are stacked, and are electrically connected to each other by a conductive bump 109 such as Cu.

  As shown in the figure, incident light is incident mainly in the positive direction of the Z-axis indicated by a white arrow. In the present embodiment, in the imaging chip 113, the surface on the side where incident light is incident is referred to as a back surface. Further, as shown in the coordinate axes, the left direction of the paper orthogonal to the Z axis is the X axis plus direction, and the front side of the paper orthogonal to the Z axis and the X axis is the Y axis plus direction. In the following several figures, the coordinate axes are displayed so that the orientation of each figure can be understood with reference to the coordinate axes of FIG.

  An example of the imaging chip 113 is a back-illuminated MOS image sensor. The PD layer 106 is disposed on the back side of the wiring layer 108. The PD layer 106 includes a plurality of PDs (photodiodes) 104 that are two-dimensionally arranged and store charges corresponding to incident light, and transistors 105 that are provided corresponding to the PDs 104.

  A color filter 102 is provided on the incident side of incident light in the PD layer 106 via a passivation film 103. The color filter 102 has a plurality of types that transmit different wavelength regions, and has a specific arrangement corresponding to each of the PDs 104. The arrangement of the color filter 102 will be described later. A set of the color filter 102, the PD 104, and the transistor 105 forms one pixel.

  On the incident light incident side of the color filter 102, a microlens 101 is provided corresponding to each pixel. The microlens 101 condenses incident light toward the corresponding PD 104.

  The wiring layer 108 includes a wiring 107 that transmits the pixel signal from the PD layer 106 to the signal processing chip 111. The wiring 107 may be multilayer, and a passive element and an active element may be provided.

  A plurality of bumps 109 are disposed on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the opposing surfaces of the signal processing chip 111, and the imaging chip 113 and the signal processing chip 111 are pressed and aligned. The bumps 109 are joined and electrically connected.

  Similarly, a plurality of bumps 109 are disposed on the mutually facing surfaces of the signal processing chip 111 and the memory chip 112. The bumps 109 are aligned with each other, and the signal processing chip 111 and the memory chip 112 are pressurized, so that the aligned bumps 109 are joined and electrically connected.

  The bonding between the bumps 109 is not limited to Cu bump bonding by solid phase diffusion, and micro bump bonding by solder melting may be employed. Further, for example, about one bump 109 may be provided for one unit region described later. Therefore, the size of the bump 109 may be larger than the pitch of the PD 104. Further, a bump larger than the bump 109 corresponding to the pixel region may be provided in a peripheral region other than the pixel region where the pixels are arranged.

  The signal processing chip 111 has a TSV (silicon through electrode) 110 that connects circuits provided on the front and back surfaces to each other. The TSV 110 is preferably provided in the peripheral area. The TSV 110 may also be provided in the peripheral area of the imaging chip 113 and the memory chip 112.

  FIG. 2 is a diagram for explaining the pixel array and the unit block 131 of the imaging chip 113. In particular, a state where the imaging chip 113 is observed from the back side is shown. For example, 20 million or more pixels are arranged in a matrix in the pixel region. In the present embodiment, for example, 64 pixels of adjacent 8 pixels × 8 pixels form one block 131. The grid lines in the figure indicate the concept that adjacent pixels are grouped to form a block 131. The number of pixels forming the block 131 is not limited to this, and may be, for example, 32 pixels × 64 pixels, or more or less. In the present embodiment, there is no circuit or wiring between the plurality of blocks 131, and the plurality of blocks are arranged densely, thereby realizing space saving.

  As shown in the partially enlarged view of the pixel region, the block 131 includes 16 so-called Bayer arrays, which are composed of four pixels of green pixels Gb, Gr, blue pixels B, and red pixels R, vertically and horizontally. The green pixel is a pixel having a green filter as the color filter 102, and receives light in the green wavelength band of incident light. Similarly, a blue pixel is a pixel having a blue filter as the color filter 102 and receives light in the blue wavelength band, and a red pixel is a pixel having a red filter as the color filter 102 and receiving light in the red wavelength band. Receive light.

  In the present embodiment, a plurality of blocks 131 are defined so that one block 131 includes at least one set of green pixels Gb, Gr, blue pixels B, and red pixels R, and each block 131 has a different control parameter. The pixels included in 131 can be controlled. That is, it is possible to acquire imaging signals having different imaging conditions between a pixel group included in one block 131 and a pixel group included in another block 131. Examples of the control parameters are a frame rate, a gain, a thinning rate, the number of addition rows or addition columns to which pixel signals are added, the charge accumulation time or accumulation count, the number of digitization bits, and the like. Furthermore, the control parameter may be a parameter in image processing after obtaining an image signal from a pixel.

  FIG. 3 is a circuit diagram illustrating the block 131 of the imaging chip 113. In FIG. 3, a rectangle surrounded by a dotted line typically represents a circuit corresponding to one pixel. In the example of FIG. 3, 16 pixels of the 64 pixels forming the block 131 are illustrated. Note that at least some of the transistors described below correspond to the transistor 105 in FIG.

  The PD 104 in each pixel is connected to a transfer transistor 302, and each gate of each transfer transistor 302 is connected to a TX wiring 307 (transfer unit control line) to which a transfer pulse is supplied. In the present embodiment, the TX wiring 307 is commonly connected to the 64 transfer transistors 302.

  The drain of each transfer transistor 302 is connected to the source of the corresponding reset transistor 303, and a so-called floating diffusion FD between the drain of the transfer transistor 302 and the source of the reset transistor 303 is connected to the gate of the amplification transistor 304. The drain of the reset transistor 303 is connected to a Vdd wiring 310 to which a power supply voltage is supplied, and the gate thereof is connected to a reset wiring 306 to which a reset pulse is supplied. In the present embodiment, the reset wiring 306 is commonly connected to the 64 reset transistors 303.

  The drain of each amplification transistor 304 is connected to a Vdd wiring 310 to which a power supply voltage is supplied. The source of each amplification transistor 304 is connected to the drain of each corresponding selection transistor 305. Each gate of the selection transistor 305 is connected to a decoder wiring 308 to which a selection pulse is supplied. In the present embodiment, the decoder wiring 308 is provided independently for each of the 64 select transistors 305. The source of each selection transistor 305 is connected to a common output wiring 309. The load current source 311 supplies current to the output wiring 309. That is, the output wiring 309 for the selection transistor 305 is formed by a source follower. Note that the load current source 311 may be provided on the imaging chip 113 side or may be provided on the signal processing chip 111 side.

  Here, the flow from the start of charge accumulation to pixel output after the end of accumulation will be described. When a reset pulse is applied to the reset transistor 303 through the reset wiring 306 and simultaneously a transfer pulse is applied to the transfer transistor 302 through the TX wiring 307, the potentials of the PD 104 and the floating diffusion FD are reset.

  When the application of the transfer pulse is canceled, the PD 104 converts the incident light to be received into electric charge and accumulates it. Thereafter, when the transfer pulse is applied again without the reset pulse being applied, the accumulated charge is transferred to the floating diffusion FD, and the potential of the floating diffusion FD changes from the reset potential to the signal potential after the charge accumulation. . When a selection pulse is applied to the selection transistor 305 through the decoder wiring 308, a change in the signal potential of the floating diffusion FD is transmitted to the output wiring 309 through the amplification transistor 304 and the selection transistor 305. Thereby, a pixel signal corresponding to the reset potential and the signal potential is output from the unit pixel to the output wiring 309.

  As shown in FIG. 3, in this embodiment, the reset wiring 306 and the TX wiring 307 are common to the 64 pixels forming the block 131. That is, the reset pulse and the transfer pulse are simultaneously applied to all 64 pixels. Accordingly, all the pixels forming the block 131 start charge accumulation at the same timing and end charge accumulation at the same timing. However, the pixel signal corresponding to the accumulated charge is selectively output from the output wiring 309 by sequentially applying the selection pulse to each selection transistor 305. Further, the reset wiring 306, the TX wiring 307, and the output wiring 309 are provided separately for each block 131.

  By configuring the circuit with the block 131 as a reference in this way, the charge accumulation time can be controlled for each block 131. In other words, pixel signals having different frame rates can be output between the blocks 131. More specifically, while one block 131 performs charge accumulation once, the other block 131 repeats charge accumulation many times and outputs a pixel signal each time. It is also possible to output each frame for moving images at different frame rates.

  FIG. 4 is a block diagram illustrating a functional configuration of the image sensor 100. The analog multiplexer 411 sequentially selects the 64 PDs 104 forming the block 131 and outputs each pixel signal to the output wiring 309 provided corresponding to the block 131. The multiplexer 411 is formed on the imaging chip 113 together with the PD 104.

  The pixel signal output via the multiplexer 411 is supplied to the signal processing chip 111 by a signal processing circuit 412 that performs correlated double sampling (CDS) / analog / digital (A / D) conversion. D conversion is performed. The A / D converted pixel signal is delivered to the demultiplexer 413. The pixel signal output from the demultiplexer 413 is input to the adder 416 corresponding to each pixel. The adder 416 adds the pixel signal output from the demultiplexer 413 and the pixel signal read from the pixel memory 414 to correspond to each pixel, and outputs the pixel signal after the addition to the pixel memory 414 again. To do.

  The pixel memory 414 stores the pixel signal from the adder 416. Each of the pixel memories 414 has a capacity capable of storing the pixel signal after addition. The demultiplexer 413, the adder 416, and the pixel memory 414 are formed in the memory chip 112.

  FIG. 5 is a diagram illustrating the flow of pixel signals per pixel. In FIG. 5, the pixel signal S output from the demultiplexer 413 is input to the corresponding adder n among the adders 416. At this time, the pixel signal P stored in the corresponding memory n of the pixel memory 414 is read from the memory n and input to the adder n.

  The adder n adds the input pixel signal S and the pixel signal P, and outputs the pixel signal S + P after the addition to the pixel memory n. The pixel memory n stores the input pixel signal S + P and waits for reading to the arithmetic circuit 415. Here, when the addition is performed by the adder n, only the pixel signal S input to the adder n is controlled by controlling the pixel memory 414 so as not to read the pixel signal P stored in the pixel memory n. Can be directly output from the adder n to the pixel memory n. That is, the pixel signal S from the imaging chip 113 can be read from the memory n to the arithmetic circuit 415 without being added by the adder n.

  The arithmetic circuit 415 processes the pixel signal stored in the pixel memory 414 and passes it to the subsequent image processing unit. The arithmetic circuit 415 may be provided in the signal processing chip 111 or may be provided in the memory chip 112.

  The drive control unit 417 is configured to send pixel signals from the imaging chip 113 to the signal processing chip 111 and the memory chip 112, read and store timings of the pixel signals in the pixel memory 414, and addition timings of the pixel signals in the adder 416. In order to synchronize the pixel signal delivery timing to the arithmetic circuit 415, a timing control signal is generated.

  FIG. 4 shows connections for one block 131, but actually these exist for each block 131 and operate in parallel. However, the arithmetic circuit 415 may not exist for each block 131. For example, one arithmetic circuit 415 may perform sequential processing while sequentially referring to the values of the pixel memory 414 corresponding to each block 131.

  As described above, the output wiring 309 is provided corresponding to each of the blocks 131. Since the image pickup device 100 has the image pickup chip 113, the signal processing chip 111, and the memory chip 112 laminated, by using electrical connection between the chips using the bump 109 for the output wiring 309, each chip is arranged in the surface direction. Wiring can be routed without increasing the size.

<Description of imaging device>
FIG. 6 is a block diagram illustrating the configuration of the imaging apparatus 1 having the imaging element 100 described above. In FIG. 6, the imaging apparatus 1 includes an imaging optical system 10, an imaging unit 20, an image processing unit 30, a work memory 40, a display unit 50, a recording unit 60, and a control unit 70.

  The imaging optical system 10 includes a plurality of lenses, and guides a light beam from the object scene to the imaging unit 20. The imaging optical system 10 may be configured integrally with the imaging device 1 or may be configured to be replaceable with respect to the imaging device 1. Further, the imaging optical system 10 may include a focus lens or a zoom lens.

  The imaging unit 20 includes the above-described imaging device 100 and a driving unit 21 that drives the imaging device 100. The image pickup device 100 can be controlled to be stored for each block 131 described above by being driven and controlled by a control signal output from the drive unit 21. The control unit 70 instructs the drive unit 21 to perform accumulation control.

  The image processing unit 30 performs image processing on the image data imaged by the imaging unit 20 in cooperation with the work memory 40. In the present embodiment, the work memory 40 temporarily stores image data before and after JPEG compression and before and after MPEG compression, and is used as a buffer memory for images captured by the imaging unit 20. The display unit 50 includes, for example, a liquid crystal display panel 51, and displays images (still images and moving images) and various information captured by the imaging unit 20, and displays an operation input screen. The display unit 50 has a configuration in which a touch panel 52 is laminated on the display surface of the liquid crystal display panel 51. The touch panel 52 outputs a signal indicating a position where the user touches the liquid crystal display panel 51.

  The recording unit 60 stores various data such as image data acquired in response to an imaging instruction (release operation described later) in a storage medium such as a memory card. The control unit 70 has a CPU and controls the overall operation of the imaging apparatus 1. The control unit 70 controls the blocks 131 of the imaging device 100 (imaging chip 113) to acquire an image at a predetermined frame rate (accumulation time) and gain, and to perform read control of the acquired image data. The parameter is instructed to the drive unit 21.

  Further, the control unit 70 causes the AWB calculation unit 71 to perform white balance adjustment based on the image data. Further, the control unit 70 generates an image for reproduction display based on the pixel signal and causes the display unit 50 to display the image.

<Monitoring sensor>
In this embodiment, the accumulation time of all the pixels included in the block 131 is determined based on the charge accumulation amount by the monitoring sensor. Here, a sensor that checks the charge accumulation amount in the pixel is called a monitoring sensor. FIG. 7 is a diagram for explaining a plurality of pixel arrangements in each block 131. The imaging apparatus 1 causes one green pixel Gr (3, 4) located substantially in the center of the block 131 to function as a monitoring sensor that represents the block 131.

  The control unit 70 sends an instruction to the drive unit 21 when, for example, taking a still image, and reads out pixel signals from the respective blocks 131 of the imaging device 100 (imaging chip 113) at predetermined time intervals. For example, the pixel signal is read out in a plurality of times at times t1, t2, t3,..., T7, t8.

  The control unit 70 checks the pixel signal level from the monitoring sensor (green pixel Gr (3,4)) among the read pixel signals. Then, when the integrated value of the read pixel signals exceeds a predetermined determination threshold (for example, time t5), all pixels in the block 131 are accumulated until the time immediately before that time (time t4). Decide as time. In this case, reading of pixel signals from the block 131 after time t6 is omitted.

  FIG. 8 is a diagram showing the relationship between all pixel positions in the block 131 and the pixel signal level. When the pixel signal level from the monitoring sensor (green pixel Gr (3,4)) exceeds the determination threshold, the control unit 70 controls all of the blocks 131 in the block 131 even when the pixel signal levels from other pixels are lower than the determination threshold. The accumulation for the pixel ends. In other words, if the pixel signal level from the monitoring sensor (green pixel Gr (3,4)) does not exceed the determination threshold, even if the pixel signal level from another pixel exceeds the determination threshold. The accumulation for all the pixels in the block 131 is continued.

  However, even if the pixel signal level from the monitoring sensor (green pixel Gr (3,4)) read out at time t8 does not exceed the determination threshold, the control unit 70 sets the time until time t8 as the upper limit of the accumulation time. To do.

<Example of pixel signal readout>
The pixel signal readout from the block 131 will be described with reference to FIG. 9 for explaining the pixel signal readout timing, the accumulation time in the imaging chip 113, and the pixel signal read out from the imaging device 100 via the arithmetic circuit 415. To do.

  The drive unit 21 controls the image sensor 100 as follows. That is, the first accumulation time is from the accumulation start time t0 to the time t1, and the second accumulation time is from the time t0 to the time t2. The drive unit 21 starts charge accumulation for the pixels included in the block 131 at time t0. Then, at time t1, the pixel signal is output from the block 131 while controlling the pixel memory 414 so as not to read out the pixel signal stored in the pixel memory n illustrated in FIG. As a result, the pixel signal a accumulated during the first accumulation time (from time t0 to time t1) is output from the demultiplexer 413, and is directly output as the signal A via the arithmetic circuit 415. This pixel signal A (= a) is also stored in the pixel memory n.

  Furthermore, when the driving unit 21 reads the pixel signal at time t1, the driving unit 21 immediately starts charge accumulation for the pixels included in the block 131. At time t2, the pixel signal is output from the block 131 while controlling the pixel memory 414 so as to read out the pixel signal a stored in the pixel memory n illustrated in FIG. As a result, the pixel signal b accumulated from time t1 to time t2 is output from the demultiplexer 413, and this pixel signal b and the pixel signal a read from the pixel memory n are added by the adder n. Is done. The pixel signal a + b after the addition is output as the signal B through the arithmetic circuit 415. This pixel signal B is also stored in the pixel memory n. Since the pixel signal B (= a + b) is the sum of the pixel signals accumulated from the time t0 to the time t1 and from the time t1 to the time t2, it is accumulated during the second accumulation time (time t0 to time t2). This corresponds to the pixel signal to be processed.

  Similarly, when the driving unit 21 reads the pixel signal at time t2, the driving unit 21 immediately starts charge accumulation for the pixels included in the block 131. At time t3, the pixel signal is output from the block 131 while controlling the pixel memory 414 so as to read out the pixel signal B stored in the pixel memory n illustrated in FIG. As a result, the pixel signal c accumulated from time t2 to time t3 is output from the demultiplexer 413, and this pixel signal c and the pixel signal B read from the pixel memory n are added by the adder n. Is done. The pixel signal B + c after the addition is output as the signal C through the arithmetic circuit 415. This pixel signal C is also stored in the pixel memory n. Since the pixel signal C (= B + c) is the sum of the pixel signals accumulated from time t0 to time t2 and from time t2 to time t3, it is accumulated during the third accumulation time (time t0 to time t3). This corresponds to the pixel signal to be processed.

  Thereafter, similarly, by reading out the pixel signal from time t4 to time t8, the fourth accumulation time (time t0 to time t4), the fifth accumulation time (time t0 to time t5), and the sixth accumulation time ( Pixel signals accumulated during time t0 to time t6), seventh accumulation time (time t0 to time t7), and eighth accumulation time (time t0 to time t8) are respectively obtained. As described above, the accumulation from the monitoring sensor (green pixel Gr (3,4)) to the eighth accumulation time (from time t0 to time t8) is based on a predetermined determination threshold. Only if it does not exceed.

<Normalization processing>
As described above, when the accumulation time is determined for each block 131, the accumulation time may differ between the blocks 131 if the incident light amount differs between the different blocks 131. Therefore, the control unit 70 performs normalization processing on the accumulation time between the blocks 131 to generate an image.

  FIG. 10 is a diagram for explaining normalization processing. For example, based on the pixel signal value of the block 131 whose accumulation time is the eighth accumulation time (time t0 to time t8), for the block 131 whose accumulation time is the second accumulation time (time t0 to time t2), The pixel signal value is calculated by 4 (= 8/2) times. For the block 131 having the fourth accumulation time (from time t0 to time t4), the pixel signal value is calculated by 2 (= 8/4) times. Further, for the block 131 whose accumulation time is the fifth accumulation time (from time t0 to time t5), the pixel signal value is calculated by 8/5 times. The same applies to other accumulation times.

  As described above, the control unit 70 adjusts the pixel signal value according to the difference in accumulation time for each block 131 and normalizes the pixel signal value to a space having a predetermined bit length (for example, 14 bits). Thereby, an image having a wide dynamic range in which the magnitude of the pixel signal value due to the difference in the accumulation time between the different blocks 131 is corrected is obtained. The control unit 70 causes the AWB calculation unit 71 to perform white balance adjustment based on the pixel signal after performing the normalization process in this way.

<Description of flowchart>
FIG. 11 is a flowchart illustrating the flow of the photographing operation performed by the control unit 70 of the imaging device 1. The control unit 70 repeatedly activates the processing of FIG. 11 when an unillustrated ON-OFF switch is turned on and energization is performed for each unit of the imaging apparatus 1. The control unit 70 also activates the process of FIG. 11 when a release button (not shown) is pressed halfway. The half-press operation refers to an operation mode in which the release button is pressed down more shallowly than in the full-press operation.

  In step S101 in FIG. 11, the control unit 70 determines whether or not a release operation (that is, a release button full-press operation) has been performed. When an operation signal indicating that the release button has been fully pressed is input, the control unit 70 makes a positive determination in step S101 and proceeds to step S102. On the other hand, if the full-press operation is not performed, the control unit 70 makes a negative determination in step S101 and repeats the determination process.

  In step S102, the control unit 70 sends an instruction to the drive unit 21, starts charge accumulation for all the blocks 131 of the image sensor 100, and proceeds to step S103 (corresponding to the time t0). In step S103, pixel signals are read from the imaging chip 113 in units of blocks 131. In step S104, the adder 416 adds the read pixel signal and the pixel signal stored in the pixel memory 414 in correspondence with each pixel. The pixel signal after the addition is stored in the pixel memory 414 again.

  In step S <b> 105, the control unit 70 determines, for each block 131, whether or not the integrated value after addition exceeds a predetermined determination threshold regarding the pixel signal from the monitoring sensor. The control unit 70 makes a positive determination in step S105 for the block 131 that has exceeded the determination threshold value, and proceeds to step S106. The controller 70 makes a negative determination in step S105 for the block 131 that does not exceed the determination threshold value, and returns to step S103. When returning to step S103, the processing described above for the block 131 is continued.

  In step S106, the control unit 70 sends an instruction to the drive unit 21, ends the charge accumulation for the corresponding block 131, and proceeds to step S107. In step S107, the control unit 70 performs normalization processing on the accumulation time between the blocks 131 as described above, and proceeds to step S108.

  In step S108, the control unit 70 sends an instruction to the AWB calculation unit 71, performs white balance processing, and proceeds to step S109. In step S109, the control unit 70 sends an instruction to the recording unit 60, causes the image data to be recorded on a storage medium such as a memory card, and ends the process shown in FIG.

According to the embodiment described above, the following operational effects can be obtained.
(1) The imaging apparatus 1 divides the imaging chip 113 into an imaging chip 113 in which a plurality of PDs 104 are arranged and a plurality of area blocks 131 including a plurality of PDs 104, and controls the accumulation time of the PD 104 in units of area blocks 131. In addition, the control unit 70 that can read the stored signal in units of area blocks 131 and at least each of the first area block 131 and the second area block 131, and a first monitoring sensor that can read the charge storage amount by the PD 104 Gr (3,4) and the second monitoring sensor Gr (3,4) are provided. Thereby, appropriate exposure can be obtained for each area of the image corresponding to each block 131. For example, proper exposure can be obtained for the main subject without saturating the background even when shooting in backlight. In addition, since a proper exposure can be obtained by one shooting operation, it is not necessary to repeat the shooting operation.

(2) Each of the plurality of PDs 104 is provided with a color filter 102 of other colors (B, R) other than green Gr (Gb) or green Gr (Gb), and the first monitoring sensor Gr (3,4) and The second monitoring sensor Gr (3, 4) is configured by the PD 104 in which the green Gr (Gb) color filter 102 is arranged in the first area block 131 and the second area block 131. By using the PD 104 provided with a green color filter 102, which is generally regarded as having high sensitivity, as a monitoring sensor, the charge accumulation amount can be obtained appropriately.

(3) The first monitoring sensor Gr (3,4) and the second monitoring sensor Gr (3,4) are arranged in the approximate center of the first area block 131 and the second area block 131. The charge accumulation amount can be obtained based on typical incident light on the block 131.

(4) When the charge accumulation amount by the first monitoring sensor Gr (3,4) reaches a predetermined accumulation amount, the control unit 70 ends the charge accumulation of the PD 104 included in the corresponding first area block 131. Therefore, the accumulation time of the first area block 131 in the PD 104 can be appropriately controlled.

(5) When the charge accumulation amount by the second monitoring sensor Gr (3,4) reaches a predetermined accumulation amount, the control unit 70 ends the charge accumulation of the PD 104 included in the corresponding second area block 131. Therefore, the accumulation time for the PD 104 in the second area block 131 can be appropriately controlled.

(6) Since the control unit 70 reads the accumulation signal from the PD 104 that has completed the charge accumulation in units of area blocks 131, the pixel signal for each area of the image corresponding to each block 131 can be appropriately read.

(7) Since the control unit 70 starts reading the charge accumulation amount from the first monitoring sensor Gr (3,4) and the second monitoring sensor Gr (3,4) in response to the release operation, It is possible to appropriately detect the charge accumulation amount after the imaging instruction).

(8) In the imaging device 100 in which a plurality of PDs 104 are arranged on the imaging chip 113, the accumulation time of the PD 104 can be controlled in units of a plurality of area blocks 131 in which the imaging chip 113 is divided so as to include a plurality of PDs 104. The accumulated signal of the PD 104 can be read in units of area blocks 131, and at least the first monitoring block Gr (3,4) and the second monitoring sensor Gr (3,4) are respectively provided in the first area block 131 and the second area block 131. ), The charge accumulation amount obtained in the PD 104 of the first area block 131 from the first monitoring sensor Gr (3,4) and the second monitoring sensor Gr (3,4), and the PD 104 of the second area block 131. The amount of charge accumulated in each can be read out.By using such an image sensor 100, appropriate exposure can be obtained for each area of the image corresponding to each block 131. For example, proper exposure can be obtained for the main subject without saturating the background even when shooting in backlight. In addition, since a proper exposure can be obtained by one shooting operation, it is not necessary to repeat the shooting operation.

(Modification 1)
The imaging device 1 according to the above-described embodiment may be configured by a high function mobile phone or a tablet terminal. In this case, a camera unit mounted on a high-function mobile phone (or tablet terminal) is configured using the multilayer image sensor 100.

(Modification 2)
In the embodiment described above, an example has been described in which the accumulation time for capturing a recording image after the release operation during still image shooting is controlled for each block 131. The control of the accumulation time for each block 131 is not limited to the time of still image shooting, and may be controlled when shooting a live view image or shooting a moving image. For example, when capturing an image of each frame when capturing a moving image, the accumulation time in each frame is controlled for each block 131.

  According to the second modification, it is possible to appropriately control the accumulation time in each frame even when shooting a live view image or a moving image.

(Modification 3)
In the above description, an example in which the accumulation time is divided into 8 stages has been described. However, the accumulation time may be divided into 4 stages or 16 stages, and may be changed as appropriate.

(Modification 4)
In the above description, the example in which the green pixel Gr (3, 4) is functioned as the monitoring sensor of the block 131 has been described. The monitoring sensor may be provided separately from the photodiode PD constituting the pixel.

(Modification 5)
In the above description, the configuration is made common between the green pixel Gr (3, 4) that functions as the monitoring sensor of the block 131 and the other pixels in the block 131. Instead, the pixels other than the green pixel Gr (3,4) that functions as the monitoring sensor are accumulated without dividing the accumulation time from time t1 to time t8 into a plurality of times, and the monitoring sensor (green pixel Gr ( The accumulation may be continued until the pixel signal level from 3, 4)) exceeds the determination threshold. In this case, readout and addition processing during accumulation are not required for pixels other than the green pixel Gr (3,4) that functions as a monitoring sensor.

(Modification 6)
As an example of selecting the green pixel Gr (3, 4) located at the approximate center in the block 131 as the pixel that functions as the monitoring sensor, the pixel position that functions as the monitoring sensor is not limited to the approximate center in the block 131. You may change suitably in the block 131. FIG.

(Modification 7)
In the above embodiment, an example in which the monitoring sensor is arranged in each block 131 has been described. However, a block 131 having no monitoring sensor may be provided. In this case, the accumulation time of the pixels in the block 131 without the monitoring sensor is determined based on the charge accumulation amounts by the plurality of monitoring sensors in the plurality of neighboring blocks 131 where the monitoring sensors are arranged. It may be determined based on the charge accumulation amount by the monitoring sensor in one adjacent block 131, or a plurality of monitoring in a plurality of adjacent blocks 131 (for example, eight or four blocks around the corresponding block). You may determine based on the charge accumulation amount by a sensor.

(Modification 8)
In the above description, an example in which the green pixel Gr functions as a monitoring sensor and represents other colors such as blue and red has been described (the green pixel may be Gr or Gb). Instead, pixels that function as monitoring sensors may be provided in different colors. That is, the green pixel Gr, the blue pixel B, and the red pixel R are caused to function as monitoring sensors, respectively.

  In the case of the modification 8, the accumulation time is determined for all the same color (green) pixels Gr and Gb included in the block 131 based on the charge accumulation amount by the monitoring sensor including the green pixel Gr. Further, the accumulation time is determined for all the same color (blue) pixels B included in the block 131 based on the charge accumulation amount by the monitoring sensor including the blue pixels B. Further, based on the charge accumulation amount by the monitoring sensor including the red pixel R, the accumulation time is determined for all the same color (red) pixels R included in the block 131.

  The normalization process in the modification 8 is performed separately for the green pixels Gr and Gb, the blue pixel B, and the red pixel R, respectively. Then, after performing the normalization process for each color, the adjustment of the pixel signal value for each color is performed as the white balance process.

  The above description is merely an example, and is not limited to the configuration of the above embodiment. You may combine suitably the structure of the said embodiment and each modification.

DESCRIPTION OF SYMBOLS 1 ... Imaging device 10 ... Imaging optical system 20 ... Imaging part 30 ... Image processing part 40 ... Work memory 50 ... Display part 51 ... Liquid crystal display panel 52 ... Touch panel 60 ... Recording part 70 ... Control part 71 ... AWB calculating part 100 ... Imaging Element 104 ... PD
111 ... signal processing chip 112 ... memory chip 113 ... imaging chip 131 ... block 413 ... demultiplexer 414 ... pixel memory 415 ... arithmetic circuit 416 ... adder 417 ... drive controller

Claims (11)

An imaging unit in which a plurality of photoelectric conversion units are arranged;
A control unit that divides the imaging unit into a plurality of area blocks including a plurality of the photoelectric conversion units, controls an accumulation time of the photoelectric conversion units in units of the area blocks, and reads out an accumulation signal in units of the area blocks; ,
A first monitoring sensor and a second monitoring sensor, which are arranged in at least the first area block and the second area block, respectively, and capable of reading the charge accumulation amount by the photoelectric conversion unit;
An imaging apparatus comprising: The imaging device according to claim 1,
The first monitoring sensor and the second monitoring sensor are configured by the photoelectric conversion unit in which a green color filter is arranged in the first area block and the second area block. apparatus. The imaging device according to claim 1 or 2,
The imaging device, wherein the first monitoring sensor and the second monitoring sensor are arranged at substantially the center of the first area block and the second area block. In the imaging device according to any one of claims 1 to 3,
The control unit terminates the charge accumulation of the photoelectric conversion unit included in the corresponding first area block when the charge accumulation amount by the first monitoring sensor reaches a predetermined accumulation amount. An imaging device. The imaging apparatus according to claim 4,
The control unit terminates the charge accumulation of the photoelectric conversion unit included in the corresponding second area block when the charge accumulation amount by the second monitoring sensor reaches a predetermined accumulation amount. An imaging device. In the imaging device according to claim 4 or 5,
The image pickup apparatus, wherein the control unit reads out an accumulation signal from the photoelectric conversion unit that has completed the charge accumulation in units of area blocks. The imaging apparatus according to claim 4,
The image pickup apparatus, wherein the control unit starts reading out the charge accumulation amount from the first monitoring sensor and the second monitoring sensor in response to a release operation. The imaging apparatus according to claim 5,
The control unit, for a third area block that does not include the first monitoring sensor and the second monitoring sensor, based on a charge accumulation amount by at least one of the first monitoring sensor and the second monitoring sensor. An image pickup apparatus, wherein charge accumulation in the photoelectric conversion unit included in an area block is terminated. The imaging device according to claim 1,
The first monitoring sensor and the second monitoring sensor are configured by a plurality of the photoelectric conversion units in which color filters of different colors are arranged in the first area block and the second area block. An imaging device. The imaging device according to claim 9,
The control unit has the same color as the color filter of the first monitoring sensor among the photoelectric conversion units included in the first area block when the charge accumulation amount by the first monitoring sensor reaches a predetermined accumulation amount. An image pickup apparatus, wherein charge accumulation in the photoelectric conversion unit provided with the color filter is terminated. An imaging device in which a plurality of photoelectric conversion units are arranged in an imaging unit,
The accumulation time of the photoelectric conversion unit can be controlled in units of a plurality of area blocks in which the imaging unit is divided so as to include a plurality of the photoelectric conversion units,
The accumulated signal of the photoelectric conversion unit can be read in the area block unit,
A first monitoring sensor and a second monitoring sensor are disposed in each of at least the first area block and the second area block,
The charge accumulation amount obtained by the photoelectric conversion unit of the first area block and the charge accumulation amount obtained by the photoelectric conversion unit of the second area block can be read from the first monitoring sensor and the second monitoring sensor, respectively. An image pickup device characterized by being made.

Images