JP2022067733A - Imaging element - Google Patents

Abstract

To appropriately perform focus detection according to a split-pupil phase difference method, in pixels arranged at a position where an image height of a photoelectric conversion region is high.SOLUTION: In an imaging element, a plurality of pixels including a microlens and a first photoelectric conversion unit and a second photoelectric conversion unit and configured to perform photoelectric conversion of first and second light fluxes passing through first and second regions of a pupil of an optical system, respectively, are arranged in a region into which light from the optical system enters. The pixel includes a first pixel in which the microlens and the first and second photoelectric conversion units have a first positional relationship corresponding to a position of a first exit pupil of the optical system, and a second pixel in which the microlens and the first and second photoelectric conversion units have a second positional relationship corresponding to a position of a second exit pupil of the optical system. A plurality of first pixel rows with a plurality of the first pixels in a first direction are formed in a second direction, and a second pixel row with a plurality of the second pixels in the first direction is arranged in place of some of the first pixel rows.SELECTED DRAWING: Figure 3

Description

The present invention relates to an image pickup device.

A technique is known in which pixels provided with two photoelectric conversion units are arranged in a photoelectric conversion region to perform focal detection by a pupil division phase difference method (see Patent Document 1). In such a technique, the detection accuracy is deteriorated in the pixel arranged at the position where the image height of the photoelectric conversion region is high.

Japanese Patent Application Laid-Open No. 2002-314062

The image pickup device according to one aspect of the invention includes a microlens, a first photoelectric conversion unit, and a second photoelectric conversion unit, and the first and second regions of the pupil of the optical system have passed through the first and second regions. A plurality of pixels for photoelectric conversion of each of the light beams are arranged in a region where light from the optical system is incident, and the pixels are such that the microlens and the first and second photoelectric conversion units are in the optical system. The first pixel having the first positional relationship corresponding to the position of the first ejection pupil of the optical system, the microlens, and the first and second photoelectric conversion units are the second ejection pupils of the optical system. A plurality of first pixel rows in which a plurality of the first pixels are arranged in the first direction are formed in the second direction, including a second pixel having a second positional relationship corresponding to the position, and the second pixel. A plurality of second pixel rows arranged in the first direction are arranged in place of a part of the first pixel row.

It is a block diagram which illustrates the camera by 1st Embodiment. It is a figure which illustrates the schematic structure of the image pickup device. It is a figure which illustrates the pixel arrangement of the image pickup element. It is a circuit diagram explaining a pixel. It is a figure which illustrates the signal of a-series and b-series. It is a figure explaining the photoelectric conversion area of an image sensor. It is sectional drawing of the image pickup element for demonstrating the focal detection light flux by the pupil division phase difference method. It is sectional drawing of the image pickup element for demonstrating the focal detection light flux by the pupil division phase difference method. It is a figure explaining the region which provides the short-distance correspondence ML row and the long-distance correspondence ML row in the modification 1 of the 1st Embodiment. It is a figure explaining the region which provides the short-distance correspondence ML row and the long-distance correspondence ML row in the modification 2 of the 1st Embodiment. FIG. 5 is an enlarged view of a pixel microlens arranged at a position where the image height of the image sensor according to the second embodiment is high. FIG. 5 is an enlarged view of a pixel microlens arranged at a position where the image height of the image sensor according to the second embodiment is high. FIG. 5 is an enlarged view of a pixel microlens arranged at a position where the image height of the image sensor is high according to the third modification of the second embodiment. FIG. 5 is an enlarged view of a pixel microlens arranged at a position where the image height of the image sensor is high according to the third modification of the second embodiment. It is a figure explaining the 1st aspect of the arrangement of the short-distance correspondence ML row and the long-distance correspondence ML row. It is a figure explaining the 2nd mode of arrangement of a short-distance correspondence ML row and a long-distance correspondence ML row. It is a figure explaining the 3rd aspect of the arrangement of the short-distance correspondence ML row and the long-distance correspondence ML row. It is a figure explaining the 4th aspect of the arrangement of the short-distance correspondence ML row and the long-distance correspondence ML row. It is a figure explaining the 5th aspect of the arrangement of the short-distance correspondence ML row and the long-distance correspondence ML row. It is a figure explaining the sixth aspect of the arrangement of the short-distance correspondence ML row and the long-distance correspondence ML row. It is a figure explaining the 7th aspect of the arrangement of the short-distance correspondence ML row and the long-distance correspondence ML row. It is a figure explaining the 8th aspect of the arrangement of the short-distance correspondence ML row and the long-distance correspondence ML row.

(First Embodiment)
FIG. 1 is a block diagram illustrating a digital camera 1 (hereinafter referred to as a camera 1) equipped with a focus detection device using a solid-state image sensor 3 (hereinafter referred to as an image sensor 3) according to the first embodiment. ..
In the image pickup device 3, pixels having a plurality of photoelectric conversion units for one microlens are arranged over the entire photoelectric conversion region in which the image pickup element 3 generates an image. Then, the photoelectric conversion signal read out for each pixel is used for the focus detection of the pupil division phase difference method.

Generally, in the central portion of the photoelectric conversion region, light is incident substantially perpendicular to the photoelectric conversion portion, whereas the peripheral portion located outside the central portion of the photoelectric conversion region (the region having a higher image height than the central portion). Then, the light is incident on the photoelectric conversion unit at an angle. For this reason, pixels suitable for obliquely incident light are provided in advance in the peripheral portion of the photoelectric conversion region.

However, due to the characteristics of the photographing lens used in the camera 1, when the distance from the exit pupil to the image pickup element 3 changes, the angle of the light incident obliquely to the peripheral portion changes. Therefore, in the present embodiment, a plurality of pixels suitable for obliquely incident light are provided in the peripheral portion of the photoelectric conversion region in correspondence with a plurality of different photographing lenses. This makes it possible to appropriately perform the focal detection of the pupil division phase difference method in the peripheral portion of the photoelectric conversion region even when the characteristics of the photographing lens are different.

Hereinafter, the description will be described in detail with reference to the drawings. In the first embodiment, an interchangeable lens type camera 1 such as a single-lens reflex type or a mirrorless type will be described as an example, but the camera may not be an interchangeable lens type camera. For example, it may be configured as an image pickup device such as a lens-integrated camera provided with a zoom lens or a camera mounted on a mobile terminal such as a smartphone. Further, the present invention is not limited to a still image, and may be configured as an image pickup device such as a video camera or a mobile camera for capturing a moving image.

<Camera configuration>
A photographing lens 2 is attached to the camera 1 as an imaging optical system. The photographing lens 2 has a focusing lens and an aperture. The focusing lens and the aperture of the photographing lens 2 are controlled by the lens control unit 2a instructed by the microprocessor 9. The photographing lens 2 forms an optical image (subject image) on the image pickup surface of the image pickup element 3. The photographing lens 2 is also referred to as an imaging optical system.

The image pickup device 3 has a plurality of pixels. As will be described later, each of the plurality of pixels has two photoelectric conversion units that photoelectrically convert the incident light transmitted through the photographing lens 2 to generate an electric charge. The two photoelectric conversion units each output a signal based on the electric charge generated by the photoelectric conversion. The image pickup device 3 is controlled by the image pickup control unit 4 instructed by the microprocessor 9. The signals output from the plurality of pixels of the image pickup device 3 are processed via the signal processing unit 5 and the A / D conversion unit 6, and then temporarily stored in the memory 7. A lens control unit 2a, an image pickup control unit 4, a memory 7, a microprocessor 9, a focus calculation unit (focus detection processing unit) 10, a recording unit 11, an image compression unit 12, an image processing unit 13, and the like are connected to the bus 8. To.
The image pickup device 3 may have a laminated configuration including a part or all of the signal processing unit 5, the A / D conversion unit 6, and the memory 7. The image pickup device 3 may be configured such that at least one of the signal processing unit 5, the A / D conversion unit 6, and the memory 7 and a plurality of pixels are laminated.

An operation signal is input to the microprocessor 9 from an operation unit 9a such as a release button. The microprocessor 9 controls the camera 1 by sending an instruction to each block based on the operation signal from the operation unit 9a.

The focus calculation unit 10 calculates the focus adjustment state by the photographing lens 2 by the pupil division type phase difference detection method based on the signal from the pixel of the image sensor 3. In the focus calculation unit 10, the image taken by the photographing lens 2 is aligned on the image pickup surface of the image pickup element 3 based on the signals based on the charges generated by the first and second photodiodes S1 and S2 of the pixel 20 described later. Calculate the in-focus position of the focus lens for focusing. Specifically, the amount of image deviation due to a plurality of light fluxes passing through different regions of the pupil of the photographing lens 2 is detected, and the amount of defocus is calculated based on the detected amount of image deviation. The defocus amount is the amount of deviation between the image plane on which the image formed by the photographing lens 2 is formed and the image pickup surface of the image pickup element 3. Since the calculation of the defocus amount by the phase difference detection method is known, detailed description thereof will be omitted. The focus calculation unit 10 calculates the amount of movement of the focus lens to the in-focus position based on the calculated defocus amount.
The microprocessor 9 instructs the lens control unit 2a to move the focusing lens, and sends the calculated movement amount of the focus lens. As a result, the focus adjustment is automatically performed. The focus calculation unit 10, the microprocessor 9, and the lens control unit 2a operate as a focus adjustment unit.

The image processing unit 13 performs predetermined image processing on the signal from the image pickup element 3 stored in the memory 7 to generate image data. The image processing unit 13 operates as an image generation unit. The image compression unit 12 compresses the image data after image processing in a predetermined format. The recording unit 11 records the compressed image data in a predetermined file format on the recording medium 11a, and reads out the image data recorded on the recording medium 11a. The recording medium 11a is composed of a memory card or the like that can be attached to and detached from the recording unit 11.

Further, the image processing unit 13 generates image data for displaying the image on the display unit 14. The display unit 14 displays an image based on the image data generated by the image processing unit 13. The images displayed by the display unit 14 include reproduced images (still images, moving images) based on the image data recorded on the recording medium 11a, and monitor images acquired by the image sensor 3 at predetermined intervals (for example, 60 fps). (Also known as a live view image) is included.

<Overview of image sensor>
FIG. 2 is a diagram illustrating a schematic configuration of the image pickup device 3. The image pickup device 3 has a plurality of pixels 20 arranged in a matrix and a peripheral circuit for outputting a signal from each pixel 20. Generally, the smallest unit constituting an image is referred to as a "pixel", but in the present embodiment, a configuration for generating a signal of the smallest unit constituting an image is referred to as a "pixel".
The photoelectric conversion region 31 indicates a region in which the pixels 20 are arranged in a matrix. In the example of FIG. 2, the range of 16 pixels of 4 rows horizontally × 4 columns vertically is illustrated as the photoelectric conversion region 31, but the actual number of pixels is much larger than that illustrated in FIG.

FIG. 3 is a diagram illustrating the pixel arrangement of the image pickup device 3. The pixel 20 is provided with a microlens ML and a color filter (not shown). As the color filter, one of three color filters having different spectral characteristics of, for example, R (red), G (green), and B (blue) is provided for each pixel 20. The R color filter mainly transmits light in the red wavelength range. Further, the G color filter mainly transmits light in the green wavelength range. Further, the color filter B mainly transmits light in the blue wavelength range. The G color filter transmits light in a wavelength region having a shorter wavelength than the R color filter. The color filter B transmits light in a wavelength region having a shorter wavelength than the color filter G. As a result, each pixel 20 has different spectral characteristics depending on the arranged color filter.

The image pickup element 3 has a pixel row (also referred to as an RG row) in which pixels 20 having R and G color filters (hereinafter, referred to as pixel 20R and pixel 20G, respectively) are alternately arranged, and a color filter of G and B. Pixel rows (also referred to as GB rows) in which pixels 20 (hereinafter, referred to as pixels 20G and pixels 20B, respectively) having the above are alternately arranged are repeatedly arranged in a two-dimensional manner. In the first embodiment, the pixels 20R, the pixels 20G, and the pixels 20B are arranged according to the Bayer arrangement.
In the following description, when the pixel 20 is referred to without R, G, and B, it is assumed that the pixel 20R, the pixel 20G, and the pixel 20B are all included.

Each pixel 20 is provided with two photoelectric conversion units. Generally, when two photoelectric conversion units are provided per pixel, when the two photoelectric conversion units are arranged in the horizontal direction, that is, in the row direction (also referred to as horizontal division), the two photoelectric conversion units are in the vertical direction, that is, There are cases where they are lined up in the column direction (also called vertical division). In the first embodiment, the horizontally divided pixels 20 are arranged over the entire area of the photoelectric conversion region 31. However, the vertically divided pixels 20 may be arranged in the predetermined area instead of the horizontally divided pixels 20.

In FIG. 3, one of the two photoelectric conversion units of each pixel 20 is hatched and shown. The hatched photoelectric conversion unit indicates that a first signal, which will be described later, is generated based on the electric charge generated by the photoelectric conversion unit. Further, the non-hatched photoelectric conversion unit indicates that a second signal, which will be described later, is generated based on the electric charge generated by the photoelectric conversion unit.
Each pixel 20 performs photoelectric conversion in two photoelectric conversion units according to a control signal from a peripheral circuit, and outputs a signal based on the electric charge generated by the photoelectric conversion unit.

This will be described again with reference to FIG. Peripheral circuits include, for example, a vertical scanning circuit 21, a horizontal scanning circuit 22, control signal lines 23 and 24 connected thereto, first and second vertical signal lines 25a and second vertical signal lines 25a that receive signals from the pixel 20. The 25b, the constant current source 26 connected to the first and second vertical signal lines 25a and 25b, the correlated double sampling circuit (CDS circuit) 27, and the horizontal signal line that receives the signal output from the CDS circuit 27. It is composed of 28 and an output amplifier 29 and the like. In the present embodiment, two vertical signal lines, that is, first and second vertical signal lines 25a and 25b are provided for one pixel row composed of a plurality of pixels 20 arranged in the row direction.

The vertical scanning circuit 21 and the horizontal scanning circuit 22 output a predetermined control signal according to an instruction from the image pickup control unit 4. Each pixel 20 is driven by a control signal output from the vertical scanning circuit 21 and outputs signals based on the charges generated by photoelectric conversion to the first and second vertical signal lines 25a and 25b. The signal output from the pixel 20 is noise-removed by the CDS circuit 27, and is output to the outside via the horizontal signal line 28 and the output amplifier 29 by the control signal from the horizontal scanning circuit 22.

<Structure with two photodiode PDs in one pixel>
FIG. 4 is a circuit diagram illustrating pixels 20 arranged in column M (arranged in the vertical direction) in FIG. 3, that is, for example, the pixel 20G in the Nth row and the pixel 20R in the N + 1th row. Each pixel 20 has two photodiodes S1 and S2 as photoelectric conversion units inside (behind) the microlens ML and the color filter. That is, each pixel 20 has a first photodiode S1 arranged on the left side of the pixel 20 and a second photodiode S2 arranged on the right side of the pixel 20.
Therefore, the light beam that has passed through the first region of the pupil of the photographing lens 2 is incident on the first photodiode S1 of each pixel 20, and the second photodiode S2 is the second of the pupil of the photographing lens 2. The luminous flux that has passed through the region of is incident.

In the present embodiment, for example, a reading unit that reads out a signal based on the charges generated by the first photodiode S1 and the second photodiode S2 and the first and second photodiodes S1 and S2 is included. Called a "pixel". An example in which the reading unit includes a transfer transistor, an FD region, an amplification transistor, and a selection transistor, which will be described later, will be described, but the range of the reading unit does not necessarily have to be as in this example.

In each pixel 20, as described above, the first and second photodiodes S1 and S2 are incident with light that has passed through different regions of the pupil of the photographing lens 2, that is, the first and second regions, respectively. .. The first and second photodiodes S1 and S2 photoelectrically convert incident light to generate electric charges, respectively. The electric charge generated by the first photodiode S1 is transferred to the first FD (floating diffusion) region FD1 via the first transfer transistor Tx-1. Further, the electric charge generated by the second photodiode S2 is transferred to the second FD region FD2 via the second transfer transistor Tx-2.

The first FD region FD1 accumulates the received charge and converts it into a voltage. The signal corresponding to the potential of the first FD region FD1 is amplified by the first amplification transistor AMP1. The first FD region FD1 and the first amplification transistor AMP1 operate as a first signal generation unit. Then, the generated signal is read out via the first vertical signal line 25a (output unit) as the signal of the row selected by the first selection transistor SEL1 for selecting the row. The first reset transistor RES1 operates as a reset unit for resetting the potential of the first FD region FD1. In the present embodiment, the signal generated based on the charge generated by the first photodiode S1 is referred to as a first signal.

Similarly, the second FD region FD2 stores the received charge and converts it into a voltage. The signal corresponding to the potential of the second FD region FD2 is amplified by the second amplification transistor AMP2. The second FD region FD2 and the second amplification transistor AMP2 operate as a second signal generation unit. Then, the generated signal is read out via the second vertical signal line 25b (output unit) as the signal of the row selected by the second selection transistor SEL2 for selecting the row. The second reset transistor RES2 operates as a reset unit for resetting the potential of the second FD region FD2. In the present embodiment, the signal generated based on the charge generated by the second photodiode S2 is referred to as a second signal.

The first signal based on the charge generated by the first photodiode S1 is read out via the first vertical signal line 25a, and the second signal based on the charge generated by the second photodiode S2 is read. Although the case where the signal is read out via the second vertical signal line 25b is illustrated, the first photodiode S1 is used by using a common FD region between the first photodiode S1 and the second photodiode S2. The first signal based on the charge generated by the second photodiode S2 and the second signal based on the charge generated by the second photodiode S2 are combined into one vertical signal line, that is, the first vertical signal line 25a or the second vertical. It may be configured to read out in a time-divided manner via the signal line 25b.

<Control signal>
In the pixel 20, the first transfer transistor Tx-1 that transfers the electric charge generated by the first photodiode S1 is turned on by the first control signal φTx1. Further, the second transfer transistor Tx-2 that transfers the electric charge generated by the second photodiode S2 is turned on by the second control signal φTx2.
The row selection first selection transistor SEL1 for outputting the first signal to the first vertical signal line 25a is turned on by the first control signal φSEL1. The first reset transistor RES1 that resets the potential of the first FD region FD1 is turned on by the first control signal φRES1.
Further, the second selection transistor SEL2 for row selection for outputting the second signal to the second vertical signal line 25b is turned on by the second control signal φSEL2. The second reset transistor RES2 that resets the potential of the second FD region FD2 is turned on by the second control signal φRES2.

<Focus adjustment>
In the camera 1 of the present embodiment, a pair of focus detection signals used for focus detection (detection of the in-focus position) are arranged alternately, for example, in the pixel rows corresponding to the focus area, the pixels 20G and the pixels 20B. It is generated based on the first signal and the second signal of the pixel 20G read from the pixel row (GB row). The focus area is an area in which the focus calculation unit 10 detects the amount of image shift as phase difference information, and is also referred to as a focus detection area, a focusing point, or an autofocus (AF) point.

For example, when an operation signal indicating that the release button has been half-pressed is input from the operation unit 9a, the microprocessor 9 instructs the image pickup control unit 4 to take an image for focus adjustment. In the image pickup for focus adjustment, the control signal for focus adjustment is supplied from the vertical scanning circuit 21 and the horizontal scanning circuit 22 to the pixel row to be read out by the image pickup element 3, and the reading for focus adjustment is performed. In the reading for focus adjustment, the first control signal φTx1 or the like is supplied to each pixel 20G of the pixel row, and the first signal based on the charge generated by the first photodiode S1 is read from the pixel 20G, and the pixel row is read. A second control signal φTx2 or the like is supplied to each pixel 20G of the above, and a second signal based on the charge generated by the second photodiode S2 is read out from the pixel 20G.

By reading for focus adjustment, the first signal and the second signal read from each pixel 20G are stored in the memory 7. A plurality of first signals A1, A2, ..., An (referred to as a-series signals) stored in the memory 7, and a plurality of second signals B1, B2, ..., Bn (b-series signals) stored in the memory 7. (Referred to as a signal) represents the intensity distribution of an image due to a plurality of light fluxes passing through different regions of the pupil of the photographing lens 2.

FIG. 5 is a diagram illustrating an a-series signal composed of a plurality of first signals and a b-series signal composed of a plurality of second signals. In FIG. 5, n a-series signals are represented by hatched circles. In addition, n b-series signals are represented by white circles. The a-series signal and the b-series signal from the pixel 20G are read out every other row in FIG. The vertical dashed line in FIG. 5 corresponds to the pixel sequence.
The focus calculation unit 10 calculates the amount of image shift of a plurality of images by performing image shift detection calculation processing (correlation calculation processing, phase difference detection processing) based on the a-series signal and the b-series signal. The defocus amount is calculated by multiplying the image shift amount by a predetermined conversion coefficient.

Next, the microprocessor 9 determines whether or not the defocus amount calculated by the focus calculation unit 10 is within the allowable value. When the defocus amount exceeds the permissible value, the microprocessor 9 determines that the subject is out of focus, and sends a lens drive instruction to the lens control unit 2a. The lens control unit 2a moves the focusing lens to a position (focusing position) where the defocus amount is within the allowable value. On the other hand, if the defocus amount is within the allowable value, the microprocessor 9 determines that the subject is in focus and does not send a lens drive instruction.

In the camera 1 described above, the focus is adjusted based on the first signal and the second signal of the pixel 20G read from the pixel row (GB row) in which the pixels 20G and the pixels 20B are alternately arranged. Image data was generated. The image data used for the focus adjustment is not limited to the first signal and the second signal by the pixel 20G, and may be generated based on the first signal and the second signal by the pixel 20B. Further, image data used for focus adjustment based on the first signal and the second signal of the pixel 20G or the pixel 20R read from the pixel row (RG row) in which the pixel 20R and the pixel 20G are alternately arranged is used. May be generated.

<Generation of image data>
The camera 1 of the present embodiment generates image data related to a subject image based on the first signal and the second signal read from each pixel 20 of the photoelectric conversion region 31 (FIG. 2). For example, when an operation signal indicating that the release button has been fully pressed is input from the operation unit 9a, the microprocessor 9 instructs the image pickup control unit 4 to take an image for recording. In the imaging for recording, the control signal for the image is supplied from the vertical scanning circuit 21 and the horizontal scanning circuit 22 to each pixel row of the image pickup element 3, and the image is read out. For reading for an image, a first control signal φTx1 or the like is supplied to each pixel 20 of the pixel row, and the first signal based on the charge generated by the first photodiode S1 is read from the pixel 20 and the pixel row is read. It means supplying a second control signal φTx2 or the like to each pixel 20 and reading out a second signal based on the charge generated by the second photodiode S2 from the pixel 20.

The first signal and the second signal read from each pixel 20 are stored in the memory 7 by reading for an image. The image processing unit 13 adds the first signal and the second signal stored in the memory 7 for each pixel 20 to generate a signal for an image, and further performs gradation processing, color interpolation processing, and the like to obtain a subject image. Generate image data for.

<Scaling>
In the present embodiment, the pixels 20 are arranged at equal intervals in the photoelectric conversion region 31. That is, the spacing in the row direction and the column direction of the first and second photodiodes S1 and S2 of the pixel 20 is evenly spaced over the entire photoelectric conversion region 31. By arranging them at equal intervals, the wiring positions and wiring densities around the first and second photodiodes S1 and S2 can be made equal in each pixel 20, so that the electrical characteristics (for example, parasitics) of each pixel 20 can be made equal. Variations in capacitance, etc.) can be suppressed.

On the other hand, in the present embodiment, the center position of the microlens ML that collects light on the pixel 20 is shifted from the center position of the pixel 20 depending on the position of the pixel 20 arranged in the photoelectric conversion region 31. The amount of shift is increased as the image height of the position where the pixels 20 are arranged is higher (as the distance from the center of the photoelectric conversion region 31 increases). In this example, the center positions of the first and second photodiodes S1 and S2 are set as the center positions of the pixel 20. Gradually changing the amount of deviation between the center position of the microlens ML and the center position of the pixel 20 in the photoelectric conversion region 31 is called scaling. The amount of deviation between the center position of the pixel 20 and the center position of the microlens ML is changed depending on the position of the pixel 20 through different regions of the pupil of the photographing lens 2, that is, the first and second regions of the pupil. This is to allow light to be appropriately incident on the first and second photodiodes S1 and S2. This will be described in more detail with reference to FIG.

FIG. 6 is a diagram illustrating a photoelectric conversion region 31 of the image pickup device 3. As described above, the pixels 20 are arranged at equal intervals over the entire photoelectric conversion region 31. Then, the center position of the microlens ML is set with respect to the center position of the pixel 20 by scaling suitable for the light incident through the photographing lens 2 having standard characteristics (for example, the focal length f = 50 mm in the 35 mm format). Arrange them in a staggered manner. In the example of FIG. 6, the direction of shifting the microlens ML is the row direction (horizontal direction), and is the direction toward the center of the photoelectric conversion region 31. In the present embodiment, the pixel row in which the microlens ML is arranged in a staggered manner according to the standard characteristics is referred to as a medium-distance corresponding ML row. By configuring the pixel rows in the entire photoelectric conversion region 31 with medium-distance compatible ML rows, the light that has passed through the first and second regions of the pupil of the photographing lens 2 having standard characteristics is emitted in all the pixels 20. , Is properly incident on the first and second photodiodes S1 and S2.

In the present embodiment, even when the photographing lens 2 having characteristics different from the standard characteristics is used, the light passing through the first and second regions of the pupil of the photographing lens 2 is the first and second photodiodes S1. And, a pixel row in which the microlens ML is arranged is provided by scaling different from the medium-distance corresponding ML row so as to be appropriately incident on S2. Specifically, for example, in a region of 30% of the left and right ends of the photoelectric conversion region 31, that is, a part of a region having a high image height, a short-distance correspondence ML row and a long-distance correspondence ML row are used instead of the medium-distance correspondence ML row. Is provided. In the present embodiment, for example, a pixel row in which the microlens ML is arranged in a staggered manner in accordance with the characteristics of a wide-angle lens having a focal length shorter than the focal length f = 50 mm is referred to as a short-distance corresponding ML row. Further, a pixel row in which the microlens ML is arranged in a staggered manner in accordance with the characteristics of a telephoto lens having a focal length longer than the focal length f = 50 mm is referred to as a long-distance compatible ML row.

The lower part of FIG. 6 shows an enlarged view of the microlens ML of the pixel 20 arranged in the peripheral portion of the photoelectric conversion region 31, for example, the region 31b at a position where the image height is high. In the present embodiment, since the pixel rows (GB rows) in which the pixels 20G and the pixels 20B are alternately arranged are used for focus detection, a part of the GB rows is replaced with a short-distance corresponding ML row or a long-distance corresponding ML row. .. In the enlarged view, the upper two rows and the lower two rows with the short-distance corresponding ML row in between are the middle-distance corresponding ML row, respectively. In the medium-distance corresponding ML line, the distance between the line CS passing through the center of the pixel 20 and the line CL (middle) passing through the center of the microlens ML is G. That is, the amount of deviation of the center position of the microlens ML with respect to the center position of the pixel 20 is G.
In FIG. 6, the pixel 20G is shown in white, the pixel 20B is shown by an oblique line, and the pixel 20R is shown by hatching.

On the other hand, in the short-distance corresponding ML line, the distance between the line CS passing through the center of the pixel 20 and the line CL (short) passing through the center of the microlens ML is Gs. That is, the amount of deviation of the center position of the microlens ML with respect to the center position of the pixel 20 is Gs. The shift amount Gs in the short-distance corresponding ML row is larger than the shift amount G in the medium-distance corresponding ML row so that G <Gs is satisfied. The reason for this will be described with reference to FIGS. 7 and 8.

<Central part of the image sensor>
FIG. 7 is a cross-sectional view of a medium-distance corresponding ML row of the image pickup device 3 for explaining the focal detection light flux by the pupil division phase difference method. FIG. 7 shows a case where light is incident substantially perpendicular to the pixel 20 in the central portion of the photoelectric conversion region 31 of FIG. 6, that is, the region 31a at a position where the image height is low. In FIG. 7, five pixels 20Ga, 20Bb, 20Gc, 20Bd, and 20Ge out of the pixels 20 in the middle-distance corresponding ML row are exemplified. The pixels 20Ga to 20Ge are the microlenses MLa to MLe, the first photodiode S1a, the second photodiode S2a, the first photodiode S1b, the second photodiode S2b, and the first photodiode S1c, respectively. It has a second photodiode S2c, a first photodiode S1d, a second photodiode S2d, a first photodiode S1e, and a second photodiode S2e.

In the central portion of the photoelectric conversion region 31, light is incident substantially perpendicular to the pixel 20. Therefore, the center position of the microlens ML is arranged so as to be aligned with the center positions of the first and second photodiodes S1 and S2. That is, it is not necessary to shift the center position of the microlens ML with respect to the center position of the pixel 20. Further, it is not necessary to provide a short-distance correspondence ML row and a long-distance correspondence ML row in the central portion of the photoelectric conversion region 31 in which light is incident substantially perpendicular to the pixel 20. According to FIG. 7, the line CS passing through the center of the pixel 20 and the line CL (middle) passing through the center of the microlens ML coincide with each other.

Further, the planar shapes of the first photodiodes S1a, S1b, S1c, S1d, and S1e are such that the first photodiodes S1a, S1b are separated from the microlenses MLa to MLe by the distance measuring pupil distance d. , S1c, S1d, and S1e are all projected onto the region 91 common to all of the, S1c, S1d, and S1e.

Further, the planar shapes of the second photodiodes S2a, S2b, S2c, S2d, and S2e are such that the second photodiodes S2a, S2b are separated from the microlenses MLa to MLe by the distance measuring pupil distance d. , S2c, S2d, and S2e are all projected onto the region 92 common to all of the, S2c, S2d, and S2e. The region 91 and the region 92 are symmetrical with respect to the line Ax passing through the optical axis. In this case, the light that has passed through the first and second regions 91 and 92 on the distance measuring pupil surface 90 is appropriately incident on the first and second photodiodes S1 and S2 of the pixel 20.

The pair of regions 91 and 92 on the distance measuring pupil surface 90 on which the plane shape is projected are referred to as distance measuring pupils. For the sake of simplicity, it is assumed that the distance measuring pupil surface 90 is substantially at the same position as the exit pupil surface of the optical system of the photographing lens 2. That is, it is considered that the distance measuring pupil distance d from the microlenses MLa to MLe to the distance measuring pupil surface 90 is substantially equal to the exit pupil distance from the microlenses MLa to MLe to the exit pupil surface of the optical system of the photographing lens 2. .. Therefore, the light that has passed through the first and second regions 91 and 92 on the ranging pupil surface 90 corresponds to the light that has passed through the first and second regions of the pupil of the photographing lens 2.

Taking the pixels 20Gc and 20Bd of the pixels 20Ga to 20Ge as an example, the pixel 20Gc receives the focal detection luminous flux 81 passing through the distance measuring pupil 91 and the microlens MLc by the first photodiode S1c. At the same time, the second photodiode S2c receives the focal detection luminous flux 82 passing through the ranging pupil 92 and the microlens MLc.

Further, the pixel 20Bd receives the focus detection luminous flux 71 passing through the distance measuring pupil 91 and the microlens MLd by the first photodiode S1d, and the distance measuring pupil 92 and the microlens by the second photodiode S2d. The focus detection luminous flux 72 passing through the MLd is received.

The same applies to the pixels 20Ga, 20Bb, and 20Ge other than the pixels 20Gc and 20Bd in FIG. 7. As described above, in the central portion of the photoelectric conversion region 31, the light that has passed through the first and second regions of the pupil of the photographing lens 2 without shifting the center position of the microlens ML with respect to the center position of the pixel 20. The focal detection light flux that has passed through the corresponding distance measuring pupils 91 and 92 is appropriately incident on the first and second photodiodes S1 and S2.

As a result, in the medium-distance correspondence ML row, the first photodiode S1 in the plurality of pixels 20 outputs the first signal corresponding to the intensity of the image formed by the focal detection light flux corresponding to the distance measuring pupil 91, respectively. .. Further, the second photodiode S2 in the plurality of pixels 20 outputs a second signal corresponding to the intensity of the image formed by the focal detection light flux corresponding to the distance measuring pupil 92, respectively. As described above, the pupil division for focus detection can be performed with high accuracy.

<Peripheral part of image sensor>
FIG. 8 is a cross-sectional view of a medium-distance corresponding ML row of the image pickup device 3 for explaining the focal detection light flux by the pupil division phase difference method. FIG. 8 shows a case where light is obliquely incident on the pixel 20 of the region 31b where the image height is higher than the peripheral portion of the photoelectric conversion region 31 of FIG. 6, that is, the central portion. In FIG. 8, five pixels 20Gg to 20Gk out of the pixels 20 in the ML row corresponding to the middle distance are exemplified. The pixels 20Gg to 20Gk are the microlenses MLg to MLk, the first photodiode S1g, the second photodiode S2g, the first photodiode S1h, the second photodiode S2h, and the first photodiode S1i, respectively. It has a second photodiode S2i, a first photodiode S1j, a second photodiode S2j, a first photodiode S1k, and a second photodiode S2k.

In the peripheral portion of the photoelectric conversion region 31, light is obliquely incident on the pixel 20. Therefore, the center position of the microlens ML is displaced from the center positions of the first and second photodiodes S1 and S2. That is, the amount of deviation between the line CS passing through the center of the pixel 20 and the line CL (middle) passing through the center of the microlens ML is G.

According to FIG. 8, the planar shape of the first photodiodes S1g, S1h, S1i, S1j, and S1k is the first photo on the ranging pupil surface 90 separated from the microlenses MLg to MLk by the ranging pupil distance d. It is projected onto the region 91 common to all of the diodes S1g, S1h, S1i, S1j, and S1k.

Further, the planar shapes of the second photodiodes S2g, S2h, S2i, S2j, and S2k are such that the second photodiodes S2g and S2h are separated from the microlenses MLg to MLk by the distance measuring pupil distance d. , S2i, S2j, and S2k are all projected onto the region 92 common to all of the, S2i, S2j, and S2k. The region 91 and the region 92 are symmetrical with respect to the line Ax passing through the optical axis. In this case, the light that has passed through the first and second regions 91 and 92 on the distance measuring pupil surface 90 is appropriately incident on the first and second photodiodes S1 and S2 of the pixel 20.

Similar to the case of FIG. 7, the pair of regions 91 and 92 on the distance measuring pupil surface 90 on which the plane shape is projected are referred to as distance measuring pupils. Further, for ease of explanation, the exit pupil distance d from the microlenses MLg to MLk to the distance measuring pupil surface 90 is the exit pupil from the microlenses MLg to MLk to the exit pupil surface of the optical system of the photographing lens 2. Considered to be substantially equal to the distance. Therefore, the light that has passed through the first and second regions 91 and 92 on the ranging pupil surface 90 corresponds to the light that has passed through the first and second regions of the pupil of the photographing lens 2.

Taking pixels 20Bh and 20Gi among the pixels 20Gg to 20Gk as an example, the pixel 20Bh receives the focal detection luminous flux 171 passing through the ranging pupil 91 and the microlens MLh by the first photodiode S1h. At the same time, the second photodiode S2h receives the focal detection luminous flux 172 passing through the ranging pupil 92 and the microlens MLh.

Further, the pixel 20Gi receives the focal detection light flux 181 passing through the distance measuring pupil 91 and the microlens MLi by the first photodiode S1i, and the distance measuring pupil 92 and the microlens by the second photodiode S2i. The focal detection light flux 182 passing through the MLi is received.

The same applies to the pixels 20Gg, 20Bj, and 20Gk other than the pixels 20Bh and 20Gi in FIG. 8. As described above, in the peripheral portion of the photoelectric conversion region 31, the center position of the microlens ML is shifted with respect to the center position of the pixel 20 in the medium-distance corresponding ML row, so that the first and second pupils of the photographing lens 2 are used. The focal detection light flux that has passed through the ranging pupils 91 and 92, which corresponds to the light that has passed through the region of the above, is appropriately incident on the first and second photodiodes S1 and S2.

As a result, in the medium-distance correspondence ML row, the first photodiode S1 in the plurality of pixels 20 outputs the first signal corresponding to the intensity of the image formed by the focal detection light flux corresponding to the distance measuring pupil 91, respectively. .. Further, the second photodiode S2 in the plurality of pixels 20 outputs a second signal corresponding to the intensity of the image formed by the focal detection light flux corresponding to the distance measuring pupil 92, respectively. As described above, the pupil division for focus detection can be performed with high accuracy.

Generally, the exit pupil distance from the microlenses MLg to MLk to the exit pupil surface of the optical system of the photographing lens 2 varies depending on the characteristics of the interchangeable lens 2. Therefore, when the distance measuring pupil distance d considered to be substantially equal to the exit pupil distance is shorter than that in the example of FIG. 8, light is incident on the pixel 20 at a further angle than in FIG. Therefore, as the short-distance corresponding ML row illustrated in FIG. 6, a pixel row in which the microlens ML is arranged with a shift amount Gs larger than the shift amount G in the medium-distance corresponding ML row is provided. In the short-distance correspondence ML line, the distance between the line CS passing through the center of the pixel 20 and the line CL (short) passing through the center of the microlens ML is Gs (G <Gs).

On the contrary, when the distance measuring pupil distance d is longer than the example of FIG. 8, light closer to vertical than that of FIG. 8 (close to the example of FIG. 7) is incident on the pixel 20. Therefore, as the long-distance corresponding ML row illustrated in FIG. 6, a pixel row in which the microlens ML is arranged with a shift amount GL smaller than the shift amount G in the medium-distance corresponding ML row is provided. Although not shown in the long-distance correspondence ML line, the distance between the line CS passing through the center of the pixel 20 and the line CL (length) passing through the center of the microlens ML is GL (GL <G).

When the distance measuring pupil distance d becomes shorter than the example of FIG. 8 by providing the short-distance correspondence ML row and the long-distance correspondence ML row, the first photo in the plurality of pixels 20 in the short-distance correspondence ML row. The diode S1 outputs a first signal corresponding to the intensity of the image formed by the focal detection light flux corresponding to the distance measuring pupil 91, respectively. Further, the second photodiode S2 in the plurality of pixels 20 outputs a second signal corresponding to the intensity of the image formed by the focal detection light flux corresponding to the distance measuring pupil 92, respectively. As described above, the pupil division for focus detection can be performed with high accuracy.

Further, when the distance measuring pupil distance d becomes long, in the long distance corresponding ML row, the first photodiode S1 in the plurality of pixels 20 is an image formed by the focal detection light flux corresponding to the distance measuring pupil 91, respectively. The first signal corresponding to the intensity of is output. Further, the second photodiode S2 in the plurality of pixels 20 outputs a second signal corresponding to the intensity of the image formed by the focal detection light flux corresponding to the distance measuring pupil 92, respectively. As described above, the pupil division for focus detection can be performed with high accuracy.

In the above description, in the region 31b located on the left side of the center of the photoelectric conversion region 31 in FIG. 6, the center position of the microlens ML is set to the left from the center positions of the first and second photodiodes S1 and S2. An example of arranging them in a staggered manner was explained. In the region 31d located on the right side of the center of the photoelectric conversion region 31 in FIG. 6, the center position of the microlens ML is arranged so as to be shifted to the right from the center positions of the first and second photodiodes S1 and S2. Although not shown, the shift amount G in the medium-distance corresponding ML row, the shift amount Gs in the short-distance corresponding ML row, and the shift amount GL in the long-distance corresponding ML row are the same as in the above-mentioned example. <Has a relationship of Gs.

According to the first embodiment described above, the following effects can be obtained.
(1) The image pickup element 3 includes a microlens ML, a first photodiode S1 and a second photodiode S2, and has passed through the first and second regions of the pupil of the photographing lens 2. A plurality of pixels 20 that photoelectrically convert the light beam of 2 are arranged in a region where the light from the photographing lens 2 is incident, and the pixels 20 are photographed by the microlens ML and the first and second photodiodes S1 and S2. It has a first positional relationship corresponding to the position of the first ejection pupil of the lens 2 (with respect to the center position of the pixel 20 by scaling suitable for light incident through the photographing lens 2 having standard characteristics). The first pixel (arranged so that the center position of the microlens ML is offset), the microlens ML, and the first and second photodiodes S1 and S2 correspond to the position of the second ejection pupil of the photographing lens 2. The first pixel is the first (arranged so that the center position of the microlens ML is shifted with respect to the center position of the pixel 20 due to scaling different from that of the medium-distance corresponding ML row). A plurality of medium-distance corresponding ML rows arranged in the horizontal) direction are formed in the second (vertical) direction, and a plurality of second pixels are arranged in the first direction. It is placed in place of a part of.
With this configuration, even when a photographing lens 2 having characteristics different from the standard characteristics is used, the light passing through the different regions of the pupil of the photographing lens 2, that is, the first and second regions of the pupil is shortened. It can be appropriately incident on the first and second photodiodes S1 and S2 in the distance-corresponding ML row. As a result, the pupil division for focus detection can be performed with high accuracy.

(2) The pixel 20 of the image sensor 3 has a third positional relationship in which the microlens ML and the first and second photodiodes S1 and S2 correspond to the position of the third ejection pupil of the photographing lens 2 (2). Further different scaling is used to shift the center position of the microlens ML with respect to the center position of the pixel 20). It is arranged in place of a part of the distance corresponding ML row, and has a medium distance corresponding ML row between the short distance corresponding ML row and the long distance corresponding ML row. Since it is configured in this way, even when the photographing lens 2 having further different characteristics is used, the light passing through the different regions of the pupil of the photographing lens 2, that is, the first and second regions of the pupil can be transmitted to the long-distance correspondence ML. It can be appropriately incident on the first and second photodiodes S1 and S2 in the row. As a result, the pupil division for focus detection can be performed with high accuracy.

(3) The length of the short-distance corresponding ML line and the long-distance corresponding ML line of the image sensor 3 is shorter than the length in the first (horizontal) direction of the region where the light from the photographing lens 2 is incident. Since it is configured in this way, the number of the second pixel and the third pixel arranged in place of the first pixel can be suppressed, and a large number of the first pixels can be left.

(4) The short-distance compatible ML line and the long-distance compatible ML line of the image sensor 3 have a region in which the distance from the center of the region in which the light from the photographing lens 2 is incident is farther than a predetermined value, in other words, the image height. Placed in a high area. With this configuration, especially in the region where light is likely to be obliquely incident on the pixel 20, the light that has passed through the first and second regions of the pupil of the photographing lens 2 is ML line or length corresponding to a short distance. It can be appropriately incident on the first and second photodiodes S1 and S2 in the distance-corresponding ML row. As a result, the pupil division for focus detection can be performed with high accuracy.

The following modifications are also within the scope of the present invention, and one or more of the modifications can be combined with the above-described embodiment.
(Modification 1)
In the first embodiment, as illustrated in FIG. 6, the length of the short-distance correspondence ML row and the long-distance correspondence ML row provided in place of a part of the medium-distance correspondence ML row is set to which pixel in the photoelectric conversion region 31. The line is also common. That is, in a part of the region of 30% at the left and right ends of the photoelectric conversion region 31, the medium-distance corresponding ML row was replaced with the short-distance corresponding ML row and the long-distance corresponding ML row. On the other hand, in the first modification of the first embodiment, the range (horizontal length) of the left and right end regions of the photoelectric conversion region 31 in which the short-distance corresponding ML row and the long-distance corresponding ML row are provided is changed. Let me.

FIG. 9 is a diagram illustrating a region at the left and right ends of a photoelectric conversion region 31 in which a short-distance ML row and a long-distance ML row are provided in place of the medium-distance ML row in the first modification of the first embodiment. be. In FIG. 9, the lengths of the short-distance corresponding ML row and the long-distance corresponding ML row, which are replaced with a part of the medium-distance corresponding ML row, are not unified in the photoelectric conversion region 31. For example, a medium-distance ML row may be replaced with a short-distance ML row and a long-distance ML row in a part of the region of 25% at the left and right ends of the photoelectric conversion region 31, or a region of 35% at the left and right ends of the photoelectric conversion region 31. In a part of, the medium-distance corresponding ML line is replaced with the short-distance corresponding ML line and the long-distance corresponding ML line.

As described above, when generating image data, the image processing unit 13 adds the first signal and the second signal read from each pixel 20 of the image pickup device 3 for each pixel 20 for an image. Generate a signal. That is, not only the medium-distance correspondence ML row but also the pixels 20 included in the short-distance correspondence ML row and the long-distance correspondence ML row are used for generating the signal for the image.

However, the first signal and the signal for the image based on the second signal photoelectrically converted by the pixel 20 included in the short-distance correspondence ML row are the first signals photoelectrically converted by the pixel 20 included in the medium-distance correspondence ML row. And when compared with the signal for the image based on the second signal, it does not exactly match.
Further, the signal for the image based on the first signal and the second signal photoelectrically converted by the pixel 20 included in the long-distance correspondence ML row is the first signal photoelectrically converted by the pixel 20 included in the medium-distance correspondence ML row. And when compared with the signal for the image based on the second signal, it does not exactly match.

From the above, if the lengths of the short-distance compatible ML row and the long-distance compatible ML row that replace the medium-distance compatible ML row in the photoelectric conversion region 31 are the same in any of the pixel rows of the photoelectric conversion region 31, the horizontal direction The boundary that changes from the medium-distance corresponding ML line to the short-distance corresponding ML line, or the boundary that changes from the medium-distance corresponding ML line to the long-distance corresponding ML line in the horizontal direction is based on the first signal and the second signal. It may be noticeable in the generated image.

However, according to the first modification of the first embodiment, the lengths of the short-distance correspondence ML row and the long-distance correspondence ML row that replace a part of the medium-distance correspondence ML row are not set in the photoelectric conversion region 31. Since it is unified, that is, irregular, the boundary that changes from the medium-distance corresponding ML line to the short-distance corresponding ML line in the horizontal direction, or the boundary that changes from the medium-distance corresponding ML line to the long-distance corresponding ML line in the horizontal direction is described above. It can be made inconspicuous in the images generated based on the first signal and the second signal.

As described above, according to the first modification of the first embodiment, the following effects can be obtained. That is, a plurality of short-distance corresponding ML rows of the image sensor 3 are arranged, and the arrangement of the short-distance corresponding ML rows adjacent to each other in the second (vertical) direction is the distance from the center of the region where the light from the photographing lens 2 is incident. Is different. Since it is configured in this way, the boundary that changes from the medium-distance corresponding ML line to the short-distance corresponding ML line in the first (horizontal) direction, and the change from the short-distance corresponding ML line to the medium-distance corresponding ML line in the first (horizontal) direction. The boundary can be made inconspicuous in the image generated based on the first signal and the second signal.

Similarly, a plurality of long-distance compatible ML rows of the image sensor 3 are arranged, and the arrangement of the long-distance corresponding ML rows adjacent to each other in the second (vertical) direction is from the center of the region where the light from the photographing lens 2 is incident. The distance is different. Since it is configured in this way, the boundary that changes from the medium-distance corresponding ML line to the long-distance corresponding ML line in the first (horizontal) direction, and the change from the long-distance corresponding ML line to the medium-distance corresponding ML line in the first (horizontal) direction. The boundary can be made inconspicuous in the image generated based on the first signal and the second signal.

(Modification 2)
In the second modification of the first embodiment, the line spacing of the short-distance corresponding ML line and the long-distance corresponding ML line, which are replaced with a part of the medium-distance corresponding ML line, is made irregular in the photoelectric conversion region 31. FIG. 10 is a diagram illustrating a region in which a short-distance correspondence ML row and a long-distance correspondence ML row are provided in place of the medium-distance correspondence ML row in the second modification of the first embodiment. In FIG. 10, the distance between the short-distance-compatible ML line and the long-distance-compatible ML line, which replaces a part of the medium-distance-compatible ML line, and the distance between the long-distance-compatible ML line and the short-distance-compatible ML line are set in the photoelectric conversion region. Arrange in the row direction (vertical direction) of 31.

For example, the number of medium-distance ML rows between the short-distance ML row and the long-distance ML row, and the number of medium-distance ML rows between the long-distance ML row and the short-distance ML row. , Each change irregularly. The irregular change means that the number of medium-distance ML rows between the long-distance ML row and the short-distance ML row is changed aperiodically in the column direction (vertical direction) of the photoelectric conversion region 31. That is.
However, in the second modification, for example, 5 to 8 patterns are prepared for the change in the number of medium-distance corresponding ML rows between the long-distance corresponding ML row and the short-distance corresponding ML row, and these 5 to 8 patterns are photoelectrically converted. Periodic repetition in the column direction (vertical direction) of the region 31 is also included in the irregular change.

According to the second modification of the first embodiment, regarding the short-distance correspondence ML row and the long-distance correspondence ML row that replace a part of the medium-distance correspondence ML row, the short-distance correspondence ML row and the long-distance correspondence ML row The number of medium-distance ML rows between and the number of medium-distance ML rows between the long-distance ML row and the short-distance ML row are irregular in the column direction (vertical direction) of the photoelectric conversion region 31. The boundary that changes from the medium-distance-compatible ML line to the short-distance-compatible ML line in the horizontal direction, or the boundary that changes from the medium-distance-compatible ML line to the long-distance-compatible ML line in the horizontal direction is the above-mentioned first. It can be made less noticeable in the image generated based on the signal and the second signal.

As described above, according to the second modification of the first embodiment, the following effects can be obtained. That is, the short-distance corresponding ML row and the long-distance corresponding ML row of the image sensor 3 are arranged aperiodically in the second (vertical) direction in place of the medium-distance corresponding ML row. Since it is configured in this way, the boundary that changes from the medium-distance corresponding ML line to the short-distance corresponding ML line in the first (horizontal) direction, or the medium-distance corresponding ML line to the long-distance corresponding ML line in the first (horizontal) direction. The boundary that changes to can be made inconspicuous in the images generated based on the first signal and the second signal.

In addition, the modification 1 of the first embodiment and the modification 2 of the first embodiment may be combined. That is, the lengths of the short-distance ML row and the long-distance ML row that replace a part of the medium-distance ML row are irregular (changed irregularly) in the photoelectric conversion region 31 and are short-distance. The number of medium-distance compatible ML rows between the corresponding ML row and the long-distance compatible ML row, and the number of medium-distance corresponding ML rows between the long-distance compatible ML row and the short-distance compatible ML row are determined in the photoelectric conversion area. It may be changed irregularly in the column direction (vertical direction) of 31.

(Second embodiment)
In the second embodiment, as compared with the first embodiment, the short-distance corresponding ML row and the long-distance corresponding ML row, which are replaced with a part of the medium-distance corresponding ML row, are replaced with three adjacent rows as a set. It differs in that.
The camera 1 in the second embodiment may or may not be of the interchangeable lens type as in the first embodiment. Further, it may be configured as an image pickup device such as a smartphone or a video camera.

FIG. 11 shows an enlargement of the microlens ML of the pixel 20 arranged in the peripheral portion of the photoelectric conversion region 31 of FIG. 6, for example, the region 31b at a position where the image height is high in the image sensor 3 according to the second embodiment. It is a figure. Also in the second embodiment, the pixel rows (GB rows) in which the pixels 20G and the pixels 20B are alternately arranged are used for the focus detection, and a part of the GB rows is used as a short-distance correspondence ML row or a long-distance correspondence ML row. Replace with.

In FIG. 11, short-distance corresponding ML rows are provided as a set of three rows. The upper three rows and the lower three rows, respectively, sandwiching the three short-distance corresponding ML rows are medium-distance corresponding ML rows. In the medium-distance corresponding ML line, the distance between the line CS passing through the center of the pixel 20 and the line CL (middle) passing through the center of the microlens ML is G. That is, the amount of deviation of the center position of the microlens ML with respect to the center position of the pixel 20 is G.
In FIG. 11, the pixel 20G is shown in white, the pixel 20B is shown by an oblique line, and the pixel 20R is shown by hatching.

In the short-distance corresponding ML line of FIG. 11, the distance between the line CS passing through the center of the pixel 20 and the line CL (short) passing through the center of the microlens ML is Gs. That is, the amount of deviation of the center position of the microlens ML with respect to the center position of the pixel 20 is Gs. The shift amount Gs in the short-distance corresponding ML row is larger than the shift amount G in the medium-distance corresponding ML row so that G <Gs is satisfied. The reason for this is as described in the first embodiment.

Further, among the short-distance corresponding ML rows of FIG. 11, the pixel 20G and the pixel 20B arranged in one of the three short-distance corresponding ML rows are alternately arranged for focus detection. It is a line (GB line). The two pixel rows tangent to the boundary between the short-distance ML row and the medium-distance ML row are not used for focus detection. The reason for this is that the amount of shift of the microlens ML between rows differs between the two pixel rows tangent to the boundary, so it cannot be denied that minute distortion may occur in the microlens ML due to manufacturing reasons for the microlens ML. It depends. Assuming that the microlens ML is distorted, even if the amount of deviation of the center position of the microlens ML with respect to the center position of the pixel 20 is strictly controlled as described above, the focus is caused by the distortion of the microlens ML. The accuracy of pupil division for detection is reduced. In order to avoid such a risk, the two pixel rows adjacent to the boundary between the short-distance corresponding ML row and the medium-distance corresponding ML row are not used for focus detection.

For the same reason, among the medium-distance corresponding ML rows in FIG. 11, the pixels that may be used for focus detection are pixels in which pixels 20G and pixels 20B are alternately arranged, which are not in contact with the boundary with the short-distance corresponding ML row. A row (GB row) and a pixel row (RG row) in which pixels 20R and pixels 20G are alternately arranged. In this case, image data used for focus adjustment is generated based on the first signal and the second signal of the pixel 20G, the pixel 20B, or the pixel 20R in the medium-distance corresponding ML row.

The same applies to the long-distance correspondence ML line. FIG. 12 is an enlarged view of the microlens ML of the pixel 20 arranged in the region 31c (FIG. 6) in which the image height of the photoelectric conversion region 31 of the image pickup device 3 according to the second embodiment is high. In FIG. 12, a set of three long-distance ML rows is provided. The upper three rows and the lower three rows are the middle-distance corresponding ML rows, respectively, with the three long-distance corresponding ML rows in between. In the medium-distance corresponding ML row, the amount of deviation of the center position of the microlens ML with respect to the center position of the pixel 20 is G.
Also in FIG. 12, the pixel 20G is shown by white, the pixel 20B is shown by an oblique line, and the pixel 20R is shown by hatching.

In the long-distance correspondence ML line of FIG. 12, the distance between the line CS passing through the center of the pixel 20 and the line CL (length) passing through the center of the microlens ML is GL. That is, the amount of deviation of the center position of the microlens ML with respect to the center position of the pixel 20 is GL. The shift amount GL in the long-distance corresponding ML row is smaller than the shift amount G in the medium-distance corresponding ML row so that GL <G is satisfied. The reason for this is as described in the first embodiment.

Further, among the long-distance corresponding ML rows of FIG. 12, the pixel 20G and the pixel 20B arranged in one of the three long-distance corresponding ML rows are alternately arranged for focus detection. It is a line (GB line). The two pixel rows tangent to the boundary between the long-distance ML row and the medium-distance ML row are not used for focus detection. The reason for this is the same as in the case of the short-distance corresponding ML line described above.

For the same reason, among the medium-distance corresponding ML rows in FIG. 12, the pixels that may be used for focus detection are pixels in which pixels 20G and pixels 20B are alternately arranged, which are not in contact with the boundary with the long-distance corresponding ML rows. A row (GB row) and a pixel row (RG row) in which pixels 20R and pixels 20G are alternately arranged. In this case, image data used for focus adjustment is generated based on the first signal and the second signal of the pixel 20G, the pixel 20B, or the pixel 20R in the medium-distance corresponding ML row.

In the above description, in the regions 31b and 31c located on the left side of the center of the photoelectric conversion region 31 in FIG. 6, the center position of the microlens ML is set to the left from the center positions of the first and second photodiodes S1 and S2. An example of arranging them in a staggered manner was explained. In the regions 31d and 31e located on the right side of the center of the photoelectric conversion region 31 in FIG. 6, the center position of the microlens ML is displaced to the right from the center positions of the first and second photodiodes S1 and S2. do. The shift amount G in the medium-distance corresponding ML row, the shift amount Gs in the short-distance corresponding ML row, and the shift amount GL in the long-distance corresponding ML row have a relationship of GL <G <Gs.

According to the second embodiment described above, in addition to the action and effect of the first embodiment, the following action and effect can be obtained.
(1) The short-distance correspondence ML row and the long-distance correspondence ML row of the image pickup element 3 are arranged in place of the medium-distance correspondence ML row of a predetermined ratio in the total number of rows in the second (vertical) direction. With this configuration, for example, a predetermined number of pixel rows having the same shift amount of the center position of the microlens ML with respect to the center position of the pixel 20 are made continuous in the second (vertical) direction, and the shift amount changes. Pixel rows can be prevented from being used for focus detection. As a result, the pupil division for focus detection can be performed with high accuracy.

(2) The short-distance corresponding ML row and the long-distance corresponding ML row of the image sensor 3 are arranged side by side in the second (vertical) direction, respectively. With this configuration, by using the pixel row in the middle of the three rows for focus detection, it is possible to accurately perform pupil division for focus detection.

(3) The short-distance corresponding ML row and the long-distance corresponding ML row of the image sensor 3 are periodically arranged in the second (vertical) direction in place of the medium-distance corresponding ML row. With this configuration, the short-distance ML row and the long-distance ML row can be arranged so as not to be biased toward the upper part or the lower part of the photoelectric conversion region 31. As a result, it is possible to perform pupil division for focus detection without bias in the photoelectric conversion region 31.

The following modifications are also within the scope of the present invention, and one or more of the modifications can be combined with the above-described embodiment.
(Modification 3)
In the third modification of the second embodiment, as compared with the second embodiment, the short-distance correspondence ML row and the long-distance correspondence ML row that replace a part of the medium-distance correspondence ML row are adjacent to each other in four rows. The difference is that they are replaced by a set.
The camera 1 in the third modification of the second embodiment may or may not be of the interchangeable lens type as in the second embodiment. Further, it may be configured as an image pickup device such as a smartphone or a video camera.

FIG. 13 is an enlarged view of the microlens ML of the pixel 20 arranged in the region 31b where the image height of the photoelectric conversion region 31 of FIG. 6 is high in the image sensor 3 according to the modification 3 of the second embodiment. Is. In the third modification of the second embodiment, the pixel rows (GB rows) in which the pixels 20G and the pixels 20B are alternately arranged, and the pixel rows (RG rows) in which the pixels 20R and the pixels 20G are alternately arranged. Is used for focus detection, and a part of the GB line and the RG line is replaced with a short-distance corresponding ML line and a long-distance corresponding ML line.

In FIG. 13, short-distance corresponding ML rows are provided as a set of four rows. The upper three rows and the lower four rows are the middle-distance corresponding ML rows, respectively, with the four short-distance corresponding ML rows in between. In the medium-distance corresponding ML line, the distance between the line CS passing through the center of the pixel 20 and the line CL (middle) passing through the center of the microlens ML is G. That is, the amount of deviation of the center position of the microlens ML with respect to the center position of the pixel 20 is G.
In FIG. 13, the pixel 20G is shown in white, the pixel 20B is shown by an oblique line, and the pixel 20R is shown by hatching.

In the short-distance corresponding ML line of FIG. 13, the distance between the line CS passing through the center of the pixel 20 and the line CL (short) passing through the center of the microlens ML is Gs. That is, the amount of deviation of the center position of the microlens ML with respect to the center position of the pixel 20 is Gs. The shift amount Gs in the short-distance corresponding ML row is larger than the shift amount G in the medium-distance corresponding ML row so that G <Gs is satisfied. The reason for this is as described in the first embodiment.

Further, among the short-distance corresponding ML rows of FIG. 13, the pixel 20G and the pixel 20B arranged in two of the four short-distance corresponding ML rows are alternately arranged for focus detection. A row (GB row) and a pixel row (RG row) in which pixels 20R and pixels 20G are alternately arranged. The two pixel rows tangent to the boundary between the short-distance ML row and the medium-distance ML row are not used for focus detection. The reason for this is as described in the second embodiment.

For the same reason, among the medium-distance corresponding ML rows in FIG. 13, the pixels that may be used for focus detection are pixels in which pixels 20G and pixels 20B are alternately arranged, which are not in contact with the boundary with the short-distance corresponding ML rows. A row (GB row) and a pixel row (RG row) in which pixels 20R and pixels 20G are alternately arranged. In this case, image data used for focus adjustment is generated based on the first signal and the second signal of the pixel 20G, the pixel 20B, or the pixel 20R in the medium-distance corresponding ML row.

The same applies to the long-distance correspondence ML line. FIG. 14 is an enlarged view of the microlens ML of the pixel 20 arranged in the region 31c (FIG. 6) in which the image height of the photoelectric conversion region 31 of the image pickup device 3 according to the modification 3 of the second embodiment is high. Is. In FIG. 14, a set of four long-distance ML rows is provided. The upper three rows and the lower four rows are the middle-distance corresponding ML rows, respectively, with the four long-distance corresponding ML rows in between. In the medium-distance corresponding ML row, the amount of deviation of the center position of the microlens ML with respect to the center position of the pixel 20 is G.
Also in FIG. 14, the pixel 20G is shown in white, the pixel 20B is shown by an oblique line, and the pixel 20R is shown by hatching.

In the long-distance correspondence ML line of FIG. 14, the distance between the line CS passing through the center of the pixel 20 and the line CL (length) passing through the center of the microlens ML is GL. That is, the amount of deviation of the center position of the microlens ML with respect to the center position of the pixel 20 is GL. The shift amount GL in the long-distance corresponding ML row is smaller than the shift amount G in the medium-distance corresponding ML row so that GL <G is satisfied. The reason for this is as described in the first embodiment.

Further, among the long-distance corresponding ML rows of FIG. 14, the pixel 20G and the pixel 20B arranged in two of the four long-distance corresponding ML rows are alternately arranged for focus detection. A row (GB row) and a pixel row (RG row) in which pixels 20R and pixels 20G are alternately arranged. The two pixel rows tangent to the boundary between the long-distance ML row and the medium-distance ML row are not used for focus detection. The reason for this is the same as in the case of the short-distance corresponding ML line described above.

For the same reason, among the medium-distance corresponding ML rows in FIG. 14, the pixels that may be used for focus detection are pixels in which pixels 20G and pixels 20B are alternately arranged, which are not in contact with the boundary with the long-distance corresponding ML rows. A row (GB row) and a pixel row (RG row) in which pixels 20R and pixels 20G are alternately arranged. In this case, image data used for focus adjustment is generated based on the first signal and the second signal of the pixel 20G, the pixel 20B, or the pixel 20R in the medium-distance corresponding ML row.

In the above description, in the regions 31b and 31c located on the left side of the center of the photoelectric conversion region 31 in FIG. 6, the center position of the microlens ML is set to the left from the center positions of the first and second photodiodes S1 and S2. An example of arranging them in a staggered manner was explained. In the regions 31d and 31e located on the right side of the center of the photoelectric conversion region 31 in FIG. 6, the center position of the microlens ML is displaced to the right from the center positions of the first and second photodiodes S1 and S2. do. The shift amount G in the medium-distance corresponding ML row, the shift amount Gs in the short-distance corresponding ML row, and the shift amount GL in the long-distance corresponding ML row have a relationship of GL <G <Gs.

As described above, according to the third modification of the second embodiment, the following effects can be obtained. That is, the short-distance corresponding ML row and the long-distance corresponding ML row of the image sensor 3 are arranged side by side in the second (vertical) direction, respectively. With this configuration, by using the two middle pixel rows out of the four rows for focus detection, it is possible to accurately perform pupil division for focus detection.

(Third embodiment)
In the third embodiment, the positions of the short-distance correspondence ML row and the long-distance correspondence ML row arranged in the photoelectric conversion region 31 will be described. Specifically, a variation of the arrangement of the short-distance correspondence ML row and the long-distance correspondence ML row which replaces a part of the medium-distance correspondence ML row is illustrated. Hereinafter, eight embodiments (Example 1) to (Example 8) will be illustrated, but any arrangement may be adopted, and a part of these arrangements may be appropriately changed.

(Example 1)
FIG. 15 is a diagram illustrating a first aspect of the arrangement of the short-distance correspondence ML row and the long-distance correspondence ML row. FIG. 15 shows, for example, in what order the medium-distance correspondence ML row, the short-distance correspondence ML row, and the long-distance correspondence ML row are arranged when paying attention to the region 31x of the photoelectric conversion region 31 of FIG. show. (Example 1) corresponds to the first embodiment described above. Therefore, only the GB row used for focus detection among the medium-distance corresponding ML rows is replaced with the short-distance corresponding ML row or the long-distance corresponding ML row.

In FIG. 15, among the pixel rows (GB rows) in which the pixels 20G and the pixels 20B of the medium-distance corresponding ML row are alternately arranged, the row No. 1 and line No. 13 and line No. A short-distance corresponding ML line is arranged at 25 ... As a result, the short-distance corresponding ML line appears repeatedly with the 11 medium-distance corresponding ML line sandwiched between them. After repeating such an arrangement for 96 rows, in the 97th and subsequent rows, among the pixel rows (GB rows) in which the pixels 20G and the pixels 20B of the medium-distance corresponding ML row are alternately arranged, the row No. 97 and line No. A long-distance correspondence ML line is arranged at 109 ... As a result, the long-distance corresponding ML line appears repeatedly with the 11 medium-distance corresponding ML line sandwiched between them.
As described above, according to (Example 1), the arrangement in which the short-distance corresponding ML rows repeatedly appear every 11 rows and the arrangement in which the long-distance corresponding ML rows repeatedly appear every 11 rows are repeated in a cycle of 96 rows.

(Example 2)
FIG. 16 is a diagram illustrating a second aspect of the arrangement of the short-distance correspondence ML row and the long-distance correspondence ML row. FIG. 16 shows the order of the medium-distance correspondence ML row, the short-distance correspondence ML row, and the long-distance correspondence ML row when paying attention to the region 31x (FIG. 6) of the photoelectric conversion region 31 as in FIG. Indicates whether it is arranged with. (Example 2) is an arrangement assuming that the RB row is used for focus detection in addition to the GB row. Therefore, the GB row and the RB row used for focus detection among the medium-distance corresponding ML rows are replaced with the short-distance corresponding ML row or the long-distance corresponding ML row.

In FIG. 16, among the pixel rows (GB rows) in which the pixels 20G and the pixels 20B of the medium-distance corresponding ML row are alternately arranged, the row No. 1 and line No. 13 and line No. A short-distance corresponding ML line is arranged at 25 ... Further, among the pixel rows (RG rows) in which the pixels 20R and the pixels 20G of the medium-distance corresponding ML row are alternately arranged, the row No. 2 and line No. 14 and line No. A short-distance corresponding ML line is arranged at 26 ... As a result, two rows of short-distance corresponding ML rows adjacent to each other in the column direction (vertical direction) appear repeatedly with 10 rows of medium-distance corresponding ML rows in between.

After repeating such an arrangement for 96 rows, in the 97th and subsequent rows, among the pixel rows (GB rows) in which the pixels 20G and the pixels 20B of the medium-distance corresponding ML row are alternately arranged, the row No. 97 and line No. A long-distance correspondence ML line is arranged at 109 ... Further, among the pixel rows (RG rows) in which the pixels 20R and the pixels 20G of the medium-distance corresponding ML row are alternately arranged, the row No. 98 and line No. A long-distance correspondence ML line is arranged at 110 .... As a result, two rows of long-distance corresponding ML rows adjacent to each other in the column direction (vertical direction) appear repeatedly with 10 rows of medium-distance corresponding ML rows in between.
As described above, according to (Example 2), there are 96 arrangements in which two consecutive short-distance ML rows appear every 10 rows and two consecutive long-distance corresponding ML rows appear every 10 rows. Repeated in a row cycle.

(Example 3)
FIG. 17 is a diagram illustrating a third aspect of the arrangement of the short-distance correspondence ML row and the long-distance correspondence ML row. FIG. 17 shows the order of the medium-distance correspondence ML row, the short-distance correspondence ML row, and the long-distance correspondence ML row when paying attention to the region 31x (FIG. 6) of the photoelectric conversion region 31 as in FIG. Indicates whether it is arranged with. (Example 3) is an arrangement assuming that the RB row is used for focus detection in addition to the GB row. Therefore, the GB row and the RB row used for focus detection among the medium-distance corresponding ML rows are replaced with the short-distance corresponding ML row or the long-distance corresponding ML row.

In FIG. 17, among the pixel rows (GB rows) in which the pixels 20G and the pixels 20B of the medium-distance corresponding ML row are alternately arranged, the row No. 1 and line No. 21 and line No. A short-distance corresponding ML line is arranged at 41 ... Further, among the pixel rows (RG rows) in which the pixels 20R and the pixels 20G of the medium-distance corresponding ML row are alternately arranged, the row No. 6 and line No. 26 and line No. A short-distance corresponding ML line is arranged at 46 ....

Further, among the pixel rows (GB rows) in which the pixels 20G and the pixels 20B of the medium-distance corresponding ML row are alternately arranged, the row No. 11 and line No. A long-distance correspondence ML line is arranged at 31 ... Further, among the pixel rows (RG rows) in which the pixels 20R and the pixels 20G of the medium-distance corresponding ML row are alternately arranged, the row No. 16 and line No. A long-distance correspondence ML line is arranged at 36 ...

As described above, according to (Example 3), the arrangement in which the short-distance corresponding ML rows repeatedly appear every four rows and the arrangement in which the long-distance corresponding ML rows repeatedly appear every four rows are repeated in a cycle of 10 rows.

(Example 4)
FIG. 18 is a diagram illustrating a fourth aspect of the arrangement of the short-distance correspondence ML row and the long-distance correspondence ML row. FIG. 18 shows the order of the medium-distance corresponding ML row, the short-distance corresponding ML row, and the long-distance corresponding ML row when paying attention to the region 31x (FIG. 6) of the photoelectric conversion region 31 as in FIG. Indicates whether it is arranged with. (Example 4) is an arrangement assuming that the RB row is used for focus detection in addition to the GB row. Therefore, the GB row and the RB row used for focus detection among the medium-distance corresponding ML rows are replaced with the short-distance corresponding ML row or the long-distance corresponding ML row.

In FIG. 18, among the pixel rows (GB rows) in which the pixels 20G and the pixels 20B of the medium-distance corresponding ML row are alternately arranged, the row No. 1 and line No. 13 and line No. 25 and line No. A short-distance corresponding ML line is arranged at 37 ... Further, among the pixel rows (RG rows) in which the pixels 20R and the pixels 20G of the medium-distance corresponding ML row are alternately arranged, the row No. 4 and line No. 16 and line No. 28 and line No. A short-distance corresponding ML line is arranged at 40 ....

Further, among the pixel rows (GB rows) in which the pixels 20G and the pixels 20B of the medium-distance corresponding ML row are alternately arranged, the row No. 7 and line No. 19 and line No. 31 and line No. A long-distance correspondence ML line is arranged at 43 ... Further, among the pixel rows (RG rows) in which the pixels 20R and the pixels 20G of the medium-distance corresponding ML row are alternately arranged, the row No. 10 and line No. 22 and line No. 34 and line No. A long-distance correspondence ML line is arranged at 46 ....

As described above, according to (Example 4), the arrangement in which the short-distance corresponding ML rows repeatedly appear every two rows and the arrangement in which the long-distance corresponding ML rows repeatedly appear every two rows are repeated in a cycle of six rows.

(Example 5)
FIG. 19 is a diagram illustrating a fifth aspect of the arrangement of the short-distance correspondence ML row and the long-distance correspondence ML row. FIG. 19 shows the order of the medium-distance correspondence ML row, the short-distance correspondence ML row, and the long-distance correspondence ML row when paying attention to the region 31x (FIG. 6) of the photoelectric conversion region 31 as in FIG. Indicates whether it is arranged with. (Example 5) corresponds to the second embodiment described above. Therefore, for example, line No. As shown in 12 to 14, short-distance corresponding ML rows are provided as a set of three rows. Also, for example, line No. As shown in 96 to 98, a set of three long-distance ML rows is provided.

In FIG. 19, three rows of short-distance corresponding ML rows adjacent to each other in the column direction (vertical direction) appear repeatedly with nine rows of medium-distance corresponding ML rows sandwiched between them. After repeating such an arrangement for 96 rows, after the 97th row, 3 rows of long-distance correspondence ML rows adjacent to each other in the column direction (vertical direction) appear repeatedly with 9 rows of medium-distance correspondence ML rows in between. ..
As described above, according to (Example 5), the arrangement in which the short-distance corresponding ML rows repeatedly appear every 9 rows in 3 consecutive rows and the arrangement in which the long-distance corresponding ML rows repeatedly appear in 3 consecutive rows every 9 rows. , Repeated in a 96-line cycle.
In (Example 5), as described above, the two pixel rows adjacent to the boundary between the short-distance corresponding ML row and the medium-distance corresponding ML row, and the long-distance corresponding ML row and the medium-distance corresponding ML row. Neither of the two pixel rows touching the boundary of is used for focus detection.

(Example 6)
FIG. 20 is a diagram illustrating a sixth aspect of the arrangement of the short-distance correspondence ML row and the long-distance correspondence ML row. FIG. 20 shows the order of the medium-distance correspondence ML row, the short-distance correspondence ML row, and the long-distance correspondence ML row when paying attention to the region 31x (FIG. 6) of the photoelectric conversion region 31 as in FIG. Indicates whether it is arranged with. (Example 6) corresponds to the modification 3 of the second embodiment described above. Therefore, for example, line No. As shown in 12 to 15, short-distance corresponding ML rows are provided as a set of four rows. Also, for example, line No. As shown in 96 to 99, a set of four long-distance ML rows is provided.

In FIG. 20, four rows of short-distance corresponding ML rows adjacent to each other in the column direction (vertical direction) appear repeatedly with eight rows of medium-distance corresponding ML rows in between. After repeating such an arrangement for 96 rows, 4 rows of long-distance corresponding ML rows adjacent to each other in the column direction (vertical direction) appear repeatedly across 8 rows of medium-distance corresponding ML rows from the 97th row onward. ..
As described above, according to (Example 6), the arrangement in which the short-distance corresponding ML rows repeatedly appear every 8 rows in a row and the arrangement in which the long-distance corresponding ML rows repeatedly appear in a row every 8 rows. , Repeated in a 96-line cycle.
In (Example 6), as described above, the two pixel rows that touch the boundary between the short-distance corresponding ML row and the medium-distance corresponding ML row, and the long-distance corresponding ML row and the medium-distance corresponding ML row. Neither of the two pixel rows touching the boundary of is used for focus detection.

(Example 7)
FIG. 21 is a diagram illustrating a seventh aspect of arrangement of a short-distance correspondence ML row and a long-distance correspondence ML row. FIG. 21 shows the order of the medium-distance correspondence ML row, the short-distance correspondence ML row, and the long-distance correspondence ML row when paying attention to the region 31x (FIG. 6) of the photoelectric conversion region 31 as in FIG. Indicates whether it is arranged with. (Example 7) corresponds to the second embodiment described above. Therefore, for example, line No. As shown in 24 to 26, short-distance corresponding ML rows are provided as a set of three rows. Also, for example, line No. As shown in 12 to 14, long-distance correspondence ML rows are provided as a set of three rows.

In FIG. 21, three rows of short-distance correspondence ML rows adjacent to each other in the column direction (vertical direction) and three rows of long-distance correspondence ML rows adjacent to each other in the column direction (vertical direction) are nine rows of medium-distance correspondence ML rows. It appears repeatedly alternately with a line in between.
In (Example 7), as described above, the two pixel rows adjacent to the boundary between the short-distance corresponding ML row and the medium-distance corresponding ML row, and the long-distance corresponding ML row and the medium-distance corresponding ML row. Neither of the two pixel rows touching the boundary of is used for focus detection.

(Example 8)
FIG. 22 is a diagram illustrating an eighth aspect of the arrangement of the short-distance correspondence ML row and the long-distance correspondence ML row. FIG. 22 shows the order of the medium-distance corresponding ML row, the short-distance corresponding ML row, and the long-distance corresponding ML row when paying attention to the region 31x (FIG. 6) of the photoelectric conversion region 31 as in FIG. Indicates whether it is arranged with. (Example 8) corresponds to the modification 3 of the second embodiment described above. Therefore, for example, line No. As shown in 24 to 27, short-distance corresponding ML rows are provided as a set of four rows. Also, for example, line No. As shown in 12 to 15, long-distance correspondence ML rows are provided as a set of four rows.

In FIG. 22, four rows of short-distance correspondence ML rows adjacent to each other in the column direction (vertical direction) and four rows of long-distance correspondence ML rows adjacent to each other in the column direction (vertical direction) are eight rows of medium-distance correspondence ML rows. It appears repeatedly alternately with a line in between.
In (Example 8), as described above, the two pixel rows that touch the boundary between the short-distance corresponding ML row and the medium-distance corresponding ML row, and the long-distance corresponding ML row and the medium-distance corresponding ML row. Neither of the two pixel rows touching the boundary of is used for focus detection.

According to the third embodiment described above, in addition to the action and effect of the first embodiment, the following action and effect can be obtained. That is, the short-distance ML row and the long-distance ML row of the image pickup element 3 are replaced with the medium-distance ML row at a ratio as shown in each of the above examples among the total number of rows in the second (vertical) direction. Be placed. With this configuration, for example, the pixel rows at the boundary where the amount of shift changes can be prevented from being used for focus detection, and the short-distance compatible ML row and the long-distance compatible ML row can be used in the photoelectric conversion region 31. It can be arranged so as not to be biased to the upper part or the lower part. As a result, the pupil division for focus detection can be performed accurately in the photoelectric conversion region 31 without bias.

Although various embodiments and modifications have been described above, the present invention is not limited to these contents. Other aspects considered within the scope of the technical idea of the present invention are also included within the scope of the present invention.

1 ... Camera 2 ... Shooting lens 3 ... Image sensor 9 ... Microprocessor 10 ... Focus calculation unit 13 ... Image processing unit 20, 20G, 20R, 20B ... Pixel 21 ... Vertical scanning circuit 22 ... Horizontal scanning circuit 23, 24 ... Control signal Line 25 ... Vertical signal line AMP ... Amplification transistor FD ... FD region ML ... Microlens S1, S2 ... Photodiode SEL ... Selective transistor Tx-1, Tx-2 ... Transfer transistor

Claims (10)

A pixel that includes a microlens, a first photoelectric conversion unit and a second photoelectric conversion unit, and photoelectrically converts first and second light fluxes that have passed through the first and second regions of the pupil of the optical system, respectively. , Multiple arrangements are made in the area where the light from the optical system is incident.
The pixel is a first pixel having a first positional relationship between the microlens and the first and second photoelectric conversion units corresponding to the position of the first exit pupil of the optical system, and the microlens. And the second pixel having a second positional relationship in which the first and second photoelectric conversion units have a second positional relationship corresponding to the position of the second exit pupil of the optical system.
A plurality of first pixel rows in which a plurality of the first pixels are arranged in the first direction are formed in the second direction, and a second pixel row in which a plurality of the second pixels are arranged in the first direction is the first. An image sensor that is placed in place of a part of the pixel row. In the image pickup device according to claim 1,
The pixel includes a third pixel in which the microlens and the first and second photoelectric conversion units have a third positional relationship corresponding to the position of the third exit pupil of the optical system.
A third pixel row in which a plurality of the third pixels are arranged in the first direction is arranged in place of a part of the first pixel row.
An image pickup device having the first pixel row between the second pixel row and the third pixel row. In the image pickup device according to claim 2,
The length of the second pixel row and the third pixel row is shorter than the length of the region in which the light from the optical system is incident in the first direction. In the image pickup device according to claim 3,
The second pixel row and the third pixel row are image pickup devices arranged in a region where the distance from the center of the region is farther than a predetermined value. In the image pickup device according to claim 4,
An image pickup device in which a plurality of the second pixel rows are arranged and the distances from the center of the region in which the second pixel rows adjacent to each other in the second direction are arranged are different. In the image pickup device according to claim 4 or 5,
An image pickup device in which a plurality of the third pixel rows are arranged and the distances from the center of the region in which the third pixel rows adjacent to each other in the second direction are arranged are different. The image pickup device according to any one of claims 2 to 6.
The second pixel row and the third pixel row are image pickup devices arranged in place of the first pixel row in a predetermined ratio of the total number of rows in the second direction. In the image pickup device according to claim 7,
The second pixel row and the third pixel row are image pickup devices that are periodically arranged in place of the first pixel row in the second direction. In the image pickup device according to claim 7,
The second pixel row and the third pixel row are image pickup devices arranged in place of the first pixel row aperiodically in the second direction. In the image pickup device according to claim 7,
An image pickup device in which at least three rows of the second pixel row and the third pixel row are arranged side by side in the second direction, respectively.

Images