JP5786355B2 - Defocus amount detection device and electronic camera - Google Patents

Description

  The present invention relates to a defocus amount detection device and an electronic camera.

  Conventionally, a pupil-divided pixel that receives a pupil-divided image for defocus amount detection is arranged in a partial area of an imaging element in which imaging pixels are arranged two-dimensionally, and at the same time an image is captured by the imaging pixel, A technique for detecting a defocus amount by detecting a positional shift between pixel data of pupil division pixels having different pupil division directions is known. Furthermore, in order to reduce the image quality degradation of the captured image, a technique has been proposed that suppresses the decrease in the pixel density of the entire pupil data for imaging while maintaining the pixel density of the defocus detection data (for example, Patent Document 1). reference).

JP 2009-141390 A

  However, in the prior art, in the process of detecting the positional deviation of the subject image using the pupil division data of the pupil division pixels and the whole pupil data of the imaging pixels, all the pupil data alternately arranged in the same column as the pupil division pixels Therefore, there is a problem in that sufficient positional deviation detection cannot be performed on a subject having a high spatial frequency component, the accuracy of the defocus amount is lowered, and the captured image becomes unfocused.

  An object of the present invention is to provide a defocus amount detection apparatus and an electronic camera that can improve the positional deviation detection accuracy using pupil division data and whole pupil data.

The defocus amount detection device according to the present invention includes a pupil division data of the pupil division pixel and an area around the pupil division pixel from an imaging element in which the pupil division pixel is arranged in a part of the two-dimensionally arranged imaging pixels . An image data input unit that reads all the pupil data of the imaging pixel, and all pupil data that interpolates and generates all the pupil data at the pupil division pixel position from the all pupil data of the imaging pixel that is read by the image data input unit A generation unit, the whole pupil data of the pupil division pixel position interpolated and generated by the whole pupil data generation unit, the whole pupil data of the imaging pixel, and the pupil division data read from the pupil division pixel A position shift detection unit that detects a position shift amount, and a defocus amount calculation unit that obtains a defocus amount according to the position shift amount detected by the position shift detection unit.

  In particular, the whole pupil data generation unit determines the direction of the image structure based on the whole pupil data read from the imaging pixels, and interpolates all the pupil data at the pupil division pixel position according to the determination result. The imaging pixel to be referred to when generating is selected.

Furthermore, when determining the direction of the image structure, the whole pupil data generation unit compares the difference between the upper and lower whole pupil data adjacent to the pupil division pixel and the difference between the left and right whole pupil data, The whole pupil data at the pupil division pixel position is generated by interpolation using the whole pupil data having a smaller difference.

  Further, the all-pupil data generating unit is configured to divide the pupil division into all-pupil data for determining the direction of the image structure and all-pupil data to be referred to when interpolating and generating all-pupil data at the pupil division pixel position. The whole pupil data of the imaging pixels in the same range around the pixel is used.

  Further, the all-pupil data generation unit generates all-pupil data for all pixel positions in a column in which the pupil division pixels are arranged.

  Further, the image data input unit receives the pupil division data and the whole pupil data from a column in which the pupil division pixels are arranged, and the whole pupil data from columns above and below the column in which the pupil division pixels are arranged. It is characterized by reading.

  Alternatively, the image data input unit reads data obtained by adding and averaging all pupil data of the plurality of imaging pixels around the pupil division pixel in the imaging element as all pupil data at the pupil division pixel position. And

  The electronic camera which concerns on this invention has the said defocus amount detection apparatus, The electronic camera characterized by the above-mentioned.

  The defocus amount detection device and the electronic camera according to the present invention can improve the detection accuracy of the positional deviation amount using the pupil division data and the whole pupil data.

It is a figure which shows the structural example of the electronic camera 101 which concerns on this embodiment. 3 is a diagram illustrating an example of pixel arrangement of an image sensor 104. FIG. It is a figure which shows the example of arrangement | positioning of a pupil division pixel. 3 is a diagram illustrating a configuration example of a control unit 108. FIG. It is explanatory drawing of the interpolation production | generation process of all the pupil data. It is a figure which shows the example of data of the pupil division | segmentation pixel after the interpolation production | generation. It is a figure which shows the mode of all the pupil data and pupil division data. It is a figure which shows the mode of the conventional whole pupil data and pupil division data. It is explanatory drawing of a position shift detection process. It is a figure which shows how to obtain | require the positional offset amount by the interpolation method. It is a flowchart of AF processing. 6 is a diagram illustrating a first modification of the pixel arrangement of the image sensor 104. FIG. It is a figure which shows the modification 2 of pixel arrangement | positioning of the image pick-up element. It is a figure which shows the modification of the image pick-up element 104. FIG.

  Hereinafter, embodiments of a defocus amount detection apparatus and an electronic camera according to the present invention will be described in detail with reference to the drawings. In the following embodiment, an example of the electronic camera 101 equipped with the defocus amount detection apparatus according to the present invention will be described. However, the present invention may be applied to a dedicated device that performs defocus amount detection.

[Configuration of Electronic Camera 101]
First, the configuration of the electronic camera 101 will be described. FIG. 1 is a block diagram illustrating a configuration example of an electronic camera 101. The electronic camera 101 includes an optical system 102, a mechanical shutter 103, an image sensor 104, an A / D converter 105, an image buffer 106, and image processing. A unit 107, a control unit 108, a memory 109, a display unit 110, an operation unit 111, and a memory card IF (interface) 112 are configured.

  The optical system 102 includes a plurality of lenses such as a zoom lens and a focus lens, a lens position driving mechanism, a diaphragm and a diaphragm driving mechanism, and the like. Then, the control unit 108 instructs the lens position driving mechanism and the aperture driving mechanism to perform position control and aperture control of the zoom lens and the focus lens.

  The mechanical shutter 103 includes a shutter and a shutter driving mechanism. The control unit 108 controls the shutter driving mechanism according to the shutter speed to open and close the shutter for a predetermined time, and causes the light flux of the optical system 102 to enter the image sensor 104.

  The image sensor 104 is configured by, for example, a CMOS solid-state image sensor, and a plurality of pixels having a photoelectric conversion unit on a light receiving surface are two-dimensionally arranged. Then, in response to a command from the control unit 108, an image signal photoelectrically converted by each pixel is read out. In particular, in the imaging device 104 of the electronic camera 101 according to the present embodiment, a pupil division pixel for performing focus detection by the pupil division method is arranged in a part of the imaging pixels of the imaging device 104 in order to obtain a defocus amount. Has been.

  The A / D conversion unit 105 converts the analog image signal output from the image sensor 104 into digital image data, and stores the image data for one captured image in the image buffer 106. For example, when the resolution of an image read from the image sensor 104 is 1000 pixels × 1000 pixels, image data for 1 million pixels is taken into the image buffer 106.

  The image buffer 106 is composed of, for example, a volatile high-speed memory. The image data output from the A / D conversion unit 105 is not only captured but also used as a buffer memory when the image processing unit 107 performs image processing. Alternatively, it is also used as a display buffer when a captured image and a captured image stored in the memory card 112 a connected to the memory card IF 112 are read and displayed on the display unit 110.

  The image processing unit 107 performs image processing such as white balance processing, color interpolation processing, gamma correction processing, saturation enhancement processing, and contour outline enhancement processing on the image data captured in the image buffer 106. Further, image compression processing is performed on the image data after image processing by an image compression method compliant with the JPEG standard or the like.

  The control unit 108 is constituted by a CPU that operates according to a program stored therein, for example, and controls the overall operation of the electronic camera 101. For example, the control unit 108 performs position control and aperture control of the focus lens of the optical system 102 when the release button of the operation unit 111 is pressed, opens and closes the mechanical shutter 103, and captures a subject image with the image sensor 104. Then, the control unit 108 converts the image signal read from the image sensor 104 into image data by the A / D conversion unit 105, and captures image data for one screen into the image buffer 106. Furthermore, the control unit 108 instructs the image processing unit 107 to perform predetermined image processing on the image data captured in the image buffer 106. Then, the control unit 108 adds a predetermined file name and header information to the image data after the image processing, and saves it in the memory card 112a attached to the memory card I / F 112. Alternatively, the image data after image processing is displayed on the display unit 110 as a captured image. In addition, the control unit 108 executes an autofocus process (AF process) described later.

  The memory 109 is composed of, for example, a nonvolatile semiconductor memory such as a flash memory, and stores parameters such as a shooting mode, exposure information, and focus information of the electronic camera 101. The control unit 108 controls the operation of the electronic camera 101 with reference to these parameters. Note that these parameters are appropriately updated according to user operations performed via the operation unit 111. In particular, in the present embodiment, the memory 109 stores in advance a table or conversion formula for converting a positional deviation amount of a subject image used for focus control into a defocus amount.

  The display unit 110 includes a liquid crystal monitor or the like. Then, the control unit 108 displays a captured image, a menu screen necessary for operation setting of the electronic camera 101, and the like.

  The operation unit 111 includes a power button, a release button, a shooting mode selection dial, a cross button for menu screen operation, and the like. The user operates these operation buttons to use the electronic camera 101. For example, the shooting mode selection dial selects a shooting mode or a focus mode. The operation information of these operation buttons is output to the control unit 108, and the control unit 108 controls the operation of the electronic camera 101 according to the operation information.

  The memory card IF 112 is an interface for connecting the memory card 112 a to the electronic camera 101. Then, the control unit 108 reads and writes image data from and to the memory card 112a via the memory card IF 112.

  The above is the configuration and basic operation of the electronic camera 101.

[Pixel Arrangement of Image Sensor 104]
Next, an example of the pixel arrangement of the image sensor 104 used in the electronic camera 101 according to the present embodiment will be described with reference to FIG. In FIG. 2, the imaging element 104 has a plurality of pixels arranged in a two-dimensional shape in the x-axis direction and the y-axis direction. Each pixel is expressed as a pixel p (x, y) using the x coordinate and the y coordinate. For example, the pixel at the position where the x coordinate is x0 and the y coordinate is y0 is the pixel p (x0, y0).

  In FIG. 2, a plurality of pixels arranged two-dimensionally are assigned four types of symbols, R, G, B, and A. R represents a pixel having a red filter, G represents a pixel having a green filter, and B represents a pixel having a blue filter. These RGB pixels are arranged in accordance with a Bayer array as imaging pixels. In the image sensor 104 of the electronic camera 101 according to the present embodiment, pupil division pixels for detecting a positional deviation amount of the subject image by the pupil division method are arranged in some of the pixels for imaging. In FIG. 2, the pixel indicated by the symbol A is a pupil-divided pixel, and in the same figure, the pixel p (x0-4, y0), the pixel p (x0-2, y0), the pixel p (x0, y0), and the pixel p Five pixels of (x0 + 2, y0) and pixel p (x0 + 4, y0) correspond to pupil division pixels, and are arranged every other pixel in row y0 corresponding to the pupil division direction.

  In FIG. 2, only five pupil-divided pixels are drawn, but the number of pupil-divided pixels that can detect the amount of positional deviation of the subject image by the pupil-dividing method is similarly arranged in the row y0. In the example of FIG. 2, the pupil division pixel is arranged at the position of the B pixel in the Bayer array, but the pupil division pixel may be arranged at the position of the R pixel in the row (y0 + 1). Or you may arrange | position in the position of G pixel of another row. The same applies when the pupil division direction is the column direction. Also, in this embodiment, as will be described later, all pupil data at the pupil division pixel position is generated by interpolation from the G pixel, and the amount of positional deviation is obtained, so the spectral transmittance of the color filter covering the pupil division pixel is Assume that it is the same as the G pixel. However, since the structures of different color components tend to be similar, it is possible to determine the image structure and detect the amount of misalignment even if the spectral transmittance is different, so the spectral transmittance of the color filter that covers the pupil-divided pixels May be different from all pupil pixels.

  Here, imaging pixels and pupil division pixels will be described. FIG. 3A is a diagram of each pixel in the row y0 in FIG. 2 viewed from the incident direction of the light beam. 3A, the row number and the column number correspond to those in FIG. 2, and the pixel p (x0-4, y0), the pixel p (x0-2, y0), the pixel p (x0, y0), and the pixel p ( Five pixels of x0 + 2, y0) and pixel p (x0 + 4, y0) are pupil division pixels. For example, the pupil-divided pixel p (x0, y0) includes a photoelectric conversion unit 361 configured by a photodiode or the like and a microlens 362, and the photoelectric conversion unit 361 is arranged to be biased to the left side of the pixel. Similarly, with respect to the other four pupil-divided pixels, the photoelectric conversion units 361 are arranged to be biased to the left side of the pixels. On the other hand, the imaging pixels arranged alternately with the pupil-divided pixels include a photoelectric conversion unit 363 configured by a photodiode or the like and a microlens 364, such as an imaging pixel p (x0 + 1, y0). The photoelectric conversion unit 363 is arranged on the entire pixel without being biased left and right. The same applies to each of the imaging pixels R, G, and B in the other rows as shown in FIG. 4A and 4B, the photoelectric conversion units 361 of the pupil-divided pixels are arranged on the left side of the pixels. However, as shown in FIG. It may be arranged to be biased to the right side. Note that image data obtained from a photoelectric conversion unit arranged on the left side or the right side like the pupil division pixels is referred to as pupil division data. In addition, image data obtained from photoelectric conversion units arranged evenly on all pixels without being biased to the left or right is referred to as all pupil data.

[Auto focus processing]
Here, the electronic camera 101 according to the present embodiment has an autofocus function and is executed by the control unit 108. When the autofocus function is used, the control unit 108 opens the mechanical shutter 103 and picks up a subject image incident via the optical system 102 with the image sensor 104 and detects the defocus amount. Then, the control unit 108 controls the focus lens position of the optical system 102 according to the defocus amount, and adjusts it to the focus position.

  Next, the autofocus process executed by the AF processing unit 200 of the control unit 108 will be described in detail. FIG. 4 is a block diagram illustrating a configuration of the AF processing unit 200 of the control unit 108. In FIG. 4, the AF processing unit 200 includes an image data input unit 201, an all-pupil data generation unit 202, a positional deviation amount detection unit 203, a defocus amount calculation unit 204, and a focus lens control unit 205. The In the example of FIG. 4, the blocks are divided into five blocks for easy understanding. However, any of the blocks may be combined into one block, and the processing of one block may be divided into a plurality of blocks. For example, the positional deviation amount detection unit 203, the defocus amount calculation unit 204, and the focus lens control unit 205 may be combined into one block and used as a focus control unit. Hereinafter, each block of FIG. 4 will be described.

  The image data input unit 201 performs a process of reading by designating a range of pixels to be read from the image sensor 104. For example, in the case of FIG. 2, the image data input unit 201 reads out the row y <b> 0 including the pupil-divided pixels from the image sensor 104. Alternatively, the image data input unit 201 reads out image signals for three rows including the row (y0-1) and the row (y0 + 1) before and after the row y0 from the image sensor 104. Then, the image signal read from the image sensor 104 is converted into image data by the A / D converter 105 and taken into the image buffer 106. Then, the image data input unit 201 outputs the image data of the pupil-divided pixels and the surrounding imaging pixels taken into the image buffer 106 to the all-pupil data generation unit 202.

  The whole pupil data generation unit 202 interpolates all the pupil data at the pupil division pixel position from all the pupil data of the imaging pixels read by the image data input unit 201 from the imaging element 104 in order to detect the defocus amount. For example, as shown in FIG. 5, the entire pupil data D of the pupil-divided pixel p (x0, y0) is converted into peripheral G pixels (pixel p (x0, y0-1), pixel p (x0, y0 + 1), pixel p (x0). -1, y0) and pixel p (x0 + 1, y0)). In particular, in the electronic camera 101 according to the present embodiment, when interpolating, the structure of the image is discriminated and the pixels used for the interpolation are determined. Here, the structure of the image means a shape in which the pixel value changes. For example, in the case of a vertical line, the change in the vertical direction is small and the change in the horizontal direction is large. Conversely, in the case of a horizontal line, the change in the horizontal direction is small and the change in the vertical direction is large.

  Accordingly, when the subject image is a vertical line, the upper and lower G pixels of the pupil division pixel are used, and when the subject image is a horizontal line, the right and left G pixels of the pupil division pixel are used. Is interpolated, so that the interpolation accuracy is improved and the thin line structure can be correctly reproduced.

  In the electronic camera 101 according to the present embodiment, pixels in the same range as the imaging pixels used for interpolation generation are used as the imaging pixels used when obtaining the image structure. Thereby, it is not necessary to separately read out image data from the image sensor 104 only for the determination of the image structure, and the processing load can be reduced and the processing speed can be improved. Of course, in order to obtain the image structure more accurately, not only the pixels in the upper and lower two rows of the pupil-divided pixel row but also a wider range of pixels may be read if the processing speed is sufficient. Also, the pixels used for interpolation are not limited to the above, and interpolation may be performed with reference to a wider range of pixels.

Next, an example of an interpolation method will be specifically described. In FIG. 5, first, when the entire pupil data generation unit 202 interpolates the entire pupil data D of the pupil division pixel p (x0, y0), the peripheral G pixel (pixel p (x0, y0-1), pixel p The image structure is determined from (x0, y0 + 1), pixel p (x0-1, y0), pixel p (x0 + 1, y0)). The image structure is determined by calculating the absolute value Dv of the difference between the upper and lower G pixels (pixel p (x0, y0-1) and pixel p (x0, y0 + 1)) of the pupil division pixel p (x0, y0) (Formula 1). Then, the absolute value Dh of the difference between the left and right G pixels (pixel p (x0-1, y0) and pixel p (x0 + 1, y0)) of the pupil division pixel p (x0, y0) is obtained (formula 2), The absolute value Dv is compared with the absolute value Dh of the difference.
Dv = | G (x0, y0-1) -G (x0, y0 + 1) | (Formula 1)
Dh = | G (x0-1, y0) -G (x0 + 1, y0) | (Formula 2)
In the case of Dv ≦ Dh, all pupil data D (x0, y0) of the pupil division pixel p (x0, y0) is obtained by (Equation 3).
D (x0, y0) = (G (x0, y0-1) + G (x0, y0 + 1)) / 2 (Formula 3)
When Dv> Dh, the total pupil data D (x0, y0) of the pupil division pixel p (x0, y0) is obtained by (Equation 4).
D (x0, y0) = (G (x0-1, y0) + G (x0 + 1, y0)) / 2 (Formula 4)
Similarly, other pupil-divided pixels p (x0-4, y0), pixel p (x0-2, y0), pixel p (x0 + 2, y0), and pixel p (x0 + 4, y0) are also interpolated from the surrounding G pixels. Thus, the entire pupil data D at each pupil division pixel position is obtained.

  For the G pixel of the imaging pixel in the same row as the pupil division pixel, the pixel data of the G pixel is used as the whole pupil data D at the G pixel position as it is. As a result, as shown in FIG. 6, at the pupil division pixel position, two pieces of image data of pupil division data A and all pupil data D generated by interpolation, and all pupil data at the G pixel position between the pupil division pixels. Using D, it is possible to detect the amount of positional deviation by the pupil division method.

  FIG. 7 is a graph illustrating an example of the pixel value of each pixel in the row y0. In FIG. 7, the solid line indicates the change in the brightness of the whole pupil image incident from the optical system 102, and the dotted line indicates the change in the brightness of the pupil divided image incident from the optical system 102. A black circle mark indicates the entire pupil data D at the G pixel position or the entire pupil data D generated by interpolation at the pupil division pixel position, and a white triangle mark indicates the pupil division data A at the pupil division pixel position. As shown in FIG. 3, since the light reception area of the pupil division pixel is only half of the light reception area of the imaging pixel, the output signal of the pupil division pixel read from the imaging element 104 is 1 of the output signal of the imaging pixel. In the graph of FIG. 7, the output level is adjusted by multiplying the pupil division data by a factor of about 2 so that the pupil division data and the entire pupil data have the same value. However, the ratio of the size of the pupil division data of the pupil division pixel to the size of the whole pupil data of the imaging pixel depends on the F value of the optical system 102, the pupil distance, the pixel position on the light receiving surface of the image sensor 104, and the like. Therefore, the coefficient may be adjusted based on these optical conditions. Alternatively, (average of all pupil data of neighboring imaging pixels / average of pupil division data of neighboring pupil division pixels) may be obtained and used as the coefficient.

  In this way, it is possible to detect the positional deviation amount of the subject image between the entire pupil data and the pupil division data having different pupil positions, and focus control can be performed by obtaining the defocus amount from the positional deviation amount. For example, in the case of FIG. 7, the pixel value other than x coordinates of 4 and 5 has an image structure with a brightness of about 200 and a valley at the center of about 70. In FIG. 7, the actual positional deviation amount between the whole pupil image indicated by the solid line and the pupil divided image indicated by the dotted line is about 0.5 pixels when attention is paid to the position of the valley between the solid line and the dotted line. Here, when the amount of positional deviation cannot be obtained accurately, an error occurs in the amount of defocus and a problem such as poor focus occurs. However, the positional deviation amount detection unit 203 of the electronic camera 101 according to the present embodiment is An accurate positional deviation amount can be obtained.

  Next, processing of the positional deviation amount detection unit 203 will be described. The positional deviation amount detection unit 203 includes the whole pupil data at the pupil division pixel position interpolated by the whole pupil data generation unit 202, the whole pupil data of the imaging pixels alternately arranged with the pupil division pixel, and the pupil at the pupil division pixel position. The position deviation amount of the subject image is obtained using the divided data.

  Here, in order to make it easy to understand the characteristics of the electronic camera 101 according to the present embodiment, a conventional method of detecting a positional deviation amount will be described with reference to FIG. FIG. 8 is a diagram corresponding to FIG. 7 described above, and the image structure indicated by the solid line of the whole pupil image and the dotted line of the pupil division image is the same as FIG. In FIG. 8, black square marks indicate all pupil data at the G pixel position in the same row as the pupil division pixels, and white triangle marks indicate pupil division data at the pupil division pixel positions as in FIG. In FIG. 8, as in FIG. 7, the pupil division data is multiplied by a factor of about two so that the values of the pupil division data and the entire pupil data are the same level.

  Thus, conventionally, since the pixel position of all pupil data and the pixel position of pupil division data are only obtained alternately for each pixel, the valley position obtained from all pupil data is the position where the x coordinate is 4. Thus, only the information that the valley position obtained from the pupil division data is the position of the x coordinate 5 can be obtained. For this reason, the amount of positional deviation between all pupil data and pupil division data is determined to be one pixel. Then, a defocus amount corresponding to the positional deviation amount of one pixel is obtained, and focus control is executed. However, since the actual positional shift amount is about 0.5 pixels, there is a problem that an error occurs in the defocus amount and the focus position becomes unclean.

  On the other hand, in the electronic camera 101 according to the present embodiment, as shown in FIG. 7, all pupil data at the pupil division pixel position is generated by interpolation from all pupil data of surrounding imaging pixels. Whole pupil data is obtained at all pixel positions including the pixel. As a result, the resolution of the image based on the whole pupil data is increased, and the positional deviation amount can be detected with higher accuracy than in the past. For example, in the case of FIG. 7, since the pixel values of the x coordinates 4 and 5 of the whole pupil data are about 100 and the pixel values of the other whole pupil data are about 200, there is a valley peak between the x coordinates 4 and 5. Can be estimated. On the other hand, in the conventional example of FIG. 8, since the pixel value of the x coordinate 4 of all pupil data is about 100 and the pixel values of all other pupil data are all about 200, the peak position of the valley is obtained with high accuracy. I can't.

  Next, a description will be given of a method for obtaining the positional deviation amount with a decimal precision and a high precision in the electronic camera 101 according to the present embodiment. In the example of FIG. 2, the pupil division pixels are arranged every other pixel from (x0-4) to (x0 + 4) on the x axis with the x axis column x0 as the center. For easy understanding, it is assumed that L pupil division pixels are arranged every other pixel from x0. In this case, the pupil divided pixels arranged every other pixel are p (x0, y0), p (x0 + 2, y0), p (x0 + 4, y0), p (x0 + 6, y0), p (x0 + 8, y0), ... P (x0 + 2 · (L−1), y0), which can be expressed as a pixel p (x0 + 2 · I, y0). I is an integer from 0 to (L-1).

  When the pixel value (pupil division data) of each pupil division pixel is represented by A, the pupil division data of each of the L pupil division pixels is A (x0, y0), A (x0 + 2, y0), A (x0 + 4). , Y0), A (x0 + 6, y0), A (x0 + 8, y0),... A (x0 + 2 · (L−1), y0), and pupil division data A (x0 + 2 · I, y0) Can be written. I is as described above.

  Further, when the interpolation-generated all-pupil data corresponding to the position of each pupil division pixel is represented by D, the interpolation-generated all-pupil data corresponding to each of the L pupil division pixels is D (x0, y0), D (x0 + 2, y0), D (x0 + 4, y0), D (x0 + 6, y0), D (x0 + 8, y0), ... D (x0 + 2 · (L-1), y0), all pupils It can be expressed as data D (x0 + 2 · I, y0). I is as described above.

  In accordance with the above definition, the difference d (I) between the pupil division data A and the whole pupil data D is obtained for each pupil division pixel by (Equation 5).

さらに、Ｌ個の瞳分割画素毎の差分ｄ（Ｉ）の絶対値の総和Ｓを（式６）により求める。尚、差分の絶対値の総和Ｓは、瞳分割データＡと全瞳データＤとの相関が大きいほど小さくなり、相関が小さいほど大きくなるので、以降の説明において相関値Ｓと称する。

Further, the sum S of absolute values of the difference d (I) for each of the L pupil-divided pixels is obtained by (Expression 6). The sum S of the absolute values of the differences is smaller as the correlation between the pupil division data A and the whole pupil data D is larger, and is larger as the correlation is smaller. Therefore, it will be referred to as a correlation value S in the following description.

尚、（式６）で求められる相関値Ｓは、瞳分割データＡと全瞳データＤの位置関係が図７のように固定されているので、相関値Ｓが最小であるか否かは判定できない。そこで、瞳分割データまたは全瞳データを少しずつずらしていき、相関値Ｓが最小になった時のずらし量を求め、これを位置ズレ量とする。

Note that the correlation value S obtained by (Equation 6) is determined as to whether or not the correlation value S is minimum since the positional relationship between the pupil division data A and the whole pupil data D is fixed as shown in FIG. Can not. Therefore, the pupil division data or the whole pupil data is shifted little by little to obtain the shift amount when the correlation value S is minimized, and this is used as the positional deviation amount.

  In an example to be described next, first, a correlation value S is obtained when the whole pupil data is shifted on the x coordinate in units of one pixel, and a pixel shift amount that minimizes the correlation value S is obtained. Here, assuming that the pixel shift amount is K (K is an integer), the correlation value S when K pixel shift is performed can be expressed as S (K). Therefore, (Expression 6) uses the variables I and K (Expression 7). )

ここで、Ｉは、座標Ｘ０の瞳分割画素をＩ＝０として、Ｉ＝０からＩ＝（Ｌ−１）までのＬ個の瞳分割画素の座標位置を示し、Ｋ画素ずらした時の差分ｄ（Ｉ，Ｋ）は（式８）で表すことができる。

Here, I indicates the coordinate position of the L pupil division pixels from I = 0 to I = (L−1), where I = 0 as the pupil division pixel at the coordinate X0, and the difference when K pixels are shifted. d (I, K) can be expressed by (Equation 8).

そして、（式７）においてＳ（Ｋ）が最小になる時のＫを求め、これをＫ０とする。また、この時の相関値Ｓ（Ｋ）の最小値はＳ（Ｋ０）となる。尚、上記の場合、図７に示したように、瞳分割データは１画素置きのデータであるが、全瞳データＤはＧ画素から得られる全瞳データと補間生成された全瞳データとを有するので、１画素単位で全瞳データが必ず存在する。これにより、全瞳データを１画素ずつずらして上式の計算を行うことができるので、精度良く位置ズレ量を検出することができる。

Then, K is calculated when S (K) is minimized in (Expression 7), and this is set as K0. Further, the minimum value of the correlation value S (K) at this time is S (K0). In the above case, as shown in FIG. 7, the pupil division data is data every other pixel, but the entire pupil data D is the total pupil data obtained from the G pixels and all the pupil data generated by interpolation. Therefore, all pupil data always exists in units of one pixel. As a result, the calculation of the above equation can be performed by shifting the entire pupil data by one pixel at a time, so that the positional deviation amount can be detected with high accuracy.

  FIG. 9 is a diagram schematically showing the above processing. FIG. 9A shows each pupil division pixel position from the difference d (0, 0) to the difference d ((L−1), 0) between the pupil division data A and the whole pupil data D when K = 0. It shows how to ask. Then, the sum of the absolute values from the difference d (0, 0) to the difference d ((L-1), 0) is obtained and set as the correlation value S (0). Similarly, in FIG. 9B, when obtaining the correlation value S (1) in the case of K = 1, in FIG. 9C, each pupil in obtaining the correlation value S (2) in the case of K = 2. The state of divided data and whole pupil data is shown.

  In this way, the correlation value S (K) is obtained while changing K in units of one pixel, and K that minimizes the correlation value S (K) is obtained. For example, in the case of FIG. 9, the correlation value S (0) when K = 0 is minimized. In the above example, K is a positive integer and K is changed as K = 0, +1, +2..., But K = 0, -1, -2, -3. Alternatively, K may be changed in a negative direction, or K may be changed to positive or negative as in K =... -3, -2, -1, 0, +1, +2, +3. Good.

  In this way, K when the correlation value S (K) is minimized can be obtained.

  Here, since K is an integer, the positional relationship between all pupil data and pupil division data changes only in units of pixels. For example, even if there is a position where the correlation value S is further reduced between pixels, it is converted into an integer. Therefore, an error occurs in the amount of positional deviation. Therefore, in the electronic camera 101 according to the present embodiment, the positional deviation amount Z of the actual image between the pixels is obtained with decimal accuracy.

  FIG. 10 is a graph showing an example of the correlation value S (K) when K is changed. In FIG. 10, correlation values S (K0), S (K0-1), and S (K0 + 1) at K0 when the correlation value S (K) is minimized and (K0 + 1) and (K0-1) before and after the correlation value S (K). The example of the size of is drawn. For example, in the case of FIG. 10, since S (K0) is smaller than S (K0 + 1), and S (K0-1) and S (K0) are substantially the same size, the actual image positional shift amount Z is (K0-1). ) And (K0).

  Here, in FIG. 10, the positional deviation amount Z with decimal precision can be obtained by, for example, (Equation 9) by a known interpolation method.

ここで、ｍａｘ（Ａ，Ｂ）は、ＡとＢの大きい方を選択することを意味する演算子である。
図１０の場合、Ｓ（Ｋ０）とＳ（Ｋ０−１）はほぼ同じなので、上記の式の分母と分子はほぼ同じ値になるので、Ｚ＝Ｋ０＋１／２となる。つまり、位置ズレ量Ｚは、Ｋ０の位置を基準（Ｋ０＝０）とした場合、Ｚ＝０．５となる。また、（Ｋ０−１）と（Ｋ０）の間は１画素分なので、位置ズレ量は０．５画素となる。

Here, max (A, B) is an operator that means selecting the larger of A and B.
In the case of FIG. 10, since S (K0) and S (K0-1) are almost the same, the denominator and the numerator of the above equation are almost the same value, so that Z = K0 + 1/2. That is, the positional deviation amount Z is Z = 0.5 when the position of K0 is used as a reference (K0 = 0). Further, since the interval between (K0-1) and (K0) is one pixel, the positional deviation amount is 0.5 pixel.

  Thus, in the electronic camera 101 according to the present embodiment, the positional deviation amount Z can be obtained with a small number of precision. For example, although the positional deviation amount is 1 pixel in the conventional technology described above, in the electronic camera 101 according to the present embodiment, the positional deviation amount is 0.5 pixel and can be accurately obtained with decimal precision. . Note that a method other than the above may be used as the interpolation method. Moreover, the calculation formula of the correlation value S is not limited to the above, and other methods may be applied. In particular, there is also known a method for detecting a positional deviation between data having different average levels. By applying such a method, a process of multiplying the pupil division data by a coefficient and performing level alignment with all pupil data can be omitted. Therefore, the processing load can be reduced. In this way, the positional deviation amount detection unit 203 can obtain the positional deviation amount with high accuracy.

  Next, the defocus amount calculation unit 204 calculates a defocus amount corresponding to the position shift amount with decimal precision obtained by the position shift amount detection unit 203. Here, the process for obtaining the defocus amount from the position shift amount is a well-known technique, and therefore a detailed description is omitted. For example, the defocus amount corresponding to the position shift amount is measured in advance, and the position shift amount and the defocus amount are determined. May be created, or the defocus amount may be obtained from the positional shift amount using a conversion formula. In this way, the defocus amount calculation unit 204 can obtain the defocus amount from the position deviation amount obtained by the position deviation detection unit 203.

  The focus lens control unit 205 controls the position of the focus lens of the lens optical system 102 according to the defocus amount obtained by the defocus amount calculation unit 204.

  In this way, the electronic camera 101 according to the present embodiment can perform highly accurate autofocus processing.

  Next, autofocus processing in the electronic camera 101 according to the present embodiment will be described with reference to the flowchart of FIG. The flowchart shown in FIG. 11 corresponds to the AF processing unit 200, which is a process executed in accordance with a program stored in advance in the control unit 108.

  (Step S <b> 101) The image data input unit 201 reads out image data from pixels in the vicinity of a row or column including pupil-divided pixels from the image sensor 104, and imports it into the image buffer 106.

  (Step S102) As described with reference to FIG. 5, the all-pupil data generation unit 202 detects the image structure.

  (Step S103) The all-pupil data generation unit 202 interpolates and generates all-pupil data at the pupil division pixel position using all-pupil data of the imaging pixels selected according to the image structure.

  (Step S104) As described with reference to FIGS. 9 and 10, the positional deviation amount detection unit 203 performs pupil division data of pupil division pixels, all pupil data of imaging pixels, and all pupils generated by interpolation of pupil division pixels. A positional shift amount between the pupil division data and the whole pupil data is detected using the data and the whole pupil data of the imaging pixels in the same row as the pupil division pixels.

  (Step S105) The defocus amount calculation unit 204 associates the position shift amount and the defocus amount stored in advance in the memory 109 or the like based on the position shift amount detected by the position shift amount detection unit 203. The defocus amount is calculated using the conversion formula.

  (Step S106) The focus lens control unit 205 moves the position of the focus lens of the optical system 102 in accordance with the defocus amount calculated by the defocus amount calculation unit 204, and the subject image is focused on the light receiving surface of the image sensor 104. Control to do.

  In this way, the electronic camera 101 according to the present embodiment can perform highly accurate autofocus processing.

[Placement example of pupil division pixel]
(Modification 1)
Next, Modification Example 1 of the pixel arrangement of the pupil division pixels will be described. In the above embodiment, as described with reference to FIG. 2, the pupil division pixels and the imaging pixels are alternately arranged one by one. However, in Modification 1, as shown in FIG. The pupil division pixels are arranged continuously in the part. For example, in the case of FIG. 12, the pupil division pixels are the pixels p (x0-3, y0), p (x0-2, y0), p (x0-1, y0), p (x0, y0), p (x0 + 1, y0), p (x0 + 2, y0), and p (x0 + 3, y0). In the case of FIG. 12 as well, as in the previous embodiment, the amount of positional deviation can be obtained with high accuracy by interpolating the entire pupil data at the pupil division pixel position from the entire pupil data of the surrounding imaging pixels. it can. In the example of FIG. 12, for example, an average value of all the pupil data of the imaging pixels above and below the pupil division pixel can be interpolated as the whole pupil data of the pupil division pixel without determining the image structure. For example, in the case of FIG. 12, the total pupil data of the pupil division pixel (x0, y0) is the average value of the total pupil data of the G pixel in the upper row and the total pupil data of the G pixel in the lower row. , Y0). Similarly, the whole pupil data of the pupil division pixel (x0 + 1, y0) is obtained by calculating the average value of the whole pupil data of the R pixel in the upper row and the whole pupil data of the R pixel in the lower row as the pupil division pixel (x0 + 1, y0). All pupil data. Note that in the case of performing interpolation generation from imaging pixels of different colors, there may be a level difference depending on the color component of the subject image. For example, all pupil data of the pupil division pixel (x0 + 1, y0) All pupil data of pixel (x0, y0-1) and G pixel (x0 + 2, y0-1) and an average value of four pixels of G pixel (x0, y0 + 1) and G pixel (x0 + 2, y0 + 1) diagonally below The whole pupil data of the divided pixel (x0 + 1, y0) may be used.

Using the entire pupil data and pupil division data obtained in this way, the amount of positional deviation can be detected as in the previous embodiment. Then, the focus control is performed by obtaining the defocus amount from the positional deviation amount. In particular, in the previous embodiment, since the entire pupil data was obtained for each pixel, the resolution was increased, and highly accurate positional deviation detection could be performed. However, in this modification, not only the entire pupil data but also the pupil Since the divided data is also obtained for each pixel, it is possible to detect the positional deviation with higher accuracy.
(Modification 2)
Next, Modification Example 2 of the pixel arrangement of the pupil division pixels will be described. In Modification 2, as shown in FIG. 13, the pupil division pixels are arranged every two pixels. For example, in the case of FIG. 13, the pupil-divided pixels are arranged every two pixels such as pixels p (x0-3, y0), p (x0, y0), and p (x0 + 3, y0). Also in the second modification, as in the previous embodiment, by determining the image structure and interpolating and generating all the pupil data at the pupil division pixel positions from all the pupil data of the surrounding imaging pixels, the positional deviation amount can be reduced. It can be obtained with high accuracy. In Modification 2, since the pupil division pixels are arranged every two pixels, the resolution of the pupil division data is lower than that in the previous embodiment, but all pupil data is generated by interpolation and is obtained for each pixel. It is possible to detect a positional shift with higher accuracy than when the conventional technique is applied in the pixel arrangement. On the contrary, in this modified example, the number of pixels where the imaging pixels of the imaging device 104 are replaced with pupil-divided pixels is reduced, so that it is possible to prevent image quality deterioration.

Using the entire pupil data and pupil division data obtained in this way, the amount of positional deviation can be detected as in the previous embodiment, and the focus control is performed by obtaining the defocus amount from the amount of positional deviation. be able to.
[Modification Example of Image Sensor 104]
Next, a modified example of the image sensor 104 will be described. In the previous embodiment, all pupil data for interpolating and generating all pupil data at the pupil division pixel position from the image sensor 104 is read out from a plurality of peripheral imaging pixels individually and taken into the image buffer 106. Thereafter, arithmetic processing such as addition averaging is performed, but there is a problem that the processing load of the control unit 108 increases. Therefore, as a modification of the image sensor 104, as shown in FIG. 14, a circuit (addition averaging circuit 301) that averages the pixel values of two imaging pixels above and below the pupil-divided pixel is provided in the image sensor 104, and the pupil All pupil data after interpolation of the divided pixel positions can be read directly from the image sensor 104. This eliminates the need for time for reading out all pupil data from the imaging pixels above and below the pupil-divided pixels and processing for obtaining all-pupil data at the pupil-divided pixel position, and eliminates the need for the all-pupil data generation unit 202. Therefore, the processing load can be greatly reduced.

  In this way, it is not necessary to interpolate the entire pupil data at the pupil division pixel position after reading from the image sensor 104, and the positional deviation amount can be detected with high accuracy as in the previous embodiment. Focus control can be performed by obtaining the defocus amount from the shift amount.

  As described above, the defocus amount detection device and the electronic camera according to the present invention have been described by way of examples in the respective embodiments, but may be implemented in various other forms without departing from the spirit or main features thereof. Can do. Therefore, the above-described embodiment is merely an example in all respects and should not be interpreted in a limited manner. The present invention is defined by the claims, and the present invention is not limited to the text of the specification. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

DESCRIPTION OF SYMBOLS 101 ... Electronic camera; 102 ... Optical system; 103 ... Mechanical shutter; 104 ... Image pick-up element; 105 ... A / D conversion part; 106 ... Image buffer; Processing unit; 108 ... Control unit; 109 ... Memory; 110 ... Display unit; 111 ... Operation unit; 112 ... Memory card IF; 12a ... Memory card 1; AF processing unit; 201, image data input unit, 202, all pupil data generation unit, 203, position shift amount detection unit, 204, defocus amount calculation unit, 205, focus lens control Part

Claims (8)

An image for reading pupil division data of the pupil division pixel and whole pupil data of the imaging pixel around the pupil division pixel from an imaging device in which pupil division pixels are arranged in a part of the two-dimensionally arranged imaging pixels. A data input section;
An all-pupil data generating unit that interpolates and generates all-pupil data at the pupil-divided pixel position from the all-pupil data of the imaging pixels read by the image data input unit;
A positional shift amount between the whole pupil data of the pupil division pixel position and the whole pupil data of the imaging pixel generated by interpolation by the whole pupil data generation unit and the pupil division data read from the pupil division pixel. A position shift detection unit to detect,
A defocus amount detection apparatus, comprising: a defocus amount calculation unit that obtains a defocus amount according to a position shift amount detected by the position shift detection unit. The defocus amount detection device according to claim 1,
The all-pupil data generation unit determines the direction of the image structure based on the all-pupil data read from the imaging pixels, and interpolates and generates all-pupil data at the pupil division pixel position according to the determination result. A defocus amount detection device, wherein the imaging pixel to be referred to is selected. In the defocus amount detection apparatus according to claim 2,
When determining the direction of the image structure, the whole pupil data generation unit compares the difference between the upper and lower whole pupil data adjacent to the pupil division pixel and the difference between the left and right whole pupil data. defocus amount detection apparatus characterized by generating interpolating all pupil data of the pupil division pixel position by using the entire pupil data smaller. In the defocus amount detection apparatus according to claim 2 or 3,
The all-pupil data generation unit includes the entire pupil data for determining the direction of the image structure and all-pupil data referred to when interpolating and generating all-pupil data at the pupil-divided pixel position. A defocus amount detection device using the whole pupil data of the imaging pixels in the same range. In the defocus amount detection apparatus according to any one of claims 1 to 3,
The defocus amount detection device, wherein the all pupil data generation unit generates all pupil data for all pixel positions in a column in which the pupil division pixels are arranged. The defocus amount detection device according to any one of claims 1 to 5, wherein the image data input unit converts the pupil division data and the whole pupil data from the row in which the pupil division pixels are arranged. The defocus amount detection apparatus, wherein the whole pupil data is read from the upper and lower rows of the row where the pixels are arranged. The defocus amount detection device according to any one of claims 1 to 5, wherein the image data input unit adds and averages all pupil data of the plurality of imaging pixels around the pupil division pixel in the imaging element. The defocus amount detection device is characterized in that the read data is read out as whole pupil data at the pupil division pixel position. An electronic camera comprising the defocus amount detection device according to any one of claims 1 to 5.

Images