JP5043730B2 - Imaging apparatus, control method therefor, image processing apparatus, and image processing method - Google Patents

Description

  The present invention relates to an imaging apparatus such as a digital camera, and more particularly to an imaging apparatus having a function of confirming a focus state of a recorded image by imaging. The present invention also relates to an image processing apparatus capable of confirming a focus state of a recorded image generated by imaging.

  There is a desire to confirm the focus state of a recorded image acquired by an imaging apparatus using the imaging apparatus or a personal computer (PC). In order to realize such a demand, a method of enlarging and displaying an area of a recorded image for which a focus state is desired to be confirmed on a display of an imaging apparatus or a monitor of a PC is used.

  Patent Document 1 discloses an imaging apparatus that includes an optical member that divides an exit pupil of an imaging optical system, projects an object image onto a focus detection imaging element through the optical member, and displays a split image on a display. Has been.

  Further, Patent Document 2 includes an optical element that enables focus detection by a phase difference detection method on the imaging surface of the image sensor, projects a subject image on the image sensor through the light beam element, and displays a split image on a display. An imaging device for display is disclosed.

Furthermore, in Patent Document 3, the value corresponding to the distance between the image plane of the imaging optical system and the light receiving surface of the imaging element is used as the amount of deviation from the signal from the distance measuring unit and the arrangement of the lenses of the imaging optical system. An imaging device that easily obtains a split image corresponding to the amount of deviation is disclosed.
JP 59-50671 A JP 2006-154506 A JP 2001-309210 A

  In any of the imaging devices disclosed in Patent Documents 1 to 3, a split image is displayed as an auxiliary display when manual focusing is performed before imaging. A method for displaying a split image with respect to a photographed image is not disclosed.

  In addition, the imaging device disclosed in Patent Document 3 has a problem in that when there are a plurality of subjects at different distances in the split image display range, the amount of shift is different from that of the conventional split image.

  The present invention provides an imaging apparatus, a control method thereof, an image processing apparatus, and an image processing method that can use an image such as a split image indicating a focus state at the time of imaging when a recorded image is reproduced after imaging.

An imaging apparatus according to an aspect of the present invention generates first image data based on a signal obtained by photoelectrically converting a first optical image formed by a light beam incident on an imaging optical system from a subject. Obtained by photoelectrically converting two second optical images having a phase difference corresponding to the focus state of the imaging optical system, formed by one image generating means and two light beams divided among the light beams. Second image generating means for generating second image data based on the received signal, recording means for recording the first image data and the second image data on a recording medium for storage, Reading means for reading out the first and second image data from the recording medium, and reproduction display of the second image data or the image data generated from the second image data together with the first image data on the display means Playback means First reproduction means operated by the author, and the reproduction means generates split image image data corresponding to the phase difference from the second image data, together with the first image data The split image is reproduced and displayed on the display means, and the reproduction means changes the display position of the split image on the display means in accordance with an operation of the first operation means .
An imaging apparatus according to another aspect of the present invention provides first image data based on a signal obtained by photoelectrically converting a first optical image formed by a light beam incident on an imaging optical system from a subject. Photoelectric conversion of two second optical images having a phase difference in accordance with the focus state of the imaging optical system, formed by a first image generating means for generating the image and two split light beams out of the light beams Second image generating means for generating second image data based on the signal obtained in this manner, and recording for recording the first image data and the second image data on a recording medium for storage Means, reading means for reading out the first and second image data from the recording medium, and displaying the second image data or image data generated from the second image data together with the first image data Replay display on the means And a second operation means operated by an operator, wherein the reproducing means enlarges a partial area of the first image data and reproduces and displays it on the display means. The reproduction means generates split image image data corresponding to the phase difference from the second image data, and causes the display means to reproduce and display the split image together with the first image data, The reproduction means is characterized in that when the second operation means is operated, the display position of the split image on the display means is fixed and the enlargement area by the enlargement process is changed.

An image processing apparatus according to another aspect of the present invention provides a first image based on a signal obtained by photoelectrically converting a first optical image formed by a light beam incident on an imaging optical system from a subject. First image generation means for generating data and two second optical images formed by two divided light beams out of the light beams and having a phase difference corresponding to a focus state of the imaging optical system are photoelectrically converted. Second image generation means for generating second image data based on the signal obtained by conversion, and the first image data and the second image data are recorded on a recording medium for storage. An image processing apparatus to which the recording medium on which the first image data and the second image data are recorded by an imaging device having a recording unit is mounted, wherein the first image data and the second image data Image data from the recording medium Read out means, reproduction means for reproducing and displaying the second image data or the image data generated from the second image data together with the first image data on the display means, and a first operated by an operator. 1, and the reproducing means generates split image image data corresponding to the phase difference from the second image data, and displays the split image together with the first image data. The reproduction unit displays the split image on the display unit according to an operation of the first operation unit .
An image processing apparatus according to another aspect of the present invention provides a first image based on a signal obtained by photoelectrically converting a first optical image formed by a light beam incident on an imaging optical system from a subject. First image generation means for generating data and two second optical images formed by two divided light beams out of the light beams and having a phase difference corresponding to a focus state of the imaging optical system are photoelectrically converted. Second image generation means for generating second image data based on the signal obtained by conversion, and the first image data and the second image data are recorded on a recording medium for storage. An image processing apparatus to which the recording medium on which the first image data and the second image data are recorded by an imaging device having a recording unit is mounted, wherein the first image data and the second image data Image data from the recording medium Read out means, reproduction means for reproducing and displaying the second image data or the image data generated from the second image data together with the first image data on the display means, and a first operated by an operator. 2, and the reproduction means can enlarge the partial area of the first image data and reproduce and display it on the display means. The reproduction means Split image data corresponding to the phase difference is generated from second image data, and the split image is reproduced and displayed together with the first image data on the display means. When the means is operated, the display position of the split image on the display means is fixed and the enlargement area by the enlargement process is changed.

According to another aspect of the present invention, there is provided a control method for an image pickup apparatus based on a signal obtained by photoelectrically converting a first optical image formed by a light beam incident on an image pickup optical system from a subject. A step of generating the image data, and photoelectrically converting two second optical images having a phase difference corresponding to a focus state of the imaging optical system, formed by the two light beams divided among the light beams. Generating second image data based on the obtained signal; recording the first image data and the second image data on a recording medium for storage; and Reading the second image data from the recording medium, and causing the display means to reproduce and display the second image data or the image data generated from the second image data together with the first image data. Yes In the reproduction and display step, split image image data corresponding to the phase difference is generated from the second image data, and the split image is reproduced and displayed on the display unit together with the first image data, In the reproduction display step, the display position of the split image on the display means is changed according to the first operation means operated by the operator .
According to another aspect of the present invention, there is provided a control method for an image pickup apparatus based on a signal obtained by photoelectrically converting a first optical image formed by a light beam incident on an image pickup optical system from a subject. A step of generating the image data, and photoelectrically converting two second optical images having a phase difference corresponding to a focus state of the imaging optical system, formed by the two light beams divided among the light beams. Generating second image data based on the obtained signal; recording the first image data and the second image data on a recording medium for storage; and Reading the second image data from the recording medium, and causing the display means to reproduce and display the second image data or the image data generated from the second image data together with the first image data. Yes In the reproducing and displaying step, an enlargement process for enlarging a partial area of the first image data and reproducing and displaying it on the display means is possible. In the reproducing and displaying step, the second image data is displayed. In the step of generating image data of a split image corresponding to the phase difference, reproducing and displaying the split image together with the first image data on the display means, and performing the reproduction display, a second operation performed by an operator is performed. When the operation means is operated, the display position of the split image on the display means is fixed and the enlargement area by the enlargement process is changed.

Furthermore, an image processing method according to another aspect of the present invention provides first image data based on a signal obtained by photoelectrically converting a first optical image formed by a light beam incident on an imaging optical system from a subject. Photoelectric conversion of two second optical images having a phase difference in accordance with the focus state of the imaging optical system, formed by a first image generating means for generating the image and two split light beams out of the light beams Second image generating means for generating second image data based on the signal obtained in this manner, and recording for recording the first image data and the second image data on a recording medium for storage Reading out the first image data and the second image data from the recording medium on which the first image data and the second image data are recorded by an imaging device comprising: The second along with the image data And reproducing and displaying image data generated from the image data or the second image data on a display means, and in the reproducing and displaying step, splitting is performed from the second image data according to the phase difference. In the step of generating image data of an image, causing the display unit to reproduce and display the split image together with the first image data, and in the step of reproducing and displaying the image data, according to the first operation unit operated by an operator The display position of the split image on the display means is changed .
Furthermore, an image processing method according to another aspect of the present invention provides first image data based on a signal obtained by photoelectrically converting a first optical image formed by a light beam incident on an imaging optical system from a subject. Photoelectric conversion of two second optical images having a phase difference in accordance with the focus state of the imaging optical system, formed by a first image generating means for generating the image and two split light beams out of the light beams Second image generating means for generating second image data based on the signal obtained in this manner, and recording for recording the first image data and the second image data on a recording medium for storage Reading out the first image data and the second image data from the recording medium on which the first image data and the second image data are recorded by an imaging device comprising: The second along with the image data Image data or image data generated from the second image data is reproduced and displayed on a display means. In the reproduction and display step, a part of the first image data is enlarged. In the step of reproducing and displaying, split image image data corresponding to the phase difference is generated from the second image data, and the first image data is reproduced. At the same time, in the step of reproducing and displaying the split image on the display means, the display position of the split image on the display means is fixed when the second operating means operated by an operator is operated. Then, the enlargement area by the enlargement process is changed.

  According to the present invention, not only the first image as the subject image recorded by imaging but also the second image indicating the focus state at the time of imaging is recorded on the recording medium. By reproducing these, the focus state of the first image can be confirmed.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows a circuit configuration of an imaging apparatus such as a digital camera that is Embodiment 1 of the present invention.

  In the imaging apparatus 1, reference numeral 101 denotes an imaging lens (imaging optical system) including a variable power lens and a focus lens, and 102 denotes a lens driving circuit that drives the focus lens. Reference numeral 103 denotes an aperture for adjusting the amount of light.

  Reference numeral 104 denotes an imaging element (sensor) that photoelectrically converts an optical image formed by a light beam incident on the imaging lens 101 from a subject and outputs an electrical signal (analog imaging signal). The image sensor 104 is configured by a CCD sensor or a CMOS sensor. Further, the image sensor 104 includes an image pickup pixel for color image pickup and an AF pixel for focus detection. As will be described in detail later, the imaging pixel photoelectrically converts a subject image (first optical image) formed by a light beam incident on the imaging lens 101 from the subject. On the other hand, the AF pixel photoelectrically converts two subject images (second optical images) formed by two so-called pupil-divided light beams among the light beams incident on the imaging lens 101 from the subject.

  Reference numeral 105 denotes a synchronization signal generator (hereinafter referred to as SSG) that generates a horizontal synchronization signal HD and a vertical synchronization signal VD, and reference numeral 106 denotes a timing for generating a control signal for operating the image sensor 104 in synchronization with the synchronization signals HD and VD. It is a generator (hereinafter referred to as TG).

  Reference numeral 107 denotes an A / D conversion circuit that converts an analog image signal output from the image sensor 104 into a digital image signal. Reference numeral 108 denotes a correction flag circuit that outputs a flag indicating the position of the AF pixel provided in the image sensor 104. A development pixel correction circuit 109 interpolates the pixel value of the AF pixel included in the digital image pickup signal output from the A / D conversion circuit 107 with reference to the flag output from the correction flag circuit 108.

  Reference numeral 110 denotes an AF pixel selection circuit that extracts pixel values of AF pixels included in the digital imaging signal output from the A / D conversion circuit 107. A development processing circuit 111 performs development processing such as color conversion processing on the digital imaging signal to generate color image data (first image data).

  A first image generation unit is mainly configured by the imaging pixels of the imaging element 104, the A / D conversion circuit 107, the development pixel correction circuit 109, and the development processing circuit 111. In addition, the second image generation unit is mainly configured by the AF pixels of the image sensor 104, the A / D conversion circuit 107, and the AF pixel selection circuit 110.

  A phase difference AF circuit 112 detects a phase difference from image signals (A image signal and B image signal) obtained by a plurality of pairs of AF pixels and calculates a defocus amount (defocus amount).

  Reference numeral 114 denotes a DRAM, 113 denotes a memory control circuit serving as an interface with the DRAM 114, 115 denotes a scaling circuit that changes the size of image data, and 116 denotes a system controller that determines the mode and parameters of each circuit.

  118 is a monitor for displaying image data, 117 is a video modulation circuit for modulating the image data to be displayed on the monitor 118, and 119 is a compression circuit for compressing the image data by a compression method such as the JPEG method. Reference numeral 121 denotes a removable recording medium (hereinafter referred to as a memory card) that records compressed image data for storage.

  A card control circuit 120 serves as an interface with the memory card 121, and a decompression circuit 122 decompresses the compressed data.

  In the imaging apparatus configured as described above, the light beam that has passed through the imaging lens 101 is adjusted to an appropriate amount of light at the aperture 103 and forms a subject image on the imaging element 104. The image sensor 104 photoelectrically converts the subject image and outputs an analog image signal.

  The TG 106 generates a timing signal so that the image sensor 104 operates in synchronization with the synchronization signals HD and VD generated in the SSG 105, and controls the operation of the image sensor 104. The analog imaging signal output from the imaging element 104 is converted into a digital imaging signal by the A / D conversion circuit 107.

  The correction flag circuit 108 reads out AF pixel position information stored in a ROM (not shown) and outputs it to the development pixel correction circuit 109 and the AF pixel selection circuit 110.

  The development pixel correction circuit 109 interpolates the pixel value of the AF pixel that is a defective pixel in the digital image signal read from the image sensor 104 using the pixel values of the surrounding image pixels. Then, the digital imaging signal in which the defective pixel is interpolated is output to the development processing circuit 111.

  The development processing circuit 111 performs development processing such as color conversion processing, gamma processing, and white balance processing on the digital imaging signal output from the development pixel correction circuit 109 to generate color image data. Color image data is written into the DRAM 114 through the memory control circuit 113.

  The AF pixel selection circuit 110 extracts the pixel value data of the AF pixel from the digital image pickup signal output from the A / D conversion circuit 107 in accordance with the AF pixel position information input from the correction flag circuit 108, and the A image signal and A B image signal is generated. The A image signal and the B image signal are signals having a phase difference corresponding to the focus state of the imaging lens 101, and are input to the phase difference AF circuit 112 and the memory control circuit 113.

  The memory control circuit 113 writes the input A image signal and B image signal in the DRAM 114.

  The phase difference AF circuit 112 calculates the phase difference of the input A image signal and B image signal by correlation calculation, and outputs information on the phase difference to the system controller 116.

  The system controller 116 calculates a defocus amount of the imaging lens 101 from the phase difference information, and obtains a target drive amount of the focus lens for obtaining a focused state from the defocus amount. Then, a drive signal corresponding to the target drive amount is output to the lens drive circuit 102 to move the focus lens. In this way, phase difference detection type autofocus (AF) using AF pixels provided in the image sensor 104 can be performed.

  Note that the imaging lens 101 of the present embodiment can also perform manual focus by moving the focus lens by operating a manual focus ring (focus operation member) (not shown).

  The color image data developed in the development processing circuit 111 and written in the DRAM 114 is read out to the scaling circuit 115 by the memory control circuit 113. The scaling circuit 115 scales the size of the color image data to a size for display on the monitor 118 or a size for recording on the memory card 121. The scaled color image data is written into the DRAM 114 by the memory control circuit 113.

  The A image signal and the B image signal written in the DRAM 114 have two subjects formed by two pupil-separated light beams because AF pixels are discretely arranged over the entire image sensor 104 as will be described later. It is also monochromatic image data corresponding to an image. The image data (second image data) of the A image and the B image is read by the memory control circuit 113 to the scaling circuit 115.

  The scaling circuit 115 scales the image data of the A image and the B image to a size for display on the monitor 118. The image data of the scaled A and B images is written into the DRAM 114 by the memory control circuit 113. The scaled color image data and the image data of the A and B images are read from the DRAM 114 to the video modulation circuit 117, converted into a format such as NTSC or PAL, and output to the monitor 118.

  As a result, as shown in FIG. 8A, on the color image data (hereinafter referred to as “main image data”) 1300, a display A image 1320 which is image data of A image and a display B image 1321 which is image data of B image are constituted. The split images are displayed in a superimposed manner. More specifically, the split image is composed of image data generated (cut out) from image data of an A image and a B image.

  The main image data scaled by the scaling circuit 115 and the image data of the A image and the B image are read from the DRAM 114 to the compression circuit 119, compressed and written again in the DRAM 114, and then the card control circuit 120. Read out. Then, the card control circuit 120 converts the compressed image data into an image file 1500 having a file format shown in FIG. 10 and records the image file 1500 on a memory card 121 that is a recording medium that can be attached to and detached from the imaging apparatus 1. The system controller 116 and the card control circuit 120 mainly correspond to recording means.

  The image file 1500 is composed of main image data (JPEG data) and a header. In the header, various information at the time of imaging, model information of the imaging apparatus 1, and image data of A and B images (FIG. Is stored as pupil division data).

  Note that the memory card 121 is illustrated as the storage of the image file 15000. However, a storage memory or the like may not be removable as long as it is a recording medium for storage. Further, the image pickup apparatus 1 may be stored in a server as a recording medium via a communication network, even if the image pickup apparatus 1 is detachable and not provided.

  Here, the pixel arrangement of the image sensor 104 is shown in FIG. The basic pixel array is a Bayer array including green pixels, red pixels, and blue pixels. Each pixel has an opening through which a light beam received by the photoelectric conversion unit passes. In FIG. 2, the opening in the green pixel is indicated as G, the opening in the red pixel is indicated as R, the opening in the blue pixel is indicated as B, and the opening in the achromatic (= white) pixel is indicated as S1 and S2.

  A color filter that allows green light to pass through the opening G of the green pixel, a color filter that allows red light to pass through the opening R of the red pixel, and a color filter that allows blue light to pass through the opening B of the blue pixel. Each is arranged. On the other hand, no color filter is arranged in the openings S1 and S2 of the achromatic pixel. Moreover, a photoelectric conversion part exists under each opening part (back side). Further, a microlens (11 in FIG. 3) is provided on each opening (on the front surface side). That is, one pixel is configured by one photoelectric conversion unit, a color filter (except for achromatic pixels) disposed thereon, and a microlens disposed further thereon.

  The green pixel, the red pixel, and the blue pixel are used as imaging pixels, and the achromatic pixel is used as an AF pixel.

  That is, the image sensor 104 includes an image pickup pixel group (first pixel group) that photoelectrically converts a subject image (first optical image) formed by the light flux from the image pickup lens 101. In addition, an AF pixel group (second pixel group) that photoelectrically converts two optical images (second optical images) formed by pupil-divided light beams out of the light beams from the imaging lens 101 is provided. An output signal from the imaging pixel group is used to generate main image data, and an output signal from the AF pixel group is used to generate phase difference detection AF and split images (image data of A and B images).

  The AF pixel constitutes one pair by two pixels obliquely adjacent to each other, and the pair is discretely (and periodically) arranged over the entire imaging element 104 (imaging pixel group).

  FIG. 3 shows an enlarged view of four pixels including two AF pixels (upper left and lower right pixels) and two green pixels (upper right and lower left) that are paired with each other. Here, the shape of the opening of each pixel will be described.

  Of the two AF pixels, one AF pixel (opening S1) is formed at the position of the red pixel in the Bayer array, and the other AF pixel (opening S2) is formed at the position of the blue pixel in the Bayer array.

  Reference numeral 11 denotes a microlens disposed on each opening as described above. 37a and 37b are the centers of the microlenses 11 in two AF pixels adjacent to each other in an oblique direction. Reference numeral 36 denotes a straight line passing through the center of the microlens 11 in the two AF pixels.

  Reference numeral 38a denotes an opening of a red pixel when the one AF pixel is not arranged. Reference numeral 38b denotes an opening of a blue pixel when the other AF pixel is not disposed.

  Reference numerals 39a and 39b denote openings of the two AF pixels, which have a shape obtained by reducing the red pixel opening 38a and the blue pixel opening 38b around the reduction centers 35a and 35b, respectively. Here, the reduction centers 35a and 35b are located on the straight line 36 away from each other in the opposite direction (direction approaching each other) from the centers 37a and 37b of the microlens 11 of the AF pixel by an equal distance.

  Since the openings 39a and 39b of two AF pixels adjacent to each other have a reduced shape around the reduction centers 35a and 35b, they are biased in different directions with respect to the centers 37a and 37b of the microlens 11. The openings 39 a and 39 b are each symmetrical with respect to a straight line 46 orthogonal to the straight line 36.

  According to such a configuration, it is equivalent to the case where the imaging lens 101 is viewed from one AF pixel and the other AF pixel when the pupil of the imaging lens 101 is divided symmetrically. That is, the microlens 11 and the openings 39a and 39b (in other words, the light shielding layer in which the openings 39a and 39b are formed) constitute a so-called pupil division optical system.

  Two images (A image and B image) that are closer to each other are formed on two adjacent AF pixels as the number of pixels of the image sensor 104 increases. In a state where the imaging lens 101 is in focus with respect to the subject, there is no phase difference between the outputs (A image signal and B image signal) obtained from the two AF pixels.

  On the other hand, if the imaging lens 101 is out of focus, a phase difference occurs between the A image signal and the B image obtained from the two AF pixels. The direction of the phase difference is reversed between the front pin state and the rear pin state.

  4 and 5 show the relationship between the phase difference and the focus state. In these drawings, two AF pixels adjacent to each other (paired) are shown as S1 and S2, respectively. Further, the imaging pixels are omitted, and a plurality of pairs of AF pixels S1 and S2 are shown close to each other on the same cross section. In these figures, the pair of AF pixels S1 and S2 is shown to be covered with the same microlens 301. However, as described above, the AF pixels S1 and S2 are covered with separate microlenses. ing.

  Light from a specific point of the subject is split into a light beam L1 that enters the AF pixel S1 through the pupil for the AF pixel S1 and a light beam L2 that enters the AF pixel S2 through the pupil for the AF pixel S2. The two light beams L1 and L2 are collected at one point on the surface of the microlens 301 as shown in FIG. 4 when the imaging lens 101 is in focus at a specific point. The same image is formed on the AF pixels S1 and S2. Thereby, the A image signal read from the AF pixel S1 and the B image signal read from the AF pixel S2 are the same.

  On the other hand, when the imaging lens 101 is not in focus with respect to the specific point, the light beams L1 and L2 intersect at a position different from the surface of the microlens 301 as shown in FIG. The distance between the surface of the microlens 301 and the intersection of the two light beams L1 and L2, that is, the defocus amount is x. In addition, it is assumed that an image shift amount (phase difference) on the AF pixels S1 and S2 generated in this state corresponds to n pixels.

When the pixel pitch is d, the distance between the centers of gravity of the two pupils is Daf, and the distance from the principal point of the imaging lens 101 to the focal position is u, the defocus amount x is
x = n × d × u / Daf (1)
Is required.

Since u is considered to be substantially equal to the focal length f of the imaging optical system 1, the defocus amount x is
x = n × d × f / Daf (2)
You can also ask for it.

  FIG. 6 shows the A image signal (S1 image) and the B image signal (S2 image) read from each of the AF pixels S1 and S2. A shift amount (that is, an image shift amount) n × d occurs in these A image and B image signals. The phase difference AF circuit 112 obtains a shift amount (phase difference) n × d between the pair of image signals using a correlation calculation method, and the system controller 116 uses the equation (1) or (2) from the phase difference. A defocus amount x is obtained. Based on the calculated defocus amount, the system controller 116 calculates a target drive amount of the focus lens for keeping the defocus amount x within a predetermined range (preferably zero). Then, the focus lens is moved by the calculated target driving amount through the lens driving circuit 102. Thereby, a focused state by the phase difference detection method AF is obtained.

  Next, the flow of imaging processing (control method) in the imaging mode in the imaging apparatus of the present embodiment will be described with reference to FIG. This process is executed according to a computer program stored in the internal memory of the system controller 116.

  When the operator turns on the main switch of the imaging apparatus shown in FIG. 1 (S101), the system controller 116 executes focus detection of the imaging lens 101 (S102). At this time, an output signal from the AF pixel of the image sensor 104 is used. Then, the defocus amount of the imaging lens 101 is calculated, and further the target drive amount of the focus lens is calculated.

  Next, based on the target drive amount calculated in S102, the system controller 116 outputs a drive signal to the lens drive circuit 102 to drive the focus lens to the in-focus position (S103).

  Next, the system controller 116 operates the image sensor 104 through the SSG 105 and the TG 106 to generate a so-called live view image (S104). The live view image is displayed on the monitor 118 as a moving image for the electronic viewfinder (S105).

  Subsequently, the system controller 116 determines whether or not the imaging switch SW2 is turned on by the operator (S106). If the operation switch SW2 is not ON, the process returns to S101.

  On the other hand, if the operation switch SW2 is ON, the system controller 116 causes the image sensor 104 to perform a main imaging operation for generating main image data through the SSG 105 and the TG 106 (S107). Thus, the main image data is generated in the development processing circuit 111 based on the output signal from the imaging pixel of the imaging element 104. At this time, the image data of the A image and the image data of the B image are also generated by the AF pixel selection circuit 110 based on the output from the AF pixel of the image sensor 104. As described above, the main image data and the image data (split image) of the A image and the B image are displayed on the monitor 118 via the video modulation circuit 117 or the like (S108), or stored in the memory via the card control circuit 120 or the like. Or recorded on the card 121. (S109). When the main imaging operation is thus completed, the system controller 116 returns to S101 and resumes focus detection and live view image display.

  As described above, according to the present embodiment, the main image data acquired by imaging, and the image data of the A image and the B image corresponding to the focus state at the time of capturing the main image data are obtained from the imaging apparatus 1. Can be recorded on the removable memory card 121. Therefore, as will be described later, when reproducing the main image data (recorded image) after imaging, a split image indicating the focus state at the time of imaging can be used.

  Next, as a second embodiment of the present invention, the reproduction function of the imaging device 1 described in the first embodiment will be described.

  The card control circuit 120 receives the main image data and the A image from the memory card 121 (the image data recorded by another imaging device) on which the main image data and the image data of the A image and the B image are recorded. The image data of the image and the B image can be read out. The read main image data and the image data of the A image and the B image are decompressed by the decompression circuit 122, subjected to the clipping process by the system controller 116, and then displayed on the monitor 118 through the video modulation circuit 117. The

  As a result, the split image composed of the display A image and the display B image that have been read out is superimposed (combined) on the main image data read out from the memory card 121 and reproduced and displayed on the monitor 118. . The card control circuit 120 and the system controller 116 mainly correspond to a reading unit and a reproducing unit.

  In FIG. 9, reference numeral 1300 denotes main image data recorded on the memory card 121. Reference numerals 1301, 1302, and 1303 are subjects appearing in the main image data 1300, and are located at different distances from the imaging device 1.

  Assume that when the main image data 1300 is recorded, the subject is located at a distance (hereinafter referred to as a focal point) where the imaging lens 101 is in focus with respect to the subject 1301. Further, it is assumed that the subject 1302 is located at a distance closer to the imaging device 1 than the focal point, and the subject 1303 is located at a distance farther from the imaging device 1 than the focal point.

  Reference numeral 1304 denotes image data of A image (herein referred to as A image data) recorded on the memory card 121 together with the main image data 1300. Reference numerals 1305, 1306, and 1307 are subjects shown in the A image data 1304, and correspond to the subjects 1301, 1302, and 1303 in the main image data 1300, respectively.

  Since the subject 1305 is located at the focal point, there is no deviation from the subject 1301 in the main image data 1300. However, since the subjects 1306 and 1307 are located at a distance closer to and far from the focal point, the subjects 1306 and 1307 are displaced in a direction approaching each other with respect to the subjects 1302 and 1303 in the main image data 1300.

  Reference numeral 1308 denotes B image data (herein referred to as B image data) recorded on the memory card 121 together with the main image data 1300. Reference numerals 1309, 1310, and 1311 are subjects shown in the B image data 1308, and correspond to the subjects 1301, 1302, and 1303 in the main image data 1300, respectively.

  Since the subject 1309 is located at the focal point, there is no deviation from the subject 1301 in the main image data 1300. However, since the subjects 1310 and 1311 are located at a distance closer to and farther from the focal point, the subjects 1310 and 1311 are shifted in directions away from the subjects 1302 and 1303 in the main image data 1300.

  Reference numerals 1330, 1322, and 1323 are superimposed on the main image data 1300 on the monitor 118 by superposing (combining) split images composed of the display A image and the display B image cut out from the A image data 1304 and the B image data 1308. A reproduced image that is reproduced and displayed is shown.

  In the reproduced image 1330, the split image (the display A image 1320 and the display B image 1321) is displayed so as to overlap a part of the subject 1301. Since the subject 1301 is in focus, the display A image 1320 and the display B image 1321 are not displaced.

  In the reproduced image 1322, the split image is displayed so as to overlap a part of the subject 1303. Since the subject 1303 located at a distance farther than the focal point is not in focus, the display A image and the display B image constituting the split image are shifted from each other. This shift amount matches the shift amount of the subjects 1307 and 1311 in the A image data 1304 and the B image data 1308.

  In the reproduced image 1323, a split image is displayed so as to overlap a part of the subject 1302. Since the subject 1302 located at a distance closer to the focal point is not in focus, the table A image and the display B image constituting the split image are shifted from each other. This shift amount matches the shift amount of the subjects 1306 and 1310 in the A image data 1304 and the B image data 1308. Further, the direction of the shift is opposite to the direction of the split image shift in the reproduced image 1322.

  In this way, by viewing the reproduction image in which the main image data and the split image are combined, the focus state of the recorded (imaged) main image data can be easily confirmed after recording.

  Here, switches provided in the imaging apparatus 1 illustrated in FIG. 8A will be described. These switches are provided on the back surface of the imaging apparatus 1 where the monitor 118 is provided.

  A zoom DOWN switch 1201 is operated to reduce the reproduced image, and a zoom UP switch 1202 is operated to enlarge the reproduced image. A monitor switch 1203 is operated to switch the monitor 118 ON / OFF.

  Reference numeral 1204 denotes a menu switch for displaying various operation menus, and reference numeral 1205 denotes a cross key (first operation means and second operation means) operated to instruct to move the split image or image.

  Reference numeral 1206 denotes a set switch for determining an item selected on the menu screen.

  FIG. 8A shows a full screen display state in which the entire reproduced image is displayed. FIG. 8B shows an enlarged display state in which a part of the reproduced image (main image data) is enlarged and displayed in accordance with the operation of the zoom UP switch 1202.

  FIG. 11 shows the flow of playback display processing (image processing method) in playback mode in the imaging apparatus 1 of the present embodiment. This process is also executed according to a computer program (image processing program) stored in the internal memory of the system controller 116.

  When the system controller 116 enters the playback mode, the image file that the operator desires to play is read from the memory card 121 via the card control circuit 120 in step S1. The read image file is transferred to the DRAM 114.

  In step S2, the system controller 116 reads the main image data stored in the image file as JPEG data from the DRAM 114 via the memory control circuit 120, and decompresses the JPEG data.

  In step S3, the system controller 116 initializes the display magnification. Here, in order to display the entire reproduced image on the monitor 118, the display magnification is set to 1.

  In step S4, the system controller 116 determines the state of a regeneration switch (not shown), and ends the regeneration mode if the regeneration switch is OFF. On the other hand, if the regeneration switch is ON, the process proceeds to the next step S5.

  In step S5, the system controller 116 determines the state of the zoom UP switch 1202, and proceeds to step S6 when the zoom UP switch 1202 is off. If the zoom UP switch 1202 is ON, the process proceeds to step S20 where the display magnification is changed.

  In step S6, the system controller 116 determines the state of the cross key 1205. If the cross key 1205 is OFF, the system controller 116 proceeds to step 8. If the cross key 1205 is ON, the process proceeds to step S7 where the display position information of the split image is changed. Note that the initial value of the display position information of the split image is set at the center of the screen.

  In step S7, the system controller 116 changes the position at which the split image is displayed (combined) on the main image data in accordance with the signal from the cross key 1205.

  Subsequently, in step S8, the system controller 116 causes the scaling circuit 115 to perform scaling processing for displaying the image data generated in step S2 on the monitor 118 on the full screen.

  In step S <b> 9, the system controller 116 reads out the A image data and B image data (denoted as pupil-divided images in the drawing) stored in the image file on the DRAM 114 via the memory control circuit 113. Then, the scaling circuit 115 performs scaling processing of the A image data and B image data at the same magnification as the magnification of the main image data subjected to the scaling processing in step S8.

  Next, in step S10, the system controller 116, based on the split image display position information, from the A image data and B image data after the scaling process, partial image data (display A image and (Display B image) is cut out. Then, the cut-out display A image and display B image are combined with the main image data after the scaling process in step S8, and the combined image is displayed on the monitor 118 as a reproduced image.

  By repeating the processing of steps S4 to S10, the split image can be freely moved on the reproduced image displayed on the monitor 118, and the focus state at an arbitrary position of the reproduced image (main image data) can be easily achieved. Can be confirmed.

  In step S20, the system controller 116 changes the display magnification in the enlargement direction each time the zoom UP switch 1202 is turned on.

  In step S21, the system controller 116 determines the state of the cross key 1205. If the cross key 1205 is OFF, the system controller 116 proceeds to step 23, and if it is ON, proceeds to step S22.

  In step S22, the system controller 116 changes the position information of the area to be enlarged in the main image data every time the cross key 1205 is turned on.

  In step S23, the system controller 116 causes the scaling circuit 115 to perform a scaling process for enlarging and displaying the area to be enlarged in the main image data generated in step S2 on the monitor 118.

  Next, in step S <b> 24, the system controller 116 reads out the A image data and B image data (pupil divided image) stored in the image file on the DRAM 114 via the memory control circuit 113. Then, the scaling circuit 115 performs scaling processing of the A image data and the B image data at the same magnification rate as that of the enlargement region of the main image data subjected to the magnification processing in step S23.

  Further, in step S25, the system controller 116 determines the partial image data (display A image) that fits within the split image display frame from the A image data and the B image data after the scaling process according to the position of the enlargement area of the main image data. And display B image). Then, the cut-out display A image and display B image are synthesized with the enlarged region image of the main image data after the scaling process in step S23, and the synthesized image is displayed on the monitor 118 as an enlarged reproduction image.

  By repeating steps S4 to S24, the enlarged area of the main image data can be freely moved (changed) in a state where the split image is displayed on the enlarged reproduced image, and any reproduction image (main image data) can be arbitrarily selected. The focus state at the position can be easily confirmed.

  In FIG. 12, 1400, 1401, and 1402 indicate enlarged reproduction images in which a partial area of the main image data 1300 is enlarged. The enlarged reproduction image 1400 is an image obtained by enlarging an area including the subject 1301 located at the focal point in the main image data 1300. The enlarged reproduction image 1401 is an image obtained by enlarging an area including the subject 1302 located at a distance closer to the focal point in the main image data 1300. The enlarged reproduction image 1402 is an image obtained by enlarging an area including the subject 1303 located at a distance farther from the focal point in the main image data 1300.

  In any enlarged reproduction image 1400 to 1402, the display position of the split image is fixed at the center of the screen, and the enlarged area is moved.

  FIG. 13 shows an image processing apparatus which is an embodiment of the present invention.

  The image processing apparatus 800 is constituted by a personal computer, and a computer program for performing basically the same processing as the reproduction display processing (image processing method) described in Embodiment 2 with reference to FIG. 11 is installed.

  The memory card 121 on which the main image data, the A image data, and the B image data are recorded in the image pickup apparatus 1 described in the first embodiment is removed from the image pickup apparatus 1 and is inserted into a memory slot of a personal computer. Accordingly, the personal computer reads the main image data, the A image data, and the B image data from the memory card 121, and generates a reproduced image.

  The reproduced image is displayed on the monitor 801 of the personal computer. The setting of the enlargement ratio of the reproduced image, the display position of the split image, the change of the enlargement area, and the like are performed through a mouse or a keyboard connected to the personal computer.

  Each embodiment described above is only a representative example, and various modifications and changes can be made to each embodiment in carrying out the present invention.

  For example, in the above-described embodiment, the case where the split image is combined with the main image data using the A image data and the B image data and reproduced and displayed has been described. However, the reproduction display method is not limited to this. For example, at least one of A image data and B image data may be superimposed on the main image data and reproduced and displayed as a double image or a triple image.

1 is a block diagram illustrating a configuration of an imaging apparatus according to a first embodiment and a second embodiment of the present invention. 3 is a diagram illustrating a pixel array of an image sensor used in the image pickup apparatus according to Embodiment 1. FIG. The enlarged view of the said pixel arrangement | sequence. The figure which shows the relationship between the focus state (focus state) and phase difference in AF pixel of the said image pick-up element. The figure which shows the relationship between the focus state (non-focus state) and phase difference in AF pixel of the said image pick-up element. The figure which shows the image signal read from AF pixel. 6 is a flowchart showing an imaging process in the imaging apparatus. FIG. 4 is a diagram showing a full screen display example in the imaging apparatus. The figure which shows the enlarged display example in the said imaging device. FIG. 6 is a diagram illustrating an example of main image data, A image data, B image data, and a reproduction image in Embodiment 2. FIG. 3 is a diagram illustrating a configuration of an image file stored in a recording medium by the imaging device. 10 is a flowchart showing playback display processing according to the second embodiment. FIG. 10 is a diagram illustrating an example of enlarged reproduction display in the second embodiment. FIG. 9 is a diagram illustrating an image processing apparatus that is Embodiment 3 of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 101 Imaging lens 104 Imaging element 109 Development pixel correction circuit 110 AF pixel selection circuit 111 Development processing circuit 112 Phase difference AF circuit 115 Scaling circuit 116 System controller 118 Monitor 121 Memory card 800 Image processing device 1300 Main image data 1320, 1321 Split image (Display A image, Display B image)
R, G, B imaging pixels (opening)
S1, S2 AF pixel (aperture)

Claims (10)

First image generation means for generating first image data based on a signal obtained by photoelectrically converting a first optical image formed by a light beam incident on an imaging optical system from a subject;
Based on a signal obtained by photoelectrically converting two second optical images having a phase difference corresponding to the focus state of the imaging optical system, formed by two divided light beams out of the light beams. Second image generating means for generating the image data of
Recording means for recording the first image data and the second image data on a recording medium for storage;
Reading means for reading the first and second image data from the recording medium;
Reproduction means for causing the display means to reproduce and display the second image data or the image data generated from the second image data together with the first image data;
Anda first operating means operated by an operator,
The reproduction means generates split image image data corresponding to the phase difference from the second image data, and causes the display means to reproduce and display the split image together with the first image data,
Said reproducing means, in response to said operation first operating means, said display means in said split image by changing the display position imaging device it said the. First image generation means for generating first image data based on a signal obtained by photoelectrically converting a first optical image formed by a light beam incident on an imaging optical system from a subject;
Based on a signal obtained by photoelectrically converting two second optical images having a phase difference corresponding to the focus state of the imaging optical system, formed by two divided light beams out of the light beams. Second image generating means for generating the image data of
Recording means for recording the first image data and the second image data on a recording medium for storage;
Reading means for reading the first and second image data from the recording medium;
Reproduction means for causing the display means to reproduce and display the second image data or the image data generated from the second image data together with the first image data;
And a second operating means operated by an operator,
It said reproducing means, said first enlarged display means to enlargement processing Ru is reproduced and displayed by the partial region of the image data are possible,
The reproduction means generates split image image data corresponding to the phase difference from the second image data, and causes the display means to reproduce and display the split image together with the first image data,
Said reproducing means, when said second operation means is operated, the display means in said split image display position wherein the to that imaging device fixed to changing the enlargement area according to the enlargement processing . It said reproducing means, image pickup device according to claim 1 or 2, characterized in that reproduced and displayed on the display unit by combining the first image data and the second image data. An image sensor having a first pixel group that photoelectrically converts the first optical image and a second pixel group that photoelectrically converts the second optical image;
The first image generation means generates the first image data using a signal from the first pixel group, and the second image generation means uses a signal from the second pixel group. The imaging apparatus according to any one of claims 1 to 3 , wherein the second image data is generated. First image generating means for generating first image data based on a signal obtained by photoelectrically converting a first optical image formed by a light beam incident on an imaging optical system from a subject, Based on a signal obtained by photoelectrically converting two second optical images formed by two split light beams and having a phase difference corresponding to the focus state of the imaging optical system, second image data is obtained. The first image data and the first image data and the second image generation means, and the first image data and the second image data are recorded by a recording unit that records the first image data and the second image data on a recording medium for storage. the recording medium of the second image data is recorded is an image processing apparatus to be mounted,
Reading means for reading out the first image data and the second image data from the recording medium;
Reproduction means for causing the display means to reproduce and display the second image data or the image data generated from the second image data together with the first image data ;
Have a, a first operating means operated by an operator,
The reproduction means generates split image image data corresponding to the phase difference from the second image data, and causes the display means to reproduce and display the split image together with the first image data,
The image processing apparatus , wherein the reproduction unit changes a display position of the split image on the display unit in accordance with an operation of the first operation unit . First image generating means for generating first image data based on a signal obtained by photoelectrically converting a first optical image formed by a light beam incident on an imaging optical system from a subject, Based on a signal obtained by photoelectrically converting two second optical images formed by two split light beams and having a phase difference corresponding to the focus state of the imaging optical system, second image data is obtained. The first image data and the first image data and the second image generation means, and the first image data and the second image data are recorded by a recording unit that records the first image data and the second image data on a recording medium for storage. An image processing apparatus to which the recording medium on which second image data is recorded is mounted,
Reading means for reading out the first image data and the second image data from the recording medium;
Reproduction means for causing the display means to reproduce and display the second image data or the image data generated from the second image data together with the first image data;
Second operating means operated by an operator,
The reproduction means can perform an enlargement process of enlarging a partial area of the first image data and reproducing and displaying it on the display means,
The reproduction means generates split image image data corresponding to the phase difference from the second image data, and causes the display means to reproduce and display the split image together with the first image data,
The image processing apparatus according to claim 1, wherein when the second operation unit is operated, the reproduction unit fixes a display position of the split image on the display unit and changes an enlargement area by the enlargement process. Generating first image data based on a signal obtained by photoelectrically converting a first optical image formed by a light beam incident on an imaging optical system from a subject;
Based on a signal obtained by photoelectrically converting two second optical images having a phase difference corresponding to the focus state of the imaging optical system, formed by two divided light beams out of the light beams. Generating image data of:
Recording the first image data and the second image data on a recording medium for storage ;
Reading the first and second image data from the recording medium;
A step of displaying means reproduces and displays the image data generated from the image data of the image data or the second of the second with the first image data, the possess,
In the reproducing and displaying step, image data of a split image corresponding to the phase difference is generated from the second image data, and the split image is reproduced and displayed on the display unit together with the first image data,
In the reproducing and displaying step, the display position of the split image on the display unit is changed in accordance with a first operation unit operated by an operator . Generating first image data based on a signal obtained by photoelectrically converting a first optical image formed by a light beam incident on an imaging optical system from a subject;
Based on a signal obtained by photoelectrically converting two second optical images having a phase difference corresponding to the focus state of the imaging optical system, formed by two divided light beams out of the light beams. Generating image data of:
Recording the first image data and the second image data on a recording medium for storage;
Reading the first and second image data from the recording medium;
Replaying and displaying on the display means the second image data or the image data generated from the second image data together with the first image data,
In the reproducing and displaying step, an enlargement process for enlarging a partial area of the first image data and reproducing and displaying it on the display means is possible.
In the reproducing and displaying step, image data of a split image corresponding to the phase difference is generated from the second image data, and the split image is reproduced and displayed on the display unit together with the first image data,
In the reproducing and displaying step, when the second operation means operated by the operator is operated, the display position of the split image on the display means is fixed and the enlargement area by the enlargement process is changed. A control method for an imaging apparatus. First image generating means for generating first image data based on a signal obtained by photoelectrically converting a first optical image formed by a light beam incident on an imaging optical system from a subject, Based on a signal obtained by photoelectrically converting two second optical images formed by two split light beams and having a phase difference corresponding to the focus state of the imaging optical system, second image data is obtained. The first image data and the first image data and the second image generation means, and the first image data and the second image data are recorded by a recording unit that records the first image data and the second image data on a recording medium for storage. a step of reading the image data and the image data of the second first from the recording medium to the second image data is recorded,
A step of displaying means reproduces and displays the image data generated from the image data of the image data or the second of the second with the first image data, the possess,
In the reproducing and displaying step, image data of a split image corresponding to the phase difference is generated from the second image data, and the split image is reproduced and displayed on the display unit together with the first image data,
In the reproducing and displaying step, the display position of the split image on the display means is changed in accordance with first operating means operated by an operator . First image generating means for generating first image data based on a signal obtained by photoelectrically converting a first optical image formed by a light beam incident on an imaging optical system from a subject, Based on a signal obtained by photoelectrically converting two second optical images formed by two split light beams and having a phase difference corresponding to the focus state of the imaging optical system, second image data is obtained. The first image data and the first image data and the second image generation means, and the first image data and the second image data are recorded by a recording unit that records the first image data and the second image data on a recording medium for storage. Reading the first image data and the second image data from the recording medium on which the second image data is recorded;
Replaying and displaying on the display means the second image data or the image data generated from the second image data together with the first image data,
In the reproducing and displaying step, an enlargement process for enlarging a partial area of the first image data and reproducing and displaying it on the display means is possible.
In the reproducing and displaying step, image data of a split image corresponding to the phase difference is generated from the second image data, and the split image is reproduced and displayed on the display unit together with the first image data,
In the reproducing and displaying step, when the second operation means operated by the operator is operated, the display position of the split image on the display means is fixed and the enlargement area by the enlargement process is changed. A featured image processing method.