JP6207360B2 - Imaging apparatus and image signal processing method - Google Patents

Description

  The present invention relates to an imaging apparatus including an imaging sensor capable of simultaneously outputting image signals from two or more paths, and an image signal processing method executed by the imaging apparatus.

  Some imaging devices using an imaging element such as a CCD sensor or a CMOS sensor have a live view function. The live view function is to sequentially output an image based on an image signal continuously read from the image sensor to a display device such as a liquid crystal display provided on the back surface of the image pickup device, thereby This is a function that can be confirmed.

  When a battery is used as the power source of the image pickup apparatus, it is desirable to suppress power consumption for live view display in a shooting standby state, but there are also situations where high-definition live view display is desired. Therefore, a technique for switching the live view display method between a live view display in a non-operation state and a shooting preparation state (for example, a state in which an AF operation or the like is performed on a subject when the shutter button is half-pressed) is known.

  A technique for high-definition live view display is proposed in Patent Document 1, for example. In Patent Document 1, high-sensitivity pixel information and low-sensitivity pixel information are acquired by controlling the exposure time for pixels to be different for each pixel area such as a row unit, and an output pixel value is based on the pixel information of these different sensitivities. Thus, it is possible to generate a wide dynamic range image.

JP 2012-105225 A

  However, in order to switch the live view display between the low image quality display in the low power consumption mode and the high definition display for preparation for shooting, it is necessary to switch the shooting driving mode for live view. Since a time lag occurs when switching the shooting driving mode for live view, a live view image during that time cannot be obtained, and the live view screen is frozen (the continuous display time of the image is extended).

  SUMMARY OF THE INVENTION An object of the present invention is to provide a technique that makes it possible to switch a shooting drive mode for live view without causing a freeze of live view display.

  An imaging device according to the present invention includes an imaging device having a plurality of pixels arranged in a row direction and a column direction, a first imaging mode for reading an image signal from pixels in a predetermined row of the imaging device, and the imaging device A second imaging mode in which the number of readout rows of the image signal from the first imaging mode is different from the number of readout rows of the image signal in the first imaging mode, and a readout row of the image signal from the imaging element is the first imaging mode. Driving means for driving the imaging device in one or a plurality of imaging modes selected from the third imaging mode that does not overlap with the readout row of the image signal in the second imaging mode, and the first imaging mode A first output path for separately outputting image signals read from the image sensor when at least two of the second imaging mode and the third imaging mode are executed. Output means having at least two output paths including a second output path and an image signal in the first imaging mode output from the first output path for image display on a display means, From the state where the image signal output from the second output path in the third imaging mode is used for an application different from the image display, the image signal used for the image display is used in the first imaging mode. When switching from an image signal to an image signal in the second imaging mode, the acquisition source of the image signal used for the image display is temporarily switched from the first output path to the second output path, The first imaging mode is switched to the second imaging mode so that the image signal in the second imaging mode is output from one output path, and the acquisition destination of the image signal used for the image display is changed. And a controlling means for switching from the serial second output path to the first output path.

  According to the present invention, an image signal is acquired from the image sensor through the first output path and the second output path, the image signal of the first output path is used for live view display (image display), and the second output The image signal of the route is used for purposes other than live view display. At this time, first, the drive mode of the image sensor is switched so that the image signal output from the second output path can be used for live view display, and the acquisition source of the image signal for live view display is the first output. Switch from the path to the second output path. Next, the drive mode of the image sensor is switched so that live view display image signals of different definition are output from the first output path, and the acquisition source of the live view display image signals is set to the second output path. To the first output path.

  As a result, it is possible to switch the shooting drive mode for live view display without causing the freeze of live view display, reducing the stress when using the user's live view display, and improving convenience. it can.

It is a perspective view which shows the external appearance of the digital camera which concerns on embodiment of this invention seeing from the back side. It is a block diagram which shows the hardware constitutions of the digital camera of FIG. FIG. 2 is a schematic perspective view (a) showing a structure of an image pickup element included in the image pickup unit of the digital camera of FIG. 1 and a block diagram (b) showing the configuration thereof. FIG. 4 is a diagram schematically illustrating a signal reading line from the image sensor of FIG. 3 during live view display in the digital camera of FIG. 1. 2 is a timing chart showing the relationship between an output signal of Ch1, an output signal of Ch2, and an independent AF operation start signal in the imaging unit during the independent AF operation of the digital camera of FIG. It is a timing chart when switching the normal live view imaging drive mode to a higher definition high-definition live view imaging drive mode that is assumed when the conventional technology is used. It is a flowchart of the switching process of the live view mode when operation (half press) of the 1st switch of a shutter button assumed when the prior art is used. It is a timing chart which shows the change etc. of the imaging signal accompanying switching of the imaging drive mode according to the flowchart of FIG. 6 is a flowchart of image signal processing executed when the first switch is operated when the independent AF operation is being performed but the AF evaluation mode is not in the imaging device in the digital camera of FIG. 1. FIG. 10 is a timing chart showing a change in an imaging signal associated with switching of an imaging drive mode according to the flowchart of FIG. 9. FIG. 6 is a flowchart of image signal processing executed when the first switch is operated in the state where the independent AF operation is not being performed in the digital camera of FIG. 1. FIG. 12 is a timing chart showing a change in an imaging signal associated with switching of a drive mode according to the flowchart of FIG. FIG. 10 is a timing chart of image signal processing in a case where the output signal from Ch2 is used for live view as it is after the acquisition source of the live view imaging signal is switched from Ch1 to Ch2 in step S904 of the flowchart of FIG. FIG. 12 is a timing chart of image signal processing when an output signal from Ch2 is used for live view as it is after the acquisition source of the live view imaging signal is switched from Ch1 to Ch2 in step S1103 of the flowchart of FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Here, although a compact digital camera (hereinafter referred to as “digital camera”) is taken up as an imaging apparatus according to the present invention, the present invention is not limited to this.

<First Embodiment>
FIG. 1 is a perspective view showing the appearance of the digital camera 100 according to the first embodiment of the present invention as seen from the back side.

  On the back of the digital camera 100, a display unit 101 for displaying images and various information is provided. The display unit 101 is, for example, a liquid crystal display or an organic EL display, and may function as an operation unit by providing an input function as a touch panel. Further, on the back of the digital camera 100, an operation unit 102 including operation members such as various switches and buttons for receiving various operations by a user, a mode switching switch 104 for switching a photographing mode for a subject, and a controller wheel that can be rotated. 103 is provided. A part of the operation unit 102 is also provided on the upper surface of the digital camera 100. Details of functions of the operation unit 102, the controller wheel 103, and the mode changeover switch 104 will be described later with reference to FIG.

  On the top surface of the digital camera 100, a shutter button 121 for instructing photographing and a power switch 122 for switching the power on / off of the digital camera 100 are provided. Details of the function of the shutter button 121 will be described later with reference to FIG.

  An external device can be connected to the side surface of the digital camera 100 via a connection cable 111 and a connector 112. The digital camera 100 can output image data (still image data, moving image data) to an external device via the connection cable 111 and the connector 112.

  A storage medium slot (not shown) that can be opened and closed by a lid 131 is provided on the lower surface of the digital camera 100, and a storage medium 130 such as a memory card can be inserted into and removed from the storage medium slot. Yes. The storage medium 130 stored in the storage medium slot can communicate with the control unit of the digital camera 100 (system control unit 210 described with reference to FIG. 2). The storage medium 130 is not limited to a memory card that can be inserted into and removed from the storage medium slot, but may be an optical disk such as a DVD-RW disk or a magnetic disk such as a hard disk. It may be built in the main body.

  FIG. 2 is a block diagram illustrating a hardware configuration of the digital camera 100. The digital camera 100 includes a barrier 201, a photographing lens 202 and a shutter 203 that constitute an imaging optical system, an imaging unit 204, an AF evaluation value calculation unit 205, and a strobe 217.

  The barrier 201 covers the imaging optical system to prevent the imaging optical system from being soiled or damaged. The photographing lens 202 is constituted by a lens group including a zoom lens and a focus lens. The shutter 203 has a diaphragm function and adjusts the exposure amount. The imaging unit 204 includes an imaging element that converts an optical image into an electrical signal (analog signal). Specifically, the image sensor is a CCD sensor or a CMOS sensor. The imaging unit 204 has an A / D conversion processing function, and converts an analog electric signal output from the imaging element into a digital signal (digital image data).

  The AF evaluation value detection unit 205 calculates an AF evaluation value from contrast information obtained from the digital signal generated by the imaging unit 204, and outputs the obtained AF evaluation value to the system control unit 210, which will be described later, through the imaging unit 204. To do. By making the strobe light 217 emit light at the time of shooting, the illuminance can be supplemented when shooting in a low-light scene or shooting in a backlight scene.

  The digital camera 100 includes an image processing unit 206, a memory control unit 207, a D / A converter 208, a memory 209, a system control unit 210, a nonvolatile memory 211, a system timer 212, a system memory 213, and a display unit 101.

  The image processing unit 206 and the memory control unit 207 receive the digital signal generated by the A / D conversion process in the imaging unit 204. The image processing unit 206 performs signal processing such as predetermined pixel interpolation and resizing processing, color conversion processing, and the like on data received from the imaging unit 204 (digital image data) or data received from the memory control unit 207. . The image processing unit 206 performs predetermined arithmetic processing using image data of the captured image, and the system control unit 210 performs exposure control and focus control using the arithmetic result generated by the image processing unit 206. . For example, the system control unit 210 performs TTL (through the lens) AF (autofocus) processing, AE (automatic exposure) processing, light control processing, AWB (auto white balance) processing, and the like. The image processing unit 206 performs AF processing. At this time, the output of the AF evaluation value detection unit 205 included in the imaging unit 204 may be used.

  A digital signal (digital image data) output from the imaging unit 204 is written into the memory 209 via the image processing unit 206 and the memory control unit 207 or via the memory control unit 207. In addition to this, the memory 209 stores image data acquired by the imaging unit 204 and subjected to A / D conversion, and image data to be displayed on the display unit 101. The memory 209 has a sufficient storage capacity capable of storing a predetermined number of still images, a moving image for a predetermined time, and audio data. The memory 209 also serves as an image display memory (video memory).

  Digital data for image display stored in the memory 209 is transmitted to the D / A converter 208. The D / A converter 208 converts the received digital data into an analog signal and supplies it to the display unit 101, whereby an image is displayed on the display unit 101. As described above, the display unit 101 is a display such as a liquid crystal display, and displays an image based on an analog signal from the D / A converter 208. Note that a digital signal converted from an analog signal by the imaging unit 204 and stored in the memory 209 is converted into an analog signal by the D / A converter 208 and sequentially transferred to the display unit 101 for display, thereby displaying an electronic view. A finder function can be realized. That is, the through image display can be performed in this way.

  The non-volatile memory 211 is a memory that can be electrically erased and stored, and is, for example, an EEPROM typified by a flash memory or the like. The nonvolatile memory 211 stores a program executed by the system control unit 210, operation constants, and the like. The program here refers to a program for executing each flowchart described later.

  The system control unit 210 controls the overall operation of the digital camera 100 by executing various programs stored in the nonvolatile memory 211, and executes various processes described below as an example. Further, the system control unit 210 performs display control by controlling the memory 209, the D / A converter 208, the display unit 101, and the like. Programs, operation constants, variables, and the like read from the nonvolatile memory 211 by the system control unit 210 are expanded on the system memory 213. A RAM is used as the system memory 213. The system timer 212 measures the time used for various controls and the time of a built-in clock.

  The operation unit 102, controller wheel 103, shutter button 121, mode switch 104, and power switch 122 shown in FIG. 2 are the same as those described with reference to FIG.

  Various operation members constituting the operation unit 102 are used for selecting various function icons displayed on the display unit 101, and functions are appropriately assigned to each scene by selecting a predetermined function icon. It is done. That is, each operation member of the operation unit 102 is used as various function buttons.

  A controller wheel 103, which is an operation member capable of rotating, is used when a selection item is instructed together with a four-way button. When the controller wheel 103 is rotated, an electrical pulse signal corresponding to the operation amount (rotation angle, number of rotations, etc.) is generated. The system control unit 210 analyzes the pulse signal and controls each unit of the digital camera 100.

  Note that the controller wheel 103 may be any operation member that can detect a rotation operation, such as a member that rotates itself, or a member that does not rotate but detects a rotation operation with a touch sensor.

  The shutter button 121 includes a first switch SW1 and a second switch SW2. The first switch SW <b> 1 is turned on when the shutter button 121 is half-pressed while the shutter button 121 is being operated. When receiving a signal that the first switch SW1 is turned on, the system control unit 210 starts operations such as AF processing, AE processing, AWB processing, and light control processing. The second switch SW2 is turned on when the shutter button 121 is fully pressed, and a signal instructing to start photographing is thereby transmitted to the system control unit 210. When receiving a signal with the second switch SW2 turned ON, the system control unit 210 performs a series of shooting operations from reading a signal from the imaging unit 204 to writing image data into the storage medium 130.

  The mode switch 104 is a switch for switching the operation mode of the digital camera 100 between various modes such as a still image storage mode, a moving image storage mode, and a reproduction mode. The still image storage mode includes, for example, an auto shooting mode, an auto scene determination mode, a manual mode, various scene modes serving as shooting settings for each shooting scene, a program AE mode, a custom mode, and the like. By operating the mode switching switch 104, it is possible to directly switch to one of these modes included in the still image shooting mode. However, the present invention is not limited to such a configuration. For example, after switching to the still image shooting mode with the mode switch 104, the operation mode is switched to one of the above modes included in the still image shooting mode using another operation member. You may do it. Similarly, the moving image shooting mode may include a plurality of modes.

  The digital camera 100 includes a power supply unit 214 and a power supply control unit 215. The power supply unit 214 is a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, or an AC adapter, and supplies power to the power supply control unit 215. The power supply control unit 215 is configured by a battery detection circuit, a DC-DC converter, a switch circuit that switches an energization block, and the like. The power supply control unit 215 detects the presence / absence of the battery in the power supply unit 214, the type of battery, the remaining battery level, etc., and controls the DC-DC converter based on the detection result and the instruction of the system control unit 210, and is necessary A sufficient voltage is supplied to each part including the storage medium 130 for a necessary period.

  The digital camera 100 includes a storage medium I / F 216 that enables communication between the storage medium 130 and the system control unit 210 when the storage medium 130 is mounted in a storage medium slot (not shown). The details of the storage medium 130 have already been described with reference to FIG.

  FIG. 3 is a schematic perspective view (a) showing the structure of the image sensor provided in the imaging unit 204 and a block diagram (b) showing the configuration thereof.

  The imaging device includes a first chip 30 in which a plurality of pixels 301 are formed and arranged on the light incident side, pixel driving circuits such as column scanning circuits 313a and 313b and a row scanning circuit 312 and an AF evaluation value detection unit 205 ( These are referred to as “peripheral circuits”) and the second chip 31 formed thereon. The imaging element is configured by laminating a first chip 30 on a second chip 31. By forming the pixel 301 on the first chip 30 and forming the peripheral circuit on the second chip 31, the manufacturing process of the peripheral circuit and the pixel 301 can be separated. High speed, small size, and high functionality can be realized by increasing the density and increasing the density.

  The first chip 30 has a plurality of pixels 301 arranged in a matrix at regular intervals in the row direction and the column direction. The first chip 30 is connected to each pixel 301 in the row direction (horizontal direction), the transfer signal line 303, the reset signal line 304 and the row selection signal line 305, and the column direction of each pixel 301. Column signal lines 302a and 302b. The connection destinations of the column signal lines 302a and 302b are distinguished by the read row unit. The pixel connection method will be described later with reference to FIG.

  The second chip 31 includes a column ADC block 311 to which the column signal lines 302a and 302b are connected, a row scanning circuit 312 that scans each row, and column scanning circuits 313a and 313b that scan each column. The second chip 31 includes a timing control circuit 314, horizontal signal lines 315a and 315b, a frame memory 317, an AF evaluation value detection unit 205, and a switch 316.

  The timing control circuit 314 receives the control signal from the system control unit 210 and controls the respective timings of the row scanning circuit 312, the column scanning circuits 313 a and 313 b, and the column ADC block 311. The horizontal signal lines 315a and 315b transfer digital signals from the column ADC block 311 according to the timing controlled by the column scanning circuits 313a and 313b. The frame memory 317 temporarily stores a later-described AF evaluation value detection imaging signal that is an image signal output from the horizontal signal line 315b. The AF evaluation value detection unit 205 (see FIG. 2 as appropriate) detects the AF evaluation value from the AF evaluation value detection imaging signal output from the frame memory 317. The switch 316 switches whether to output the AF evaluation value detection imaging signal output to the horizontal signal line 315b to the AF evaluation value detection unit 205 or to the image processing unit 206.

  In the imaging device, the pixel 301 includes a photodiode PD, a floating diffusion FD, a transfer transistor M1, a reset transistor M2, an amplification transistor M3, and a selection transistor M4. Here, each transistor is assumed to be an n-channel MOSFET.

  A transfer signal line 303, a reset signal line 304, and a row selection signal line 305 are connected to the gates of the transfer transistor M1, the reset transistor M2, and the selection transistor M4. These signal lines extend in the row direction and simultaneously drive the pixels 301 included in the same row, thereby enabling a line-sequential operation type rolling shutter and an all-row simultaneous operation type global shutter. It is possible to control the operation. The column signal line 302a or the column signal line 302b is connected to the source of the selection transistor M4 in units of rows.

  The photodiode PD accumulates electric charges generated by photoelectric conversion, the P side is grounded, and the N side is connected to the source of the transfer transistor M1. When the transfer transistor M1 is turned on, the charge of the photodiode PD is transferred to the floating diffusion FD. However, since the floating diffusion FD has a parasitic capacitance, the charge is accumulated in this portion.

  The drain of the amplification transistor M3 is set to the power supply voltage Vdd, and the gate of the amplification transistor M3 is connected to the floating diffusion FD. The amplification transistor M3 converts the voltage of the floating diffusion FD into an electric signal. The selection transistor M4 is for selecting a pixel from which a signal is read out in units of rows. The drain of the selection transistor M4 is connected to the source of the amplification transistor M3, and the source of the selection transistor M4 is connected to the column signal line 302a or the column signal line 302b. When the selection transistor M4 is turned on, a voltage corresponding to the voltage of the floating diffusion FD is output to the column signal line 302a or the column signal line 302b. The drain of the reset transistor M2 is the power supply voltage Vdd, and the source of the reset transistor M2 is connected to the floating diffusion FD. The reset transistor M2 resets the voltage of the floating diffusion FD to the power supply voltage Vdd.

  Next, pixel selection of the column signal lines 302a and 302b will be described with reference to FIG. FIG. 4 is a diagram for schematically explaining a signal reading line from the image sensor at the time of live view display in the digital camera 100. On the left side of FIG. 4, the arrangement of the pixels 301 of the image sensor is indicated by each color (R, G (Gb, Gr), B) of a color filter having a Bayer arrangement arranged corresponding to the pixel arrangement. The right side of FIG. 4 shows an example of a selected row in each readout mode described below.

  In a state where live view display is performed, the live view imaging signal is output to the column signal line 302a. The live view imaging signal output to the column signal line 302 a is converted from an analog signal to a digital signal in the column ADC block 311. The live view imaging signal converted into a digital signal in the column ADC block 311 is read from the column ADC block 311 to the horizontal signal line 315a by the operation of the column scanning circuit 313a. The digitalized live view imaging signal read out to the horizontal signal line 315a is output from the imaging unit 204 to the image processing unit 206.

  Here, when the system control unit 210 detects that the AF evaluation value in the image processing unit 206 based on the movement of the main subject and the live view imaging signal is small, the system control unit 210 outputs an independent AF operation start signal to the imaging unit 204. To do. In the imaging unit 204, when the independent AF operation start signal reaches the second chip 31, the readout line of the pixel 301 is divided so that live view imaging and AF evaluation value detection imaging can be performed simultaneously, and AF An imaging signal for evaluation value detection is output to the column signal line 302b.

  In the present embodiment, row numbers 1 and 2 shown in FIG. 4 are AF imaging rows, and signal readout from row numbers 1 and 2 is performed in the AF imaging drive mode (third imaging mode). Row numbers 3 and 4 shown in FIG. 4 are imaging rows for performing a normal live view. In the normal live view imaging drive mode (first imaging mode), signal reading from the row numbers 3 and 4 is performed. In the present embodiment, it is assumed that readout scanning from row numbers 1 and 2 and row numbers 3 and 4 is sequentially performed in units of rows, and readout scanning is repeatedly performed in units of 8 rows. In addition, both the AF imaging and the normal live view imaging are driven by thinning-out readout of 3 out of 4 pixels of the same vertical color. Therefore, since the output image signal in the AF evaluation value detection imaging has the same image quality as the output image signal in the normal live view imaging, it can be used for live view display. Similarly, the output image signal of normal live view imaging can be used for AF evaluation value detection.

  On the other hand, by operating the operation unit 102 or the like, it is possible to perform imaging with a large number of readout lines in order to emphasize image quality as live view imaging. For example, in the imaging drive mode for the high-definition live view A1 (second imaging mode) that captures the high-definition live view A1 with higher definition than the normal live view, the row numbers 3, 4, 7, and 8 are imaged rows. And It is also possible to image the high-definition live view A2 having the same definition as the high-definition live view A1 so that the image lines include the row numbers 1 and 2 that are AF imaging rows. In the imaging drive mode (fourth imaging mode) for high-definition live view A2 that performs imaging of high-definition live view A2, row numbers 1, 2, 5, and 6 are taken as imaging rows. Note that the readout line for reading out the image signal in the AF imaging drive mode and the high-definition live view A2 imaging drive mode is the readout line for reading out the image signal in the normal live view imaging drive mode and the high-definition live view A1 imaging drive mode. Is set so as not to overlap.

  High-definition live view B with higher definition can also be captured. In this case, in the high-definition live view B imaging drive mode (fifth imaging mode), 1 pixel thinning out of 4 pixels of the same color is added. It is also possible to drive. In this case, row numbers 3, 4, 5, 6, 7, and 8 are live view imaging rows. In this way, by separating AF imaging and live view imaging for each selected row, it is possible to acquire image data of different frame rates with different data sizes in different charge accumulation times.

  The AF evaluation value detection imaging signal output to the column signal line 302 b is converted from an analog signal to a digital signal in the column ADC block 311. The imaging signal for AF evaluation value detection converted into a digital signal in the column ADC block 311 is read from the column ADC block 311 to the horizontal signal line 315b by the operation of the column scanning circuit 313b. The AF evaluation value detection imaging signal converted into a digital signal read to the horizontal signal line 315b is output to the switch 316.

  When the in-image sensor AF evaluation mode is set by the control signal from the system control unit 210, the AF evaluation value detection imaging signal stored in the frame memory 317 from the horizontal signal line 315b via the switch 316 is detected by the AF evaluation value. Sent to the unit 205. The AF evaluation value detection unit 205 calculates an AF evaluation value based on the contrast information of the received AF evaluation value detection imaging signal, and outputs only the AF evaluation value signal from the imaging element of the imaging unit 204 to the system control unit 210. . On the other hand, when the image sensor AF evaluation mode is not set by the control signal from the system control unit 210, the AF evaluation value detection imaging signal is output from the horizontal signal line 315b to the image processing unit 206 via the switch 316. The

  In the following description, a path (first output path) for outputting an imaging signal from the column signal line 302a to the horizontal signal line 315a is referred to as channel 1 (hereinafter referred to as “Ch1”). A path (second output path) for outputting an imaging signal from the column signal line 302b to the horizontal signal line 315b is referred to as channel 2 (hereinafter referred to as “Ch2”). In this embodiment, it is possible to execute two modes with different selected rows used for reading image signals to the image sensor, and to output the image signals read in each mode separately from Ch1 and Ch2.

  FIG. 5 is a timing chart showing the relationship between the output signal of Ch1, the output signal of Ch2, and the independent AF operation start signal in the imaging unit 204 during the independent AF operation. The vertical synchronization signal is output at a timing at which a high-speed frame rate output that can be output from Ch1 and Ch2 can be realized. Here, an image signal for live view is output from Ch1, and the image signal for live view is driven and output at a frame rate of 30 FPS, while a vertical synchronization signal is input at 240 Hz. In this case, the output of the live view imaging signal from Ch1 is controlled to be output once every four times, ignoring the vertical synchronization signal three times. Until the timing T1 when the live view imaging signal is output from Ch1, the system control unit 210 performs the operation based on the result calculated by the image processing unit 206 based on the live view imaging signal input to the image processing unit 206. Auto focus control (AF control) is performed.

  At timing T1, the system control unit 210 determines that an independent AF operation is necessary, and an independent AF operation start signal is input. When the independent AF operation start signal is inputted, output control of the imaging evaluation signal for AF evaluation value detection of Ch2 is started in synchronization with the next inputted vertical synchronizing signal. After the pixel 301 is exposed, the first AF evaluation value detection imaging signal is output at timing T2. At this time, when the imaging device AF evaluation mode is set, the AF evaluation value detection unit 205 in the imaging unit 204 detects the AF evaluation value based on the AF evaluation value detection imaging signal, and detects the detected AF evaluation. The value is output to the system control unit 210. On the other hand, when the imaging element AF evaluation mode is not set, the AF evaluation value detection imaging signal is input to the image processing unit 206, and the image processing unit 206 calculates the result based on the AF evaluation value detection imaging signal. Based on this, AF control is performed by the system control unit 210.

  FIG. 5 shows that the output of the imaging signal for AF evaluation value detection from Ch2 is variable in frame rate. That is, at timing T3, the system control unit 210 inputs a control signal for reducing the frame rate of the output of the AF evaluation value detection imaging signal from Ch2. Thus, after the next vertical synchronizing signal is input, the vertical synchronizing signal is ignored once and is controlled to be output once every two times. Since the longest exposure time can be changed by changing the frame rate, the system control unit 210 performs control to ensure AF accuracy by changing the frame rate according to the shooting scene. The frame rates of the Ch1 output and the Ch2 output can be controlled independently, and the output of the AF evaluation value detection imaging signal from Ch2 is stopped in response to the input end of the independent AF operation start signal at timing T4.

  Here, with reference to FIG. 6 to FIG. 8, problems of the prior art assumed at the time of shooting using live view display will be described. In the description of FIG. 6 to FIG. 8, processing and operation with components equivalent to those of the digital camera 100 will be described with appropriate reference numerals.

  FIG. 6 is a timing chart for switching the normal live view imaging drive mode to a higher definition high-definition live view imaging drive mode, which is assumed when a general prior art is used. The pixel 301 of the image sensor is exposed in synchronization with the vertical synchronizing signal (imaging surface exposure), and an image signal of one frame is output from the imaging unit 204. The imaging drive and signal output for each row of the plurality of pixels 301 provided in the imaging device are controlled in synchronization with a horizontal synchronization signal (not shown in FIG. 6). The row scanning circuit 312 sequentially performs reset processing by inputting a signal to the reset signal line 304 from the upper row to the lower row of the imaging surface. After the exposure time has elapsed, the column ADC block 311 sequentially performs A / D conversion. As a result, an imaging signal is output. Such imaging surface exposure is represented by a parallelogram in FIG. 6 taking the horizontal axis as time.

  The image pickup signal read out by the image processing unit 206 is stored in the video memory area of the memory 209 through signal processing by the memory control unit 207 through image processing (development / image processing, etc.) in the image processing unit 206. The D / A converter 208 converts the image display data stored in the memory 209 into an analog signal and supplies it to the display unit 101, thereby performing live view display.

  When the normal live view imaging drive mode is switched to the high-definition live view imaging drive mode, a frame that cannot be used is generated in association with the switching. Therefore, as shown in FIG. Temporary freeze occurs. In the high-definition live view imaging drive mode, since the number of readout rows is larger than that in the normal live view imaging drive mode, the time required for line scanning from the upper row to the lower row of the imaging surface becomes longer. In FIG. 6, this is represented by the shape of a parallelogram with a large inclination angle.

  As an example of the case where switching from the normal live view display to the high-definition live view display described above occurs, there is an operation of the first switch SW1 of the shutter button 121 (half-pressing of the shutter button 121). FIG. 7 is a flowchart of live view mode switching processing when the first switch SW1 of the shutter button 121 is operated, which is assumed when a general conventional technique is used.

  When the first switch SW1 is operated in the normal live view display state, in step S701, the system control unit 210 performs AE processing. Note that the AE process may be ended before the operation of the first switch SW1. In subsequent step S702, the system control unit 210 switches the imaging drive mode from the normal live view imaging drive mode to the AF imaging drive mode. Note that the AF imaging drive mode may be the same as the normal live view imaging drive mode, and in this case, switching of the drive mode does not occur. Since the AF imaging drive mode is driven at a higher frame rate than the normal live view imaging drive mode, the drive is switched to the AF imaging drive mode in step S702, thereby speeding up AF control. Shall be allowed to.

  Next, in step S703, the system control unit 210 performs AF processing. Subsequently, in step S704, the system control unit 210 switches the imaging drive mode from the AF imaging drive mode to the high-definition live view imaging drive mode. Thereby, in the operation standby state of the second switch SW2 of the shutter button 121, high-definition live view display is realized.

  FIG. 8 is a timing chart showing the change of the imaging signal and the like accompanying the switching of the imaging drive mode according to the flowchart of FIG. It can be seen that two freezes occur between the time when the operation of the first switch SW1 of the shutter button 121 is performed and the time when the operation of the second switch SW2 is entered.

  As described with reference to FIGS. 6 to 8, in the related art, there is a problem that the display screen is frozen as the drive mode is switched. On the other hand, in the present embodiment, as described below with reference to FIGS. 9 to 12, the drive mode can be switched without causing the display screen to freeze.

  FIG. 9 is a flowchart of image signal processing executed when the first switch SW1 is operated when the digital camera 100 is in the independent AF operation but is not in the imaging element AF evaluation mode.

  In step S901, the system control unit 210 performs AE processing using a normal live view imaging signal (denoted as “Ch1 imaging signal” in FIG. 9) output from Ch1 and used for normal live view display. . Note that the AE process may be ended before the operation of the first switch SW1. In subsequent step S902, the system control unit 210 performs an AF process using an AF evaluation value detection imaging signal output from Ch2 (denoted as “Ch2 imaging signal” in FIG. 9).

  Next, in step S903, the system control unit 210 changes from the AF imaging drive mode to the live view imaging drive mode so that the live view imaging signal is output from Ch2 instead of the AF evaluation value detection imaging signal. Switch the drive mode. Note that the live view imaging drive mode switched in step S903 may be the normal live view imaging drive mode or the high definition live view imaging drive mode (high definition live view A2). Here, it is assumed that the mode is switched to the normal live view imaging drive mode.

  In the present embodiment, as shown in FIG. 4, since the number of readout rows is the same (the readout rows are different) between the AF imaging and the normal live view imaging, drive switching is not necessary. However, in this embodiment, the system control unit 210 controls the frame rate of Ch2 after switching to be the same as that of Ch1, and the exposure (sensitivity, exposure time) of the imaging signal of Ch2 is the same as that of Ch1. To control.

  Next, in step S904, the system control unit 210 switches the acquisition source of the live view imaging signal from Ch1 to Ch2. Subsequently, in step S905, the system control unit 210 switches the drive mode so that the output signal from Ch1 is switched from the normal live view imaging signal to the high definition live view imaging signal (high definition live view A1). Further, in step S906, the system control unit 210 switches the acquisition source of the live view imaging signal from Ch2 to Ch1. As a result, high-definition live view display is realized in the operation standby state of the second switch SW2.

  After that, if the independent AF operation is necessary continuously, the system control unit 210 outputs the imaging signal for Ch2, and the imaging drive mode for AF (or AF) so that the AF evaluation value detection imaging signal is output again. To frame rate). On the other hand, the system control unit 210 stops the output of the imaging signal if the AF scanning is unnecessary.

  FIG. 10 is a timing chart showing the change of the imaging signal accompanying the switching of the imaging drive mode according to the flowchart of FIG. By switching the acquisition source of the live view imaging signal, it is understood that the display screen is switched to the high-definition live view display in the operation standby state of the second switch SW2 without causing the freeze. The time from the start of the output of the image signal in the high-definition live view imaging drive mode to the time when the image signal is processed for live view display and output to the display unit 101 is the normal live view imaging drive. The time for performing the same processing in the mode is the same. Further, in this embodiment, live view display is performed after image processing is completed during normal live view display so that signal processing (development, image processing, etc.) during high definition live view display is completed in time for the display cycle. A margin is given to the time until it is displayed.

  FIG. 11 is a flowchart of image signal processing executed when the first switch SW1 is operated in the digital camera 100 in a state where the independent AF operation is not being performed.

  In step S1101, the system control unit 210 performs AE processing using the normal live view imaging signal output from Ch1 and used for normal live view display. Note that the AE process may be ended before the operation of the first switch SW1. In subsequent step S1102, the system control unit 210 sets the imaging drive mode to the live view imaging drive mode so that the output of the live view imaging signal is started from Ch2.

  Note that the live view imaging drive mode at this time may be the normal live view imaging drive mode or the high definition live view imaging drive mode (high definition live view A2). Here, it is assumed that the normal live view imaging drive mode is set, and in this embodiment, since the number of read rows is the same between the AF imaging and the normal live view imaging, drive switching is not necessary. However, in the present embodiment, the system control unit 210 controls the frame rate of Ch2 after switching to be the same as that of Ch1, and controls the sensitivity and exposure time of the imaging signal of Ch2 to be the same as that of Ch1. doing.

  In step S1103, the system control unit 210 switches the acquisition source of the live view imaging signal from Ch1 to Ch2. In subsequent step S1104, the system control unit 210 changes the drive mode from the normal live view drive mode to the high definition live view so that the high-definition live view imaging signal (first high-definition live view A1) is output from Ch1. Switch to view imaging drive mode. Further, in step S1105, the system control unit 210 switches the acquisition source of the live view imaging signal from Ch2 to Ch1. As a result, high-definition live view display is realized in the operation standby state of the second switch SW2. Thereafter, in step S1106, the system control unit 210 stops the output of the imaging signal from Ch2.

  FIG. 12 is a timing chart showing a change in the imaging signal and the like associated with switching of the drive mode according to the flowchart of FIG. Again, the imaging signals output from each of Ch1 and Ch2 are switched, and the acquisition source of the live view imaging signal is switched. Thereby, high-definition live view display can be performed in the operation standby state of the second switch SW2 without causing a freeze on the display screen.

  In the above embodiment, instead of the image pickup signal described for AF, the image pickup signal used for applications such as exposure control, white balance control, subject feature point detection control, subject movement detection control, etc. Either of these can be used.

Second Embodiment
In the first embodiment, the live view imaging signal is output from both of the two imaging signal outputs Ch1 and Ch2 from the imaging unit 204, while the AF evaluation value detection imaging signal is output only from Ch2. It is configured as follows. In contrast, the second embodiment is configured such that the output signal from Ch1 can be used as the AF evaluation value detection imaging signal. Note that the hardware configuration of the digital camera (imaging device) used in the second embodiment is the same as that of the digital camera 100 used in the first embodiment, and thus description thereof is omitted here.

  FIG. 13 is a timing chart of image signal processing in a case where the acquisition source of the live view imaging signal is switched from Ch1 to Ch2 and the output signal from Ch2 is used as it is for live view in step S904 of the flowchart of FIG. It is. However, the switching of the imaging drive mode of Ch2 in step S903 is assumed to be switched to the imaging drive mode for high-definition live view (high-definition live view A2). As a result, the AF process performed in step S902 is temporarily stopped.

  After switching the acquisition source of the live view imaging signal, when an independent AF operation is required, the AF imaging drive mode is switched so that the AF evaluation value detection imaging signal is output from Ch1. Thereby, the AF process can be resumed. On the other hand, when AF processing is unnecessary, the output of the imaging signal from Ch1 may be stopped. FIG. 13 shows an example of switching to the AF imaging drive mode. In this embodiment, since the number of read rows is the same between the AF imaging and the normal live view imaging, the drive switching is not necessary, but the frame rate is switched to AF.

  FIG. 14 is a timing chart of image signal processing when the output signal from Ch2 is used for live view as it is after the acquisition source of the live view imaging signal is switched from Ch1 to Ch2 in step S1103 of the flowchart of FIG. It is. However, the switching of the imaging drive mode of Ch2 in step S1102 is switched to the imaging drive mode for high definition live view (high definition live view A2). After the acquisition source of the live view imaging signal is switched from Ch1 to Ch2, output of the imaging signal from Ch1 is stopped.

  In the second embodiment described above, as in the first embodiment, high-definition live view display can be performed in the operation standby state of the second switch SW2.

<Other embodiments>
Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program code. It is a process to be executed. In this case, the program and the storage medium storing the program constitute the present invention.

DESCRIPTION OF SYMBOLS 101 Display part 204 Image pick-up part 205 AF evaluation value detection part 206 Image processing part 210 System control part

Claims (16)

An image sensor having a plurality of pixels arranged in a row direction and a column direction;
The first imaging mode in which image signals are read from pixels in a predetermined row of the image sensor and the number of image signal readout rows from the image sensor are different from the number of image signal readout rows in the first imaging mode. The second imaging mode and the third imaging mode in which the readout line of the image signal from the imaging device does not overlap with the readout line of the image signal in the first imaging mode and the second imaging mode are selected. Driving means for driving the image sensor in one or more imaging modes;
In order to separately output image signals read from the image sensor when at least two of the first imaging mode, the second imaging mode, and the third imaging mode are executed. Output means having at least two output paths including a first output path and a second output path;
The image signal in the first imaging mode output from the first output path is used for image display on the display means, and the image signal in the third imaging mode output from the second output path is used. When the image signal used for the image display is switched from the image signal in the first imaging mode to the image signal in the second imaging mode from a state where the image display is used for a different purpose from the image display, the image The acquisition source of the image signal used for display is temporarily switched from the first output path to the second output path, and the image signal in the second imaging mode is output from the first output path. Control means for switching the first imaging mode to the second imaging mode and switching the acquisition destination of the image signal used for the image display from the second output path to the first output path. Imaging device according to claim and.   The imaging apparatus according to claim 1, wherein the first imaging mode and the third imaging mode have the same number of readout rows of image signals from the imaging element.   Before the image signal used for the image display is temporarily switched from the first output path to the second output path, an exposure for obtaining an image signal output from the second output path, The imaging apparatus according to claim 1, wherein the exposure is the same as the exposure for obtaining an image signal output from the first output path. The first imaging mode is a mode for acquiring an image signal in order to perform image display with a predetermined definition;
The second imaging mode is a mode in which an image signal is acquired in order to display an image with higher definition than the first imaging mode,
The imaging apparatus according to claim 1, wherein the third imaging mode is a mode for obtaining an image signal for performing autofocus on a subject. An image sensor having a plurality of pixels arranged in a row direction and a column direction;
The first imaging mode in which image signals are read from pixels in a predetermined row of the image sensor and the number of image signal readout rows from the image sensor are different from the number of image signal readout rows in the first imaging mode. The second imaging mode, the third imaging mode and the fourth imaging mode in which the readout line of the image signal from the imaging element does not overlap with the readout line of the image signal in the first imaging mode and the second imaging mode. Drive means for driving the image sensor in one or a plurality of imaging modes selected from an imaging mode;
An image signal read from the image sensor when at least two of the first imaging mode, the second imaging mode, the third imaging mode, and the fourth imaging mode are executed. Output means having at least two output paths including a first output path and a second output path for separately outputting
The image signal in the first imaging mode output from the first output path is used for image display on the display means, and the image signal in the third imaging mode output from the second output path is used as the image signal. When the image signal used for the image display is switched from the image signal in the first imaging mode to the image signal in the second imaging mode from a state where it is used for an application different from the image display, the second The third imaging mode is switched to the fourth imaging mode so that the image signal in the fourth imaging mode is output from the output path, and the acquisition destination of the image signal used for the image display is the first Is temporarily switched from the output path to the second output path, and the first imaging mode is changed to the second output path so that an image signal in the second imaging mode is output from the first output path. Imaging model The image signal used for the image display is switched from the second output path to the first output path, and the image signal in the third imaging mode is output from the second output path. As described above, an imaging apparatus comprising: a control unit that switches the fourth imaging mode to a mode for switching to the third imaging mode.   The imaging apparatus according to claim 5, wherein the second imaging mode and the fourth imaging mode have the same number of readout rows of image signals from the imaging element. The first imaging mode is a mode for acquiring an image signal in order to perform image display with a predetermined definition;
Each of the second imaging mode and the fourth imaging mode is a mode for acquiring an image signal in order to perform an image display with a higher definition than the first imaging mode,
The imaging apparatus according to claim 5 or 6, wherein the third imaging mode is a mode for obtaining an image signal for performing autofocus on a subject.   Before the image signal used for the image display is temporarily switched from the first output path to the second output path, the frame rate of the image signal output from the second output path is changed to the first output path. The imaging apparatus according to claim 1, wherein the imaging apparatus has the same frame rate as that of an image signal output from the output path. An image sensor having a plurality of pixels arranged in a row direction and a column direction;
The first imaging mode in which image signals are read from pixels in a predetermined row of the image sensor and the number of image signal readout rows from the image sensor are different from the number of image signal readout rows in the first imaging mode. The second imaging mode and the third imaging mode in which the readout line of the image signal from the imaging device does not overlap with the readout line of the image signal in the first imaging mode and the second imaging mode are selected. Driving means for driving the image sensor in one or more imaging modes;
In order to separately output image signals read from the image sensor when at least two of the first imaging mode, the second imaging mode, and the third imaging mode are executed. Output means having at least two output paths including a first output path and a second output path;
The image signal in the first imaging mode output from the first output path is used for image display on the display means, and the image signal in the third imaging mode output from the second output path is used as the image signal. When the image signal used for the image display is switched from the image signal in the first imaging mode to the image signal in the second imaging mode from a state where it is used for a purpose different from the image display, the third The processing for the purpose different from the image display using the image signal in the imaging mode is temporarily stopped, and the image signal in the second imaging mode is output from the second output path. Driving in the second imaging mode, switching the acquisition destination of the image signal used for the image display from the first output path to the second output path, and from the first output path to the third imaging mode The first imaging mode is switched to the third imaging mode so that the image signal is output, and the image is output using the image signal in the third imaging mode output from the first output path. An image pickup apparatus comprising: control means for resuming processing for a use different from display.   The use different from the image display is any of auto focus control, exposure control, white balance control, subject feature point detection control, and subject motion detection control. The imaging device according to any one of 1 to 9.   The time from the start of output of the image signal in the second imaging mode until the image signal is processed for signal display and output to the display means is the image signal in the first imaging mode. 11. The method according to claim 1, wherein the time from the start of output to the display means after performing signal processing for displaying the image is output to the display means. The imaging device described. An image signal is acquired from an imaging device having a plurality of pixels arranged in a row direction and a column direction through a plurality of output paths including a first output path and a second output path, and displayed using the acquired image signals An image signal processing method for displaying an image on a means,
From a state in which the image signal output from the first output path is used for the image display by driving the image sensor in a first imaging mode in which an image signal is read from pixels in a predetermined row of the image sensor. The image display using the image signal output from the first output path by driving the image sensor in the second imaging mode in which the number of rows of image signals to be read is larger than that in the first imaging mode. When switching,
Using an image signal output from the first output path in the first imaging mode for the image display;
By driving the imaging device in a third imaging mode in which a readout row of image signals from the imaging device does not overlap with readout rows of image signals in the first imaging mode and the second imaging mode. Using the image signals output from the two output paths for an application different from the image display;
Temporarily switching the acquisition destination of the image signal used for the image display from the first output path to the second output path;
Switching the driving in the first imaging mode to the driving in the second imaging mode so that an image signal in the second imaging mode is output from the first output path;
And a step of switching the acquisition destination of the image signal used for the image display from the second output path to the first output path. An image signal is acquired from an imaging device having a plurality of pixels arranged in a row direction and a column direction through a plurality of output paths including a first output path and a second output path, and displayed using the acquired image signals An image signal processing method for displaying an image on a means,
From a state in which the image signal output from the first output path is used for the image display by driving the image sensor in a first imaging mode in which an image signal is read from pixels in a predetermined row of the image sensor. The image display using the image signal output from the first output path by driving the image sensor in the second imaging mode in which the number of rows of image signals to be read is larger than that in the first imaging mode. When switching
Using an image signal output from the first output path in the first imaging mode for the image display;
By driving the imaging device in a third imaging mode in which a readout row of image signals from the imaging device does not overlap with readout rows of image signals in the first imaging mode and the second imaging mode. Using the image signal output from the two output paths for an application different from the image display;
The third drive mode of the image pickup device is changed to a fourth image pickup mode in which an image signal read-out row from the image pickup device does not overlap an image signal read-out row in the first image pickup mode and the second image pickup mode. Switching from the imaging mode of
Temporarily switching the acquisition source of the image signal used for the image display from the first output path to the second output path;
Switching the first imaging mode to the second imaging mode so that an image signal in the second imaging mode is output from the first output path;
Switching the acquisition source of the image signal used for the image display from the second output path to the first output path;
Switching the fourth imaging mode to a mode for switching to the third imaging mode so that an image signal in the third imaging mode is output from the second output path. Image signal processing method. An image signal is acquired from an imaging device having a plurality of pixels arranged in a row direction and a column direction through a plurality of output paths including a first output path and a second output path, and displayed using the acquired image signals An image signal processing method for displaying an image on a means,
From a state in which the image signal output from the first output path is used for the image display by driving the image sensor in a first imaging mode in which an image signal is read from pixels in a predetermined row of the image sensor. The image display using the image signal output from the second output path by driving the image sensor in the second imaging mode in which the number of rows of image signals read out is larger than that in the first imaging mode. And switching the image sensor from the image sensor and driving the image sensor in a third imaging mode in which the image signal readout line does not overlap the image signal readout line in the first imaging mode and the second imaging mode. From the state where the image signal output from the second output path is used for a different purpose from the image display, the third imaging mode is When the image signal at the de is to be outputted,
Using an image signal output from the first output path in the first imaging mode for the image display;
Using the image signal output from the second output path in the third imaging mode for an application different from the image display;
Temporarily stopping processing for an application different from the image display using the image signal in the third imaging mode;
Starting driving in the second imaging mode so that an image signal in the second imaging mode is output from the second output path;
Switching the acquisition destination of the image signal used for the image display from the first output path to the second output path;
Switching the first imaging mode to the third imaging mode so that an image signal in the third imaging mode is output from the first output path;
And a step of resuming processing for an application different from the image display using the image signal in the third imaging mode output from the first output path. An imaging apparatus capable of acquiring an image signal from an imaging element through a first output path and a second output path,
When the image signal from the first output path is used for a live view on a display unit, and the image signal from the second output path is used for an application different from the live view, the second output The drive mode of the image sensor is switched so that the image signal output from the path can be used for live view display at the first definition, and the acquisition destination of the image signal used for the live view is the first output. An image signal that can be used for live view at a second definition different from the first definition from the first output route is switched from the first output route to the second output route. Control means for switching the drive mode of the image sensor so that the image signal is output from the second output path to the first output path. Imaging device according to claim Rukoto. An image signal processing method in an imaging apparatus capable of acquiring an image signal from an imaging element through a first output path and a second output path,
Using the image signal from the first output path for live view on a display means;
Using the image signal from the second output path for an application different from the live view;
Switching the drive mode of the image sensor so that an image signal output from the second output path can be used for live view at a first definition;
Switching the acquisition source of the image signal used for the live view from the first output path to the second output path;
Driving the imaging device so that an image signal that can be used for live view at a second definition different from the first definition from the first output path is output from the first output path. A step of switching modes,
Switching the acquisition source of the image signal for the live view from the second output path to the first output path;
The image signal that can be used for an application different from the live view from the second output path is output from the second output path, or the image signal is not output from the second output path. And a step of switching the drive mode of the image sensor.

Images