JP6924089B2 - Imaging device and its control method - Google Patents

Description

The present invention relates to an image pickup apparatus and a control method thereof.

Images taken under a light source (flicker light source) whose light intensity changes periodically, such as a fluorescent tube, have image quality deterioration (brightness and color unevenness) due to changes in the light intensity of the light source in all or part of the image. Sometimes.

As a method for suppressing deterioration of image quality due to a flicker light source, Patent Document 1 detects the peak timing and period of flicker from an image that is not affected by the flicker light source and an image that is affected by the flicker light source, and detects the peak timing and period of flicker at a time when there is little change in the amount of light. It is disclosed to shoot.

Further, Patent Document 2 discloses that the peak timing and period of flicker are detected from a change in brightness of an image acquired at a frame rate higher than the flicker frequency, and an image is taken at a time when the change in light intensity is small.

Japanese Unexamined Patent Publication No. 2015-888917 Japanese Unexamined Patent Publication No. 2014-220764

In the method described in Patent Document 1, flicker may not be detected correctly when the influence of the flicker light source exists only in a part of the frame (here, the lower part) as in the image A shown in FIG. In Patent Document 1, the peak timing and period of flicker are detected from the change in brightness in the vertical direction of the image C obtained by dividing the image A affected by the flicker light source by the image B not affected by the flicker light source. However, in the example of FIG. 10, since the peak cannot be correctly detected from the change in brightness in the vertical direction in the image C, the peak timing and period of the flicker cannot be detected.

The method described in Patent Document 2 can detect even a flicker as shown in the image A of FIG. 10 in principle, but it is necessary to make the frame rate of the image for flicker detection sufficiently higher than the flicker frequency. (600 fps in Patent Document 2). In order to achieve such a high frame rate, it is necessary to significantly reduce the number of pixels per frame, but as a result, the image for flicker detection can be diverted to other uses (for example, live view display). difficult.

The present invention has been made in view of such problems of the prior art. An object of the present invention is to provide an image pickup apparatus capable of appropriately switching and using a plurality of flicker detection methods and a control method thereof.

The above-mentioned object is an imaging device capable of performing flicker detection of the first method and flicker detection of the second method based on an image acquired by an image sensor, and the first method is used for flicker detection before detection of a shooting start instruction. The first method has a control means for controlling the flicker detection after the detection of the shooting start instruction to use the second method. The first method includes an image that is not affected by the flicker light source of a predetermined frequency and the influence of the flicker light source. The second method is achieved by an image pickup apparatus characterized in that the method is different from the first method by using the image to be received.

According to the present invention, it is possible to provide an image pickup apparatus capable of appropriately switching and using a plurality of flicker detection methods and a control method thereof.

A block diagram showing a functional configuration example of a camera system as an example of an imaging device according to an embodiment. The figure regarding the flicker detection processing of the 1st method in embodiment Flow chart for flicker detection processing of the first method in the embodiment The figure regarding the flicker detection processing of the 2nd method in an embodiment Flow chart for flicker detection processing of the second method in the embodiment Time chart related to shooting timing control in the embodiment Flowchart related to processing from start-up to shooting in the embodiment Flowchart for processing from startup to shooting in another embodiment The figure regarding the flicker detection necessity determination of the 2nd method in another embodiment A diagram schematically showing a state in which a part of an image is affected by flicker.

● (1st embodiment)
Hereinafter, exemplary embodiments of the present invention will be described with reference to the drawings. In the following, an embodiment in which the present invention is applied to an interchangeable lens digital camera as an example of an imaging device will be described. However, the present invention is applicable to an electronic device having an automatic focus adjustment function based on an image signal. Such electronic devices include, but are not limited to, digital cameras, mobile phones, personal computers (desktops, laptops, tablets, etc.), projectors, game consoles, robots, home appliances, drive recorders, and the like. ..

FIG. 1 is a block diagram showing a functional configuration example of an interchangeable-lens digital single-lens reflex camera (camera system), which is an example of an image processing device according to an embodiment of the present invention.
The camera system includes a camera body 100 and a lens unit 200 that is detachably attached to the camera body 100. The camera body 100 and the lens unit 200 are connected by a mount portion of each, and the mount portion is provided with an electrical contact group 210. The contact group 210 communicates control signals, status signals, data signals, etc. between the camera body 100 and the lens unit 200, supplies power to the photographing lens from the camera body 100, and determines whether the lens unit 200 is connected or not. It is possible to detect with the main body 100. The contact group 210 may transmit a signal other than an electric signal, for example, an optical signal, as long as it enables communication between the camera body 100 and the lens unit 200. For convenience, the photographing lens 201 built in the lens unit 200 is shown as if it were a single lens, but it is actually composed of a plurality of lenses.

The luminous flux from the subject is guided to the quick return mirror 102 via the photographing lens 201 and the aperture 202. The quick return mirror 102 is movable in the direction of the arrow, and in the illustrated state (down state), the central portion is formed in a half mirror so that a part of the light flux incident from the lens unit 200 is transmitted. The luminous flux transmitted through the half mirror portion is reflected in the direction of incident on the AF sensor unit 104 by the sub mirror 103 provided on the back surface of the quick return mirror 102.

The AF sensor unit 104 is composed of a field lens, a reflection mirror, a secondary imaging lens, an aperture, a line sensor, etc. arranged near the imaging surface, and is used for automatic focus detection (phase difference AF) of the phase difference method. Generate a pair of image signals to use. The focus detection circuit 105 uses a pair of image signals generated by the AF sensor unit 104 to acquire the defocus amount and the defocus direction. The system controller 120 drives and controls the focus lens of the lens unit 200 based on the defocus amount and the defocus direction, and adjusts the focus of the lens unit 200.

On the other hand, of the luminous flux incident from the lens unit 200, the luminous flux reflected by the quick return mirror 102 in the down state is emitted through the pentaprism 101 and the eyepiece 106. The photographer can confirm the photographing range by observing the emitted light. A photometric sensor for obtaining luminance information of the subject is provided in the vicinity of the eyepiece 106, and the output of the photometric sensor is supplied to the system controller 120 via the photometric circuit 107. The system controller 120 performs automatic exposure control (AE) using the brightness information of the subject.

At the time of shooting, the quick return mirror 102 moves upward (up state), and the light beam incident from the lens unit 200 is incident on the image sensor 112 via the focal plane shutter 108 and the optical filter 109, which are mechanical shutters. When the quick return mirror 102 is up, the sub mirror 103 is folded.

The optical filter 109 has the functions of both an infrared cut filter and an optical low-pass filter. The focal plane shutter 108 has a front curtain and a rear curtain, and transmits or blocks the light flux from the lens unit 200 according to the control of the system controller 120.

The system controller 120 includes a programmable processor such as a CPU and an MPU, a non-volatile memory (ROM) for storing programs, setting values, GUI data, etc., and a volatile memory (RAM) for expanding or using the program as a work area. ). The system controller 120 executes a program to control the operation of the components of the camera body 100 and the lens unit 200, thereby realizing the functions of the entire camera system including the flicker detection operation described later. In the present specification, the operation executed by the system controller 120 as the main body is realized by executing the program read into the RAM by the programmable processor. At least a part of the operation realized by the system controller 120 by software may be realized by a hardware circuit such as an ASIC.

The system controller 120 is communicably connected to the lens controller 207 in the lens unit 200 via the contact group 210 described above. The lens controller 207 controls the operation of the lens control circuit 204 and the aperture control circuit 206 by the control from the system controller 120, and transmits the information of the lens unit 200 to the system controller 120. The information of the lens unit 200 may include, for example, the position of the focus lens, the set aperture value, the focal length of the photographing lens 201, and the like.

Further, the lens controller 207 is provided with a rewritable non-volatile memory that stores unique information of the lens unit 200 (for example, focal length, open aperture, individual identification information (lens ID)) and information received from the system controller 120. ing.

The lens driving mechanism 203 drives the focus lens included in the photographing lens 201 in the optical axis direction under the control of the lens control circuit 204. The aperture drive mechanism 205 drives the aperture 202 according to the control of the aperture control circuit 206.

A shutter charge mirror drive mechanism 110 that controls the up / down drive of the quick return mirror 102 and the shutter charge of the focal plane shutter 108 is connected to the system controller 120. Further, a shutter control circuit 111 for controlling the traveling of the front curtain and the rear curtain of the focal plane shutter 108 is connected to the system controller 120. The EEPROM 122 stores parameters that need to be adjusted when the system controller 120 controls the camera system, individual identification information (camera ID) of the camera body 100, AF correction data adjusted by the reference lens, AE correction data, and the like. ..

The system controller 120 controls the exposure at the time of shooting according to the aperture value (Av) and the shutter speed (Tv) determined by the above-mentioned AE control. Specifically, the size of the aperture 202 of the aperture 202 is controlled via the lens controller 207, and the operation of the focal plane shutter 108 is controlled via the shutter control circuit 111.

The front curtain and the rear curtain of the focal plane shutter 108 are spring-driven and require a spring charge operation before operation. The shutter charge mirror drive mechanism 110 controls the spring charge operation and also controls the up / down operation of the quick return mirror 102.

The image data controller 115 is composed of, for example, a DSP (digital signal processor) and controls the driving and reading operations of the image sensor 112. Further, the image data controller 115 corrects or processes the image data digitized by the A / D converter 113 based on the control of the system controller 120. The correction / processing performed by the image data controller 115 includes, for example, color interpolation and white balance adjustment.

The image data controller 115 can acquire, for example, the luminance information for each divided region of the image from the image data and supply it to the system controller 120. In this way, the system controller 120 can obtain the luminance information of the subject even when the output of the photometric sensor cannot be obtained in the mirror lockup state.

The timing pulse generation circuit 114 outputs a pulse signal necessary for driving the image pickup device 112. The A / D converter 113 converts the analog signal corresponding to the subject image output from the image pickup device 112 into a digital signal according to the timing pulse from the timing pulse generation circuit 114. The DRAM 121 is an example of a storage device for temporarily storing image data (digital data) before processing or data conversion into a predetermined format, for example.

The image codec 119 is a circuit for encoding the image data stored in the DRAM 121 in a predetermined format (for example, JPEG format) and decoding the encoded image data. The encoded image data is recorded on the recording medium 400 as an image file. The recording medium 400 is a built-in memory or a detachable storage medium, and a memory card is typical, but the recording medium 400 is not limited thereto. Further, it may be recorded in an external device by using wireless communication or the like.

The image data controller 115 converts the image data on the DRAM 121 into an analog signal by the D / A converter 116 and outputs the image data to the encoder circuit 117. The encoder circuit 117 converts the output of the D / A converter 116 into a video signal (for example, an NTSC signal) necessary for driving the image display circuit 118, which is generally a liquid crystal display panel.

The operation display circuit 123 views information on the operation mode of the camera system, exposure information (shutter speed, aperture value, etc.) through the first display device 124 provided on the surface of the housing of the camera body 100 and the eyepiece 106. It is displayed on a second display device 125 capable of displaying the image.

The system controller 120 is connected to buttons and switches that constitute an operation unit for the user to give various instructions and settings to the camera body 100. The following are shown in FIG. 1 as these buttons and switches. Shooting mode selection button 130, main electronic dial 131, decision SW 132, AF point selection button 133, AF mode selection button 134, metering mode selection button 135, release SW1 136, release SW2 137, finder mode selection SW 138. These are just examples.

Release SW1 136 and Release SW2 137 are switches that are turned on by pressing the release button halfway or fully. Turning on the release SW1 136 corresponds to an instruction to start a shooting preparation operation (AF, AE, etc.). Further, turning on the release SW2 137 corresponds to an instruction to start a shooting operation for recording.

The finder mode selection SW138 is a switch for selecting whether or not to make the image display circuit 118 function as an electronic viewfinder. When the image display circuit 118 functions as an electronic viewfinder, the system controller 120 reads an image signal from the image sensor 112 at a predetermined frame rate, generates an image for live view display, and sequentially displays the image on the image display circuit 118. The image display circuit 118 can function as an electronic viewfinder by displaying continuously captured images in substantially real time. On the other hand, when the image display circuit 118 is not functioned as an electronic viewfinder, the user can check the shooting range from the eyepiece 106 (optical viewfinder mode).

Next, the flicker detection process in this embodiment will be described. In the present specification, the flicker means a periodic change in the amount of light, and a light source having flicker (the amount of light changes periodically) is called a flicker light source. Further, the flicker detection is the detection of the presence or absence of flicker of the light source and the timing (for example, based on the vertical synchronization signal) of the (maximum or minimum) peak of the flicker frequency (period) and the amount of light. In the case of a general fluorescent tube, the flicker frequency is 100 Hz or 120 Hz because it is twice the frequency of the power supply used for lighting.

The camera system of the present embodiment can detect flicker by using the first method and the second method. First, the first method will be described with reference to FIGS. 2 and 3. The flicker detection by the first method uses an image that is not affected by the flicker light source of a predetermined frequency and an image that is affected by the flicker light source. Here, as a typical example, it is assumed that the flicker frequency is 100 Hz (period 10 ms). FIG. 2 is a diagram schematically showing a photographing operation and a flicker detection process at the time of flicker detection of the first method. Further, FIG. 3 is a flowchart relating to the flicker detection process of the first method.

In S201, the system controller 120 controls the image data controller 115, drives the image sensor 112 so as to perform reading in the first storage time, and acquires the image data of the first image. The first image has at least the resolution of the live view image to be displayed when the image display circuit 118 functions as an electronic viewfinder. The system controller 120 sets the first accumulation time to the assumed flicker cycle or a multiple thereof so that the first image is not affected by the flicker even if the environmental light source includes the flicker light source. Here, 100 Hz flicker is assumed, and it is assumed that the system controller 120 sets 10 ms as the first storage time. The system controller 120 stores the acquired image data in, for example, the DRAM 121.

S202 is a procedure to be executed as needed, and the system controller 120 causes the image display circuit 118 to display a live view image based on the first image acquired in S201. As described above, since the resolution of the first image has a resolution that can be used as a live view image, it is not necessary to acquire the live view image again. If necessary, image processing (for example, scaling or signal format conversion) for converting the first image into a live view image may be executed.

In S203, the system controller 120 controls the image data controller 115, drives the image sensor 112 so as to perform reading in the second storage time, and acquires the image data of the second image. The second image may have the same resolution as the first image. When the environmental light source includes a flicker light source, the system controller 120 sets the second accumulation time shorter than the assumed flicker cycle so that the second image is affected by the flicker. Here, it is assumed that the system controller 120 sets 1 ms as the second storage time. Regarding the second image, the influence of flicker generated on the image differs depending on the time for reading the accumulated charge corresponding to the second image from the image sensor 112. Therefore, in the present embodiment, in order to detect flicker with high accuracy using the second image, the reading time of the second image is controlled so as to be a period of one cycle or more of the change in the amount of light of the flicker. Specifically, in the present embodiment, the system controller 120 sets 16.7 ms as the read time of the image sensor 112 corresponding to the second image, and controls the drive of the image sensor 112. Then, the system controller 120 stores the acquired image data in, for example, the DRAM 121.

In S204, the system controller 120 calculates the ratio of the first and second images acquired in S201 and S203. Specifically, the system controller 120 aligns the first image and the second image, and then obtains a value ratio for each corresponding pixel between the corresponding regions. Therefore, a third image is obtained by calculating the ratio.

In the example shown in FIG. 2, the pixel value of the second image B affected by flicker is divided by the pixel value of the first image A not affected by flicker. By this division, a third image B'with the influence of the subject removed is obtained, and if flicker is present, a change in brightness appears in the third image B'.
In addition to the method of obtaining the ratio, after adjusting the pixel value of one image so that the exposure times of the first image and the second image are the same, the first image is subtracted from the second image. A third image may be obtained.

Further, the third image may be generated by using the same first image. For example, in the case of FIG. 2, one third image may be generated from the image C and the image B, and one third image may be generated from the image C and the image D. Alternatively, after acquiring the first image A, the second images B1 and B2 are acquired in succession for two frames, and the third image is obtained from the first image A and the second image B1, one of the first images. One may be generated from the image A and the second image B2.

Although the flicker detection accuracy can be improved by obtaining the third image, it is also possible to detect the flicker from the second image without obtaining the ratio.

In S205, the system controller 120 detects a change in the brightness of the third image B'in the vertical direction by obtaining, for example, the average value of each pixel row of the third image B'. Then, the system controller 120 can detect, for example, the timing at which the luminance change becomes the minimum starting from the vertical synchronization signal as the peak (maximum and minimum) timing of the luminance change. When the sign of the amount of change in brightness changes from positive to negative before and after the peak, it can be determined to be maximum, and when it changes from negative to positive, it can be determined to be minimum.

In S206, the system controller 120 determines whether or not S205 has been executed twice, proceeds to S207 if it is determined to be executed, returns the process to S201 if it is not determined to be executed, and first image. C, the second image D, and the third image D'are acquired.

In S207, the system controller 120 obtains the flicker period and / or frequency from the interval (time difference) between the adjacent maximum positions (or minimum positions) of the brightness change. When the magnitude of the brightness change is equal to or less than the threshold value in S205, the system controller 120 determines that the flicker does not exist. Further, when the brightness change exceeds the threshold value but the peak cannot be detected, the system controller 120 determines that the flicker detection has failed.

Although the configuration in which S205 is executed twice in order to detect the peak interval is described here, there is no limit to the number of times S205 is executed as long as two or more adjacent peaks of the same type can be detected. In other words, it may be determined by S206 whether or not two or more adjacent peaks of the same type are detected. Further, in order to improve the detection accuracy of the flicker cycle and / or the frequency, the flicker cycle and / or the frequency detected a plurality of times may be averaged. Further, the reading time of the image pickup device 112 may be controlled so that at least two or more flicker peaks occur in the image, and the flicker cycle may be detected based on the image. In this case, for example, the image sensor 112 is driven by setting the flicker light intensity change to two cycles or more, and it is determined by S206 whether or not two or more flicker peaks adjacent to each other are detected in the acquired image. Just do it.

As described above, in the first method, since the image used for flicker detection can be used as a live view image, the image display circuit 118 can function as an electronic viewfinder during shooting standby or moving image recording. It is useful for flicker detection in.

Next, the second method of flicker detection will be described with reference to FIGS. 4 and 5. FIG. 4 is a diagram schematically showing a photographing operation and a flicker detection process at the time of flicker detection of the second method. Further, FIG. 5 is a flowchart relating to the flicker detection process of the second method.

In S301, the system controller 120 controls the image data controller 115 so as to acquire the captured image at a frame rate having a cycle shorter than the assumed flicker cycle. Here, it is assumed that 12 frames are continuously read at a frame rate of 600 fps, which is the least common multiple of the flicker frequencies (100 Hz and 120 Hz) assumed in advance. Therefore, the system controller 120 controls the image data controller 115 so as to be driven with an accumulation time of about 1.667 ms. Note that 600 fps is an example, and a lower or higher frame rate may be adopted. The system controller 120 stores the acquired image data in, for example, the DRAM 121.

Here, the reason for reading 12 frames continuously is to set the total storage time to 12 × 1.667 ms = about 20 ms so that flicker of two consecutive cycles is included regardless of the frequency of the commercial power supply. Is. Thereby, the flicker cycle and / or the frequency can be detected from the peak interval (time difference of the peak timing). Further, the system controller 120 sets the image data controller 115 to be thinned out or added read in order to realize reading at 600 fps. The ratio of thinning out and the number of pixels to be added depend on the time required for reading out the image sensor 112. For example, in the ROM in the system controller 120, the ratio of thinning out according to the frame rate and the number of pixels to be added can be stored in advance.

It is also possible to register in advance the image reading operation for flicker detection performed by each of the first method and the second method in the image data controller 115 as a reading mode. In this case, the system controller 120 need only set the read mode according to the flicker detection method in the image data controller 115.

In S302, the system controller 120 calculates a representative luminance value for each of the acquired 12 frames of images. Here, the integrated value of the brightness of all pixels is used as the representative brightness value, but other values such as the average brightness of all pixels may be used.

In S303, the system controller 120 detects the presence / absence of flicker and the period and / or frequency from the obtained change in the representative luminance value. When the commercial power frequency is 50 Hz and the storage time is 1.667 ms, the flicker is read out 6 times per cycle. Therefore, since the n + 6th frame (n is an integer of 0 or more) corresponds to a state in which the amount of light of the flicker light source is substantially equal, the representative luminance values are also substantially equal. Similarly, when the commercial power frequency is 60 Hz, the n + 5th frame (n is an integer of 0 or more) corresponds to a state in which the amount of light of the flicker light source is substantially equal, so that the representative luminance values are also substantially equal.

The system controller 120 describes the representative luminance values for 12 frames continuously read out.
First evaluation value = sum of difference absolute values between the representative luminance value of the nth frame and the representative luminance value of the (n + 6) frame (n = 1 to 6)
Second evaluation value = sum of difference absolute values between the representative luminance value of the nth frame and the representative luminance value of the (n + 5) frame (n = 1 to 5)
To be calculated respectively.

Then, the system controller 120
If the first evaluation value <threshold value and the second evaluation value <threshold value, there is no flicker. If the first evaluation value <threshold value and the second evaluation value ≥ threshold value, there is a flicker of 50 Hz. If the evaluation value <threshold value, it is determined that 60 Hz flicker exists. Here, the threshold value can be experimentally determined in advance.

In S304, the system controller 120 detects the peak of the luminance change from the representative luminance value. There is no particular limitation on the peak detection method, but among the representative luminance values of the frames obtained within one cycle of flicker, the maximum value L (n) and the frames before and after the frame in which the maximum value is obtained are obtained. It can be obtained from the representative luminance values L (n-1) and L (n + 1). Specifically, a straight line (slope a) passing through L (n), one of L (n-1) and L (n + 1) having a smaller value, and L (n-1) and L (n + 1). It can be obtained as a peak at the intersection with a straight line having a slope −a through one of the larger values. Further, the brightness corresponding to this intersection corresponds to the peak light intensity. The peak may be obtained based on the minimum value of the representative luminance value.

In the second method, by obtaining the representative luminance value reflecting all the pixel values in the frame, for example, as shown in the lowermost part of FIG. 4, only a part of the image is affected by the flicker. Even the flicker period and / or frequency can be detected. However, by dividing the image for flicker detection into a plurality of regions in the same manner and executing the flicker detection process described above for each region, the flicker cycle and / or frequency can be detected more accurately. When flicker is detected only in a part of the areas, the system controller 120 may determine that the flicker detected in those areas actually exists, for example, if the areas where the flicker is detected are arranged in a certain direction. .. Alternatively, the system controller 120 may determine that the detected flicker actually exists if the variation in the detected flicker frequency and peak timing is less than a predetermined threshold value.

Next, the flickerless shooting operation in the camera system of the present embodiment will be described with reference to FIG. FIG. 6 is a schematic diagram relating to exposure timing control according to the detected flicker in order to obtain an image in which the influence of flicker is suppressed. Here, it is assumed that the flicker detection is completed before the system controller 120 detects the still image shooting start instruction (release SW2 137 is on).

Since the peak timing and period of the flicker are detected by the flicker detection of the first method or the second method described above, the peak timing of the flicker is known. In FIG. 6, the peak timing is represented by a flicker synchronization signal. When the system controller 120 detects the shooting start instruction of the still image, the system controller 120 controls the exposure start timing so that the exposure is performed in a specific section of the flicker cycle. Specifically, the system controller 120 controls the timing of exposure start so that exposure is performed in the same section within the flicker cycle if the shutter speed is the same. As a result, since shooting is performed with the same amount of light, the influence of flicker can be suppressed.

In the example shown in FIG. 6, the exposure start timing is controlled so that the flicker peak timing is located at the center of the exposure period. By determining the exposure timing so as to include the timing at which the brightness of the flicker light source is maximized, the brightness of the image can be kept substantially constant even when the shutter speed determined by the AE changes. At this time, the system controller 120 controls the operation of the focal plane shutter 108 at a timing in consideration of the shutter speed by the AE and the time lag from when the instruction is given until the focal plane shutter 108 starts operating. As a result, the influence of flicker on the captured image can be suppressed. If flicker is not detected at the time of detecting the shooting start instruction, the exposure may be started after the flicker detection is executed after the shooting start instruction is detected.

Next, the flicker detection operation in the camera system according to the present embodiment will be described with reference to the flowchart of FIG. 7. FIG. 7 is a flowchart relating to the operation after the power of the camera system is turned on until the still image shooting is executed.

For example, when the power is turned on by operating the power switch or the like, the system controller 120 executes startup processing such as checking the power supply voltage and the recording medium in S401. If no abnormality is detected in the startup process, the system controller 120 executes the flicker detection process according to the first method described with reference to FIG. 3 as a shooting standby operation in S402.

In S403, the system controller 120 determines whether or not the shooting start instruction is detected, proceeds to S404 if it is determined to be detected, and repeatedly executes S402 if it is not determined to be detected. In the flicker detection process according to the first method, since the first image is used as a live view image and displayed on the image display circuit 118, the image display circuit 118 functions as an electronic viewfinder.

In S402, after the flicker detection is completed, even if only the acquisition of the first image and the display on the image display circuit 118 (S201 and S202 in FIG. 3) are executed until, for example, a scene change is detected. good. This is because the flicker characteristics basically do not change in the same environment. Further, the frame rate of the live view image can be improved by not acquiring the second image.

If a shooting start preparation instruction (release SW1 136 is turned on) is detected before the shooting start instruction, the system controller 120 executes a shooting start preparation process such as AF or AE.

In S404, the system controller 120 determines whether or not flicker is detected by the first method during the shooting standby operation, and proceeds to S405 if it is determined to be detected, and to S407 if it is not determined to be detected. .. When flicker is not detected by the first method, there are cases where flicker does not actually exist and cases where flicker detection fails. When flicker detection fails, for example, a part of the image is affected by flicker, and even if it is determined that flicker is present from the magnitude of the brightness change, adjacent peaks of the same type are present. Sometimes the timing cannot be detected. Therefore, in the present embodiment, when the flicker is not detected by the first method, the flicker is detected by the second method.

In S407, the system controller 120 executes the flicker detection process according to the second method described with reference to FIG.
In S408, the system controller 120 determines whether or not flicker has been detected by the second method, and proceeds to S409 if it is determined to be detected, and to S406 if it is not determined to be detected.

In S405, the system controller 120 determines the shooting start timing based on the flicker detection result (peak timing and period or frequency) by the first method and, for example, the shutter speed determined by the AE process. The system controller 120 can determine the shooting start timing as described with reference to FIG.

In S409, the system controller 120 determines the shooting start timing based on the flicker detection result (peak timing and period or frequency) by the second method and, for example, the shutter speed determined by the AE process. The system controller 120 can determine the shooting start timing as described with reference to FIG.

In S406, the system controller 120 executes the shooting operation according to the shooting start timing determined in S405 or S409. If no flicker is detected by either the first method or the second method, the system controller 120 immediately starts the shooting operation.

Note that FIG. 7 describes the operation when still image shooting is performed from the shooting standby state. However, this embodiment can also be applied to still image shooting during moving image recording. In this case, the flicker detection process by the first method is executed in the shooting standby state before the start of the moving image recording, and the operation after S403 is executed when the shooting start instruction of the still image is detected during the moving image recording. Just do it.

As described above, according to the present embodiment, the flicker detection is executed by different methods before and after the detection of the shooting start instruction. In particular, before the detection of the shooting start instruction, the flicker detection is performed by a method that allows the image for flicker detection to be used for the live view display, and after the detection of the shooting start instruction, a method different from that before the detection of the shooting start instruction is performed. Changed to perform flicker detection. This enables flicker detection while displaying the live view before detecting the shooting start instruction. Further, after the detection of the shooting start instruction, various methods such as a method capable of detecting flicker that cannot be detected by the flicker detection method before the detection of the shooting start instruction can be used, so that the flicker detection accuracy can be improved. can.

● (Second embodiment)
Next, a second embodiment of the present invention will be described. In the first embodiment, when flicker is not detected by the first method, flicker is always detected by the second method. The present embodiment is different in that when the flicker is not detected by the first method, it is determined whether or not the flicker is detected by the second method. Also in this embodiment, the configuration of the camera system may be the same as that of the first embodiment.

FIG. 8 is a flowchart relating to the operation of the camera system of the present embodiment from the time the power is turned on until the still image shooting is executed, and the same reference numbers as those of FIG. 7 are added to the same operation as that of the first embodiment. The explanation is omitted.

In S404, the system controller 120 determines whether or not flicker is detected by the first method during the shooting standby operation, and proceeds to S405 if it is determined to be detected, and to S501 if it is not determined to be detected. ..

In S501, the system controller 120 determines whether or not the amplitude of the luminance change (for example, the difference between the minimum luminance value and the maximum luminance value) obtained by the flicker detection of the first method executed most recently is equal to or less than the threshold value. The luminance change may be, for example, at least one obtained from the images corresponding to the image B'and the image D'of FIG. The system controller 120 proceeds to S406 if the amplitude of the luminance change is determined to be less than or equal to the threshold value, and proceeds to S502 if it is not determined to be less than or equal to the threshold value. The threshold value used here can be predetermined. Further, the threshold value may be changeable by the user. The threshold value may be the same as or smaller than the threshold value for determining the presence or absence of flicker in the flicker detection process of the first method.

In S502, the system controller 120 determines whether or not the number of peaks in the luminance change obtained by the flicker detection process in the first method is a predetermined number or more, and if it is determined to be a predetermined number or more, the system controller 120 goes to S407, which is a predetermined number or more. If it is not determined, the process proceeds to S406.

The predetermined number used here has a value determined according to the accumulation / reading cycle and the flicker cycle. A predetermined number will be described with reference to FIG. For example, consider a case where an image is read out at 60 fps (period 16.7 ms) in the flicker detection process of the first method. In this case, the number of flicker luminance peaks included in each frame is 1 or 2 when the flicker frequency is 100 Hz, and 2 when the flicker frequency is 120 Hz.

However, for example, when the flicker frequency exceeds 120 Hz, the number of luminance peaks included in each frame is larger than 2 (3 or 4 in the example of FIG. 9). In the present embodiment, the flicker detection process of the second method assumes that the flicker frequency is 100 Hz or 120 Hz, so that other flicker frequencies cannot be correctly determined. For example, when the flicker detection process is performed in an environment where flicker does not occur, ideally there is no change in brightness and the number of peaks is zero. However, the luminance change occurs due to the variation and noise of the signal output from the image sensor 112, and many peaks of the luminance change are detected. In this case, the process proceeds to S406, and control is performed so that the flicker detection process of the second method is not executed. At this time, the processing of S501 determines that the amplitude of the luminance change is equal to or less than the threshold value, and basically transitions to S406. However, when a high-brightness subject is present in the subject, the signal output from the image sensor 112 is saturated according to the accumulation time, so that the image acquired in the first accumulation time and the image acquired in the second accumulation time are acquired. A brightness difference occurs between the image and the image, and the process proceeds to S502.

That is, when the number of peaks of the luminance change obtained by the flicker detection process in the first method is a predetermined number larger than the number of peaks expected when the flicker frequency is 100 Hz or 120 Hz, the flicker detection process in the second method is used. Is likely not to be detected correctly. Therefore, the flicker detection process of the second method is not performed.

According to the present embodiment, the same effect as that of the first embodiment can be obtained, and since the flicker detection by the second method is adaptively executed, an unnecessary shutter time lag can be suppressed. Is obtained.

Although the present invention has been described above as an exemplary embodiment, the present invention is not limited to the specific embodiment described. Various variants and modifications within the scope of the claims are also included in the invention.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 ... Camera body, 200 ... Lens unit, 112 ... Image sensor, 114 ... Timing pulse generation circuit, 115 ... Image data controller, 118 ... Image display circuit, 120 ... System controller

Claims (12)

An image pickup apparatus capable of first method flicker detection and second method flicker detection based on an image acquired by an image sensor.
It has a control means for controlling so that the first method is used for flicker detection before the detection of the shooting start instruction and the second method is used for the flicker detection after the detection of the shooting start instruction.
The first method uses an image that is not affected by a flicker light source of a predetermined frequency and an image that is affected by the flicker light source.
The second method is different from the first method.
An imaging device characterized by this. The imaging device according to claim 1, wherein an image that is not affected by the flicker light source is used for live view display. The first method detects the period and / or frequency of the brightness change of the flicker light source and the peak timing of the brightness from the image not affected by the flicker light source and the image affected by the flicker light source. The imaging device according to claim 1 or 2, wherein the imaging device is characterized. The imaging apparatus according to any one of claims 1 to 3, wherein the second method is a method capable of detecting flicker that cannot be detected by the first method. The imaging device according to any one of claims 1 to 4, wherein the second method uses a plurality of images acquired within the flicker cycle of the flicker light source. 5. The second method is characterized in that the period and / or frequency of the brightness change of the flicker light source and the peak timing of the brightness are detected based on the representative brightness values of the plurality of images. The imaging device described. The control means
When the shooting start instruction is detected, if flicker is not detected by the flicker detection of the first method, the flicker detection of the second method is executed.
If flicker is detected by the flicker detection of the first method when the shooting start instruction is detected, the flicker detection of the second method is not executed.
The imaging device according to any one of claims 1 to 6, wherein the imaging device is characterized by the above. The control means
When the shooting start instruction is detected, flicker is not detected by the flicker detection of the first method, and the amplitude of the brightness change of the flicker light source detected by the flicker detection of the first method is a threshold value. The imaging device according to claim 7, wherein the flicker detection of the second method is executed when the following is not the case. The control means
When the shooting start instruction is detected, flicker is not detected by the flicker detection of the first method, and the number of peak brightness of the flicker light source detected by the flicker detection of the first method is predetermined. The imaging apparatus according to claim 7, wherein the flicker detection of the second method is executed when the number is not more than the number. The control means further claims 1 to 9, wherein the shooting in response to the shooting start instruction is performed at a timing based on the peak timing of the flicker detected by the first method or the second method. The imaging apparatus according to any one of the following items. It is a control method of an image pickup apparatus capable of performing flicker detection of the first method and flicker detection of the second method based on an image acquired by an image sensor.
The control means has a control step that changes the flicker detection method depending on whether the shooting start instruction is before or after the detection.
Before the detection of the shooting start instruction, a method using an image that is not affected by the flicker light source of a predetermined frequency and an image that is affected by the flicker light source is used.
A control method for an imaging device, characterized in that. A program for causing a programmable processor included in an image pickup apparatus to function as a control means for the image pickup apparatus according to any one of claims 1 to 10.

Images