JP2005210216A - Image pickup device and method of processing noise in motion image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image pickup device capable of easily deciding a proper feedback coefficient in a circulation filtering processing in motion image photographing. <P>SOLUTION: The image pickup device 1A is provided with a circulation filter 26 in which an adder 263 adds a signal obtained by multiplying one-frame preceding image outputted from a frame delay 265 having a frame memory to hold image data by a gain coefficient (feedback coefficient) at a feedback amplifier 264 to a signal obtained by multiplying the present frame image by a gain coefficient to be set in an amplifier 262, thereby eliminating noises. In a correlation detecting section 261 of this circulation filter section 26, correlation between frames is obtained on the basis of an AF evaluation value for evaluating a focused state at the time of photographing, and a feedback coefficient corresponding to the correlation is decided. In this way, a proper feedback coefficient can be easily decided. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a technique of an imaging apparatus that captures an image of a subject and sequentially generates frames constituting a moving image.

  In moving image shooting with a digital camera (imaging device), the sensitivity tends to be insufficient with the increase in the number of pixels, and an increase in noise at low illumination is inevitable.

  Therefore, for example, Patent Document 1 discloses a technique for removing noise based on the correlation between frames using a circulation filter (feedback filter) as a technique for improving moving image shooting at low illumination. Such feedback filter processing will be briefly described with reference to FIG.

  FIG. 12 is a block diagram for explaining the cyclic filter processing according to the prior art.

  Image data (Bayer data) generated by the image sensor is subjected to pixel interpolation by the pixel interpolation unit 81, and the pixel-interpolated image data is converted into two circulation filter units 83 after the color space is converted by the color difference matrix unit 82. Entered. Then, noise removal during moving image shooting is performed by feedback filter processing in the circulation filter unit 83.

JP-A-6-38098

  In the technique of Patent Document 1 described above, the circulation filter process is performed using the feedback coefficient set based on the shutter speed. However, the optimum feedback coefficient cannot always be easily determined only by the shutter speed information.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a technique for an imaging apparatus that can easily determine an appropriate feedback coefficient in a cyclic filter process in moving image shooting.

  In order to solve the above problems, the invention of claim 1 is an image pickup apparatus, wherein (a) moving image shooting means for shooting a subject and sequentially generating frames constituting a moving image; and (b) the moving image shooting. Cyclic filter means for performing noise processing by multiplying a frame image sequentially generated by the means by a feedback coefficient, and (c) processing for activating the cyclic filter means and performing the noise processing according to a first imaging condition And (b-1) determining means for determining the feedback coefficient according to the second imaging condition.

  According to a second aspect of the present invention, in the imaging apparatus according to the first aspect of the invention, the first photographing condition is a condition relating to an exposure time.

  Further, the invention of claim 3 is the imaging apparatus according to claim 1 or 2, wherein (d) evaluation information generating means for generating evaluation information for evaluating the in-focus state based on the image data of the frame The second imaging condition is a condition related to the evaluation information.

  According to a fourth aspect of the present invention, in the imaging apparatus according to any one of the first to third aspects of the present invention, (e) a focal length changing unit that changes a focal length of the photographic lens is further provided. The shooting conditions are conditions related to the focal length.

  In addition, according to a fifth aspect of the present invention, in the imaging apparatus according to the fourth aspect of the invention, the processing means prohibits the noise processing during the change of the focal length by (c-1) the focal length changing means. Have means.

  The invention of claim 6 is the imaging apparatus according to any one of claims 1 to 5, wherein (f) measurement information generating means for measuring the amount of camera shake relating to the imaging apparatus and generating measurement information The second imaging condition is a condition related to the measurement information.

  The invention according to claim 7 is the imaging apparatus according to the invention according to claim 6, further comprising: (g) camera shake correction means for performing camera shake correction based on the measurement information, and the processing means includes (c− 2) When the camera shake correction means is disabled and the camera shake correction is not performed, the camera has a means for prohibiting the noise processing.

  Further, the invention of claim 8 is a moving image noise processing method, wherein (a) a moving image shooting step of capturing an image of a subject and sequentially generating frames constituting the moving image; and (b) a moving image shooting step sequentially in the moving image shooting step. A cyclic filter step of performing noise processing by multiplying a generated frame image by a feedback coefficient; and (c) a processing step of performing noise processing when the first imaging condition is satisfied, wherein the feedback coefficient is , And is set based on the second imaging condition.

  According to the first to eighth aspects of the present invention, the circulating filter means is activated according to the first imaging condition to perform noise processing, and the feedback coefficient of the circulating filter means is determined according to the second imaging condition. Therefore, it is possible to easily determine an appropriate feedback coefficient in the circulation filter processing in moving image shooting.

  In particular, in the second aspect of the invention, since the first imaging condition is a condition relating to the exposure time, the circulation filter process can be performed appropriately.

  In the invention of claim 3, since the second imaging condition is a condition relating to evaluation information for evaluating the in-focus state, an appropriate feedback coefficient can be determined more easily.

  In the invention of claim 4, since the second photographing condition is a condition relating to the focal length of the photographing lens, an appropriate feedback coefficient can be determined more easily.

  Further, in the invention of claim 5, since the noise processing is prohibited during the change of the focal length, it is possible to prevent the cyclic filter processing when the frame correlation before and after is low.

  In the invention of claim 6, since the second photographing condition is a condition relating to the measurement information of the camera shake amount, an appropriate feedback coefficient can be determined more easily.

  In addition, in the invention of claim 7, since noise processing is prohibited when camera shake correction is not performed, it is possible to prevent cyclic filter processing when the preceding and following frame correlation is low.

<First Embodiment>
<Principal configuration of imaging device>
FIG. 1 is a diagram illustrating a main configuration of an imaging apparatus 1A according to the first embodiment of the present invention. Here, FIGS. 1A to 1C correspond to a front view, a rear view, and a top view of the imaging apparatus 1A, respectively.

  The imaging device 1 </ b> A is configured as a digital camera and includes a photographing lens 10. In addition, a photometric sensor 11 is provided on the front surface of the image pickup apparatus 1A to perform photometry on the subject and generate a luminance signal.

  The image pickup apparatus 1A is provided with a mode change switch 12, a shutter button 13, and an NR mode selection switch 17 on the upper surface thereof.

  The mode changeover switch 12 is recorded in a still image shooting mode (REC mode) for capturing an image of a subject and recording the still image, a moving image mode for moving image shooting (MOVE mode), and a memory card 9 (see FIG. 2). This is a switch for switching between a playback mode (PLAY mode) for playing back an image.

  The shutter button 13 is a two-stage switch that can detect a half-pressed state (S1 on) and a fully pressed state (S2 on). When the shutter button 13 is half-pressed in the still image shooting mode, the zoom / focus motor driver 47 (see FIG. 2) is driven, and the operation of moving the shooting lens 10 to the in-focus position is performed. On the other hand, when the shutter button 13 is fully pressed in the still image shooting mode, a main shooting operation, that is, a recording shooting operation is performed.

  The NR (noise reduction) mode selection switch 17 includes a noise removal process “NR1” using a circulation filter process (described later) in moving image shooting at low illumination, a noise removal process “NR2” using multiframe exposure (described later), This is a switch for switching between “Auto” which automatically selects two types of noise processing NR1 and NR2 depending on the situation.

  An LCD (Liquid Crystal Display) monitor 42, an electronic viewfinder (EVF) 43, a camera shake correction switch 14, and a frame advance / zoom switch 15 are displayed on the back of the image pickup apparatus 1A. Is provided.

  The camera shake correction switch 14 controls the camera shake correction actuator driver 48 (FIG. 2) so that the camera shake detected by the camera shake sensor 49 (FIG. 2) that measures the amount of camera shake related to the imaging apparatus 1A does not adversely affect the shooting. It is a switch for setting a camera shake correction mode in which it is driven to perform camera shake correction. Here, the imaging sensor 16 can move two-dimensionally in a direction orthogonal to the optical axis of the photographing lens 10 by, for example, two actuators that perform linear driving.

  The frame advance / zoom switch 15 is composed of four buttons, and is a switch for instructing frame advance of a recorded image in playback mode and zooming at the time of shooting. By operating the frame advance / zoom switch 15, the zoom / focus motor driver 47 is driven to change the focal length related to the photographing lens 10.

  FIG. 2 is a diagram illustrating functional blocks of the imaging apparatus 1A.

  The imaging apparatus 1A includes an imaging sensor 16, a signal processing unit 2 connected to the imaging sensor 16 so as to be able to transmit data, an image processing unit 3 connected to the signal processing unit 2, and a camera control unit 40A connected to the image processing unit 3. And.

  The imaging sensor 16 is configured as an area sensor (imaging device) in which R (red), G (green), and B (blue) primary color transmission filters are arranged in a checkered pattern (Bayer array) in units of pixels. This is a pixel readout type. The imaging sensor 16 functions as a moving image capturing unit that captures an image of a subject and sequentially generates frames constituting the moving image as RAW image data (Bayer data).

  The signal processing unit 2 includes a CDS 21, an AGC 22, and an A / D conversion unit 23.

  The analog image signal acquired and output by the imaging sensor 16 is sampled by the CDS 21 and noise is removed, and then the AGC 22 multiplies the analog gain corresponding to the imaging sensitivity to perform sensitivity correction.

  The A / D conversion unit 23 is configured as a 14-bit converter, and digitizes the analog signal normalized by the AGC 22. The digitally converted image signal is subjected to predetermined image processing by the image processing unit 3 to generate an image file.

  The image processing unit 3 includes a RAW addition unit 30, a digital processing unit 3p, and an image compression unit 35. Further, the image processing unit 3 includes a ranging calculation / motion detection unit 36, an OSD unit 37, a video encoder 38, and a memory card driver 39.

  The digital processing unit 3p includes a pixel interpolation unit 31, a resolution conversion unit 32, a white balance control unit 33, and a gamma correction unit 34.

  The image data input to the image processing unit 3 is written into the image memory 41 in synchronization with the reading of the image sensor 16. Thereafter, the image data stored in the image memory 41 is accessed, and various processes are performed in the image processing unit 3.

  In the image data in the image memory 41, first, the white balance control unit 33 performs gain correction of RGB pixels independently, and RGB white balance correction is performed. In this white balance correction, a portion that is originally white from a photographic subject is estimated from brightness, saturation data, and the like, and an average value, a G / R ratio, and a G / B ratio for each of R, G, and B are obtained. Based on these pieces of information, the R and B correction gains are controlled.

  After the white balance correction of the image data, the pixel interpolation unit 31 masks each of the RGB pixels with the respective filter pattern, and the G pixel having the pixel value up to the high band is subjected to the median (intermediate value) filter. Replace with the average of the intermediate binary values. For the R pixel and the B pixel, average interpolation is performed for the same color in the surrounding nine pixels.

  The pixel-interpolated image data is subjected to nonlinear conversion suitable for each output device by the gamma correction unit 34, specifically, gamma correction and offset adjustment, and stored in the image memory 41.

  The image data stored in the image memory 41 is horizontally or vertically reduced or thinned out to the number of pixels set by the resolution conversion unit 32, and after compression processing by the image compression unit 35, the memory card driver 39 Is recorded on the memory card 9 set in At the time of this image recording, a photographic image with a designated resolution is recorded, and a screen nail image (VGA) for reproduction display is created and recorded linked to the photographic image. At the time of image reproduction, a screen nail image is displayed on the LCD monitor 42, thereby enabling high-speed image display.

  In addition, the resolution conversion unit 32 performs pixel thinning when displaying an image, and creates a low resolution image to be displayed on the LCD monitor 42 or the EVF 43. At the time of preview, a low resolution image of 640 × 240 pixels read out from the image memory 41 is encoded into NTSC / PAL by the video encoder 38, and this is reproduced as a field image on the LCD monitor 42 or the EVF 43.

  The ranging calculation / motion detection unit 36 divides the frame image into a plurality of blocks, performs BPF processing for each block, extracts the frequency component in the focus area, and compares and evaluates this for each frame. Then, the focus lens is driven. After such video (video) AF is performed and focusing is completed, the movement of the subject is detected based on the amount of change in the evaluation value in each block, and AF control is performed again.

  The AF evaluation value (evaluation information) for evaluating the in-focus state at the time of shooting described above is a low value when the ranging target is moved during the AF operation, and when the focus state is shifted to a constant in-focus state. High value is output. By using the AF evaluation value having such characteristics, the movement of the subject can be detected.

  The OSD (on-screen display) unit 37 can generate various characters, symbols, frames, and the like, and can superimpose them on an arbitrary position of the display image. The OSD unit 37 can display various characters, symbols, frames, and the like on the LCD monitor 42 as necessary.

  The camera control unit 40A includes a CPU and a memory, and is a part that comprehensively controls each unit of the imaging apparatus 1A. Specifically, the operation input performed by the photographer is processed with respect to the camera operation switch 50 having the mode switch 12 and the shutter button 13. In addition, the camera control unit 40A switches to a still image shooting mode, a moving image mode, and a playback mode in which a subject is imaged and the image data is recorded by operating the mode setting switch 12 by the photographer.

  The tripod detector 51 is a part that detects that a tripod is attached to a tripod hole 52 (shown by a broken line in FIGS. 1A and 1B) provided on the lower surface of the imaging apparatus 1A.

  In the imaging device 1A, the optical aperture of the aperture 44 is fixed open by the aperture driver 45 during preview display (live view display) in which the subject is displayed on the LCD monitor 42 in a moving image manner in the shooting preparation state before the actual shooting. Regarding the charge accumulation time (exposure time) of the image sensor 16 corresponding to the shutter speed (SS), the camera control unit 40A calculates exposure control data based on the live view image acquired by the image sensor 16. Then, feedback control is performed on the timing generator sensor driver 46 so that the exposure time of the image sensor 16 is appropriate based on a program diagram set in advance based on the calculated exposure control data.

  When the charge accumulation in the image sensor 16 is completed, the photoelectrically converted signal is shifted to the light-shielded transfer path in the image sensor 16 and read out from the transfer path via the buffer.

  In the imaging apparatus 1A having the above-described configuration, two types of noise removal processing can be performed in moving image shooting at low illuminance. These noise processing will be described in detail below.

<About circulation filter processing (NR1)>
FIG. 3 is a diagram for explaining the circulation filter processing in the imaging apparatus 1A.

  In the imaging apparatus 1A, the image processing unit 3 includes a black correction unit 24, a flaw correction unit 25, a circulation filter unit 26, a shading correction unit 27, and a white balance (WB) amplifier unit 28. In each of these units, image processing is performed on the RAW image data of the digital signal generated by the imaging sensor 16 and output from the signal processing unit 2. The RAW image data is image data output from the image sensor 16 and is image data before pixel interpolation.

  In addition to the pixel interpolation unit 31 and the gamma correction unit 34 shown in FIG. 2, the image processing unit 3 includes a matrix unit 61 and a color difference matrix unit 62 that perform color correction using, for example, a 3 × 3 matrix. And a contour correcting unit 63 is provided.

  The black correction unit 24 is a part that performs black level correction on the RAW image data of the frame generated by the imaging sensor 16.

  The flaw correction unit 25 is a part that corrects a pixel defect when a flaw, that is, a pixel defect exists in the RAW image data of the frame generated by the imaging sensor 16.

  The cyclic filter unit 26 includes a correlation detection unit 261, an amplifier unit 262, an addition unit 263, a feedback amplifier unit 264, and a frame delay unit 265.

  In this circular filter unit 26, a signal obtained by multiplying an image one frame before output from a frame delay unit 265 having a frame memory and holding image data by a feedback amplifier unit 264 by a gain coefficient (feedback coefficient), A signal obtained by multiplying the frame image by a gain coefficient set by the amplifier unit 262 is added by the adding unit 263. In other words, the cyclic filter unit 26 performs noise processing for multiplying the difference between the RAW image of the current frame generated by the imaging sensor 16 and the RAW image of the previous frame by the feedback coefficient and subtracting from the RAW image of the current frame. It becomes.

  The feedback coefficient set by the feedback amplifier unit 264 is determined according to the correlation between the current frame image and the previous frame image. Specifically, when the correlation between frames is large, the feedback coefficient of the feedback amplifier unit 264 is increased to increase the feedback amount and suppress uncorrelated noise. On the other hand, when the feedback coefficient is increased when the correlation between frames is small, image blurring occurs. Therefore, when the frame correlation is small, the feedback coefficient is decreased to reduce the amount of feedback and minimize image degradation. .

  Regarding the control of the feedback coefficient, the correlation detection unit 261 obtains a correlation between frames based on the AF evaluation value generated by the distance measurement calculation / motion detection unit 36, and a feedback coefficient corresponding to the correlation is determined.

  The shading correction unit 27 is configured such that the amount of light differs between the central portion and the peripheral portion of the image by the photographing lens 10 or the like with respect to the RAW image data of the frame subjected to the filtering process by the circulation filter unit 26, specifically, It is a part which corrects that a part becomes dark. In the shading correction unit 27, each part of the image is subjected to a process of amplifying with a different amplification factor.

  The WB amplifier unit 28 performs RGB white balance correction on the RAW image data of the frame subjected to the filter processing by the circulation filter unit 26 at a portion corresponding to the white balance control unit 33 described above.

  The RAW image data of the frame for which the white balance correction has been performed by the WB amplifier unit 28 is subjected to pixel interpolation by the pixel interpolation unit 31, and matrix calculation is performed by the matrix unit 61. The gamma correction unit 34 performs gamma correction, the color difference matrix unit 62 converts the luminance signal (Y) and the color difference signal (C), and then the contour correction unit 63 performs contour correction.

  In the imaging apparatus 1A, the above-described circulation filter unit 26 can perform noise removal processing NR1 in moving image shooting at low illuminance.

  FIG. 4 is a flowchart showing the operation of the circulating filter process in the imaging apparatus 1A. This operation is executed by the camera control unit 40A.

  In step S1, the feedback coefficient is set to a default value (for example, 0.2). That is, the gain coefficient in the feedback amplifier unit 264 of the circulation filter unit 26 is set to an initial value.

  In step S2, the VGA image is read as a moving image. That is, a frame image is read from the image sensor 16.

  In step S3, the exposure amount is calculated based on the image data read in step S2, and the optimum shutter speed, aperture value (Fno), and AGC gain of the AGC unit 22 are calculated.

  In step S4, the optical aperture of the aperture 44 is driven by the aperture driver 45 based on the aperture value calculated in step S3.

  In step S5, the gain coefficient of AGC 22 is set based on the AGC gain calculated in step S3.

  In step S6, it is determined whether the set gain in the AGC 22 is greater than a predetermined threshold value Ref1. Here, when the AGC gain corresponding to the photographing sensitivity is greater than the predetermined value Ref1, it is determined that the illumination is low, and the process proceeds to step S7. When the AGC gain is equal to or less than the predetermined value Ref1, the process proceeds to step S13.

  In step S7, it is determined whether the AF evaluation value calculated by the distance measurement calculation / motion detection unit 36 is larger than a predetermined threshold value Ref2. This is because, when the AF evaluation value is a certain value or more, it can be determined that the subject is still to some extent as described above. In this case, the filter processing in the circulation filter unit 26 is actively performed. This is the determination operation for If the AF evaluation value is larger than the predetermined value Ref2, the process proceeds to step S8. If the AF evaluation value is equal to or smaller than the predetermined value Ref2, the process proceeds to step S13.

  In step S8, it is determined whether or not the degree of correlation between frames based on the AF evaluation value is high. If the degree of correlation is high, the process proceeds to step S9. If the degree of correlation is low, the process proceeds to step S10.

  In step S9, the feedback coefficient of the feedback amplifier unit 264 is increased. In this case, for example, a feedback coefficient is added by 0.1.

  In step S10, the feedback coefficient of the feedback amplifier unit 264 is decreased. In this case, for example, 0.1 is subtracted from the feedback coefficient.

  Through the operations in steps S8 to S10 described above, the feedback coefficient is determined according to the condition (second imaging condition) regarding the AF evaluation value (evaluation information). As a result, an appropriate feedback coefficient can be easily determined.

  In step S11, a feedback coefficient limiter process is performed. That is, when the feedback coefficient is larger than the upper limit value (for example, 0.8), the upper limit value is limited, and when the feedback coefficient is smaller than the lower limit value (for example, 0), the feedback coefficient is limited to the lower limit value. Thereby, excessive feedback is suppressed and appropriate noise removal can be performed.

  In step S12, the circulation filter unit 26 performs circulation filter processing.

  In step S13, the feedback coefficient of the feedback amplifier unit 264 is set to a default value (for example, 0.2).

  In step S14, the image processing unit 3 performs an imaging process on the image data of the frame.

  In step S15, a video image (moving image) is output.

  In step S16, the video image output in step S15 is displayed on a display unit such as an LCD monitor 42.

  In step S17, it is determined whether or not the moving image shooting has been completed. Specifically, it is determined whether the photographer has operated the shutter button 13 for ending the moving image shooting. Here, when the moving image shooting is not finished, the flow returns to step S2 and the moving image shooting is continued.

<About multi-frame exposure (NR2)>
The imaging apparatus 1A can perform multiframe exposure as noise processing NR2 in moving image shooting at low illuminance.

  In this multi-frame exposure, noise processing is performed to increase the exposure time of the image sensor 16 by reducing the frame rate of moving image shooting under the control of the timing generator sensor driver 46. In this case, since the sensitivity at the time of shooting can be kept constant, that is, the amplifier gain of the AGC 22 of the signal processing unit 2 can be lowered, noise can be reduced. In multi-frame exposure, it is necessary to secure a large amount of exposure of the image sensor 16 at low illuminance to achieve proper exposure. For example, the charge storage time of the image sensor 16 is a plurality of frames with the aperture 44 opened. It will be set to the time over (multiframe exposure).

  The operation of the imaging apparatus 1A that can perform the two types of noise processing NR1 and NR2 as described above will be described below.

<Operation of Imaging Device 1A>
FIG. 5 is a flowchart showing the basic operation of the imaging apparatus 1A. This operation is executed by the camera control unit 40A.

  In steps S21 to S24, operations similar to those in steps S2 to S5 shown in the flowchart of FIG. 4 are performed.

  In step S25, it is determined whether or not moving image shooting is set. Specifically, it is determined whether or not the mode changeover switch 12 is selected in the moving image mode. If the moving image shooting is set, the process proceeds to step S26. If the moving image shooting is not set, the process proceeds to step S32.

  In step S26, it is determined whether the moving image NR is automatically set. That is, it is determined whether or not the NR mode selection switch 17 is set to “Auto”. If the moving picture NR is automatically set, the process proceeds to step S27a. If not set, the process proceeds to step S27b.

  In step S27a, it is determined whether the set gain in the AGC 22 is greater than a predetermined threshold value Ref1. Here, when the AGC gain corresponding to the photographing sensitivity is larger than the predetermined value Ref1, it is determined that the illuminance is low, and the process proceeds to step S28, and when it is equal to or smaller than the predetermined value Ref1, the process proceeds to step S32.

  In step S27b, as in step S27a, it is determined whether the set gain in AGC 22 is greater than a predetermined threshold value Ref1. If the AGC gain is greater than the predetermined value Ref1, the process proceeds to S29. If the AGC gain is less than the predetermined value Ref1, the process proceeds to step S32.

  By the operations in steps S27a and S27b described above, the cyclic filter process performed in step S30 is performed according to the condition related to the AGC gain (first imaging condition).

  In step S28, it is determined whether the focal length of the taking lens 10 is set to telephoto. This telephoto setting means that it is set to 85 mm or more, preferably 100 mm or more in terms of 135 mm, for example.

  In the case of the cyclic filter processing, the amount of blurring (flow amount) of the image can be controlled by the magnitude of the feedback coefficient. On the other hand, when performing multi-frame exposure at a telephoto position, the exposure time of each frame is long, so that image blurring is likely to occur, and the frame rate is also low, and image distortion due to blurring is likely to be noticeable.

  On the other hand, in the case of a wide angle of view that is not telephoto, subject blurring is less likely to occur, so multi-frame exposure that lowers the effective sensitivity increases the quality of each frame and improves image quality.

  For the reasons described above, it is determined in step S28 whether or not the telephoto setting has been made.

  In step S28, if the telephoto focal length is set, the process proceeds to step S30. If the telephoto focal distance is not set, the process proceeds to step S31.

  As a result, one of cyclic filter processing and multiframe exposure is selected according to the focal length, and the noise processing is switched.

  In step S29, it is determined whether the circulation filter process is manually selected as the noise process during moving image shooting. That is, it is determined whether or not the NR mode selection switch 17 is set to “NR1”. If cyclic filter processing is selected, the process proceeds to step S30. If multi-frame exposure is selected instead of cyclic filter processing, the process proceeds to step S31.

  In step S29 described above, one of cyclic filter processing and multi-frame exposure is selected according to an operation input to the NR mode selection switch 17. Thus, noise processing intended by the photographer can be performed.

  In step S30, a circulation filter process is performed.

  In step S31, the above-described multiframe exposure is performed.

  In steps S32 to S35, operations similar to those in steps S14 to S17 shown in the flowchart of FIG. 4 are performed.

  As described above, in the imaging apparatus 1A, since the feedback coefficient of the circulation filter unit is set based on the AF evaluation value that is control information related to shooting, an appropriate feedback coefficient can be easily determined in moving image shooting. In addition, since the circular filter process or the multi-frame exposure is selected according to the presence / absence of the telephoto setting, that is, the focal length of the photographing lens 10, it is possible to perform an optimal noise reduction process while preventing the occurrence of image distortion.

  In step S28, it is not essential to strictly switch between the cyclic filter processing (step S30) and the multiframe exposure (step S31) with the threshold focal length as a boundary, and hysteresis is provided. You may make it switch. For example, even if the focal length is gradually decreased from a focal length exceeding 85 mm and becomes a threshold focal length of 85 mm or less, noise control is performed to maintain the state of the circulating filter processing up to 70 mm, for example. As a result, even if the focal length fluctuates around the threshold focal length, the switching between the cyclic frame processing and the multiframe exposure does not frequently occur, and smooth noise removal processing can be performed.

  Further, the imaging apparatus 1A according to the first embodiment may perform the operation shown in the flowchart of FIG. 6 described below.

  In the flowchart shown in FIG. 6, the operation of step S36 is added to the flowchart shown in FIG.

  In this step S36, it is determined whether the fixing of the tripod has been detected. Specifically, it is determined whether or not the tripod detector 51 detects that the tripod is attached to the tripod hole 52. Here, when fixing of a tripod is detected, it progresses to step S31 and multiframe exposure is performed. On the other hand, when the fixing of the tripod is not detected, the process proceeds to step S30 and the circulation filter process is performed.

  Thereby, one of the circular filter processing and the multi-frame exposure is selected according to whether or not the tripod is connected.

  By determining whether or not a tripod is mounted as described above, even if the camera is set to telephoto, if the tripod is fixed, the cause of camera shake is eliminated. Noise removal can be performed.

Second Embodiment
The imaging device 1B according to the second embodiment of the present invention has a configuration similar to that of the imaging device 1A according to the first embodiment, as shown in FIGS. Is different.

  That is, in the circulation filter unit 26 of the imaging apparatus 1B, the correlation detection unit 261 shown in FIG. 3 has a feedback coefficient based not only on the AF evaluation value but also on the measurement value (detection value) of the camera shake sensor 49 as in the first embodiment. Is set.

  The operation of the circulation filter process (NR1) of the imaging apparatus 1B having such a circulation filter unit 26 will be described below.

<About circulation filter processing (NR1)>
FIG. 7 is a flowchart showing the operation of the circulating filter process in the imaging apparatus 1B. This operation is executed by the camera control unit 40B.

  In steps S41 to S47, operations similar to those in steps S1 to S7 shown in the flowchart of FIG. 4 are performed.

  In step S48, it is determined whether camera shake correction is on. Specifically, it is determined whether or not the camera shake correction switch 14 is pressed and the camera shake correction mode is set. If the camera shake correction is on, the process proceeds to step S49. If not, the process proceeds to step S50 and the feedback coefficient is set to zero. As described above, when the camera shake correction function is turned off (disabled), the circulation filter process is prohibited in step S50, thereby preventing the circulation filter process when the preceding and following frame correlation is low.

  In step S49, it is determined whether or not the detected value of the camera shake detected by the camera shake sensor 49 is smaller than a predetermined threshold value Ref3. At this time, since the AF evaluation value exceeds the predetermined value Ref2 in step S87, it can be estimated that the subject is stationary to some extent. Therefore, when the detected value of camera shake is smaller than the predetermined value Ref3, it can be determined that the frame correlation is high, and thus the feedback coefficient is increased in step S51. On the other hand, if the camera shake detection value is greater than or equal to the predetermined value Ref3, an error in feedback processing due to camera shake occurs, so the feedback coefficient is decreased in step S52.

  By the operations in steps S49 and S51 to S52 described above, the feedback coefficient is determined according to the condition (second imaging condition) related to the camera shake detection value (measurement information). As a result, an appropriate feedback coefficient can be easily determined.

  In steps S51 to S59, operations similar to those in steps S9 to S17 shown in the flowchart of FIG. 4 are performed.

  As described above, in the imaging apparatus 1B, the feedback coefficient of the circulation filter unit is set based on the camera shake detection value that is control information related to shooting. Therefore, an appropriate feedback coefficient can be easily determined in moving image shooting at low illuminance.

  In step S50 described above, it is not essential to set the feedback coefficient to 0, and the filtering process may be prohibited by turning off the function of the circulation filter unit 26.

<Third Embodiment>
The imaging device 1C according to the third embodiment of the present invention has a configuration similar to that of the imaging device 1A according to the first embodiment, as shown in FIGS. Is different.

  That is, in the circulation filter unit 26 of the imaging apparatus 1C, a feedback coefficient corresponding to the focal length of the photographing lens 10 is set in the correlation detection unit 261 shown in FIG.

  Since the occurrence level of camera shake varies depending on the focal length (angle of view) of the photographic lens 10, it is assumed that in the case of a long focal length, a slight blur of the subject due to camera shake occurs. If the feedback coefficient is increased when the correlation between the previous and subsequent frames is reduced due to the camera shake, the image is deteriorated. Therefore, as will be described later, in the imaging apparatus 1E, a feedback amplifier is set in accordance with the setting of a feedback coefficient according to the focal length at the time of shooting, for example, each focal length of less than 30 mm, 30 to 50 mm, and 50 to 100 mm in terms of 135 mm. The feedback coefficient of the unit 264 is set to 0.8, 0.5, and 0.2, and cyclic filter processing is performed. Thereby, an appropriate feedback coefficient can be easily determined.

  The operation of the circulating filter process (NR1) of the imaging apparatus 1C having such a circulating filter unit 26 will be described below.

<About circulation filter processing (NR1)>
FIG. 8 is a flowchart showing the operation of the circulating filter process in the imaging apparatus 1C. This operation is executed by the camera control unit 40C.

  In steps S61 to S65, operations similar to those in steps S2 to S6 shown in the flowchart of FIG. 4 are performed.

  In step S66, it is determined whether the focal length of the taking lens 10 is greater than 100 mm. If the focal length is greater than 100 mm, the process proceeds to step S73, and if it is 100 mm or less, the process proceeds to step S67.

  In step S67, it is determined whether the focal length of the taking lens 10 is greater than 50 mm. If the focal length is greater than 50 mm, the process proceeds to step S69, and if it is 50 mm or less, the process proceeds to step S68.

  In step S68, it is determined whether the focal length of the taking lens 10 is greater than 30 mm. If the focal length is greater than 30 mm, the process proceeds to step S70, and if it is 30 mm or less, the process proceeds to step S71.

  In step S69, the feedback coefficient of the feedback amplifier unit 264 is set to 0.2.

  In step S70, the feedback coefficient of the feedback amplifier unit 264 is set to 0.5.

  In step S71, the feedback coefficient of the feedback amplifier unit 264 is set to 0.8.

  Through the operations in steps S66 to S71 described above, the feedback coefficient is determined according to the condition regarding the focal length (second imaging condition). As a result, an appropriate feedback coefficient can be easily determined.

  In step S72, the same operation as step S12 shown in the flowchart of FIG. 4 is performed.

  In step S73, the feedback coefficient of the feedback amplifier unit 264 is set to zero. That is, the circulation filter process is prohibited. Thereby, it is possible to prevent the cyclic filter process when the frame correlation between the front and rear frames is low.

  In steps S74 to S77, operations similar to those in steps S14 to S17 shown in the flowchart of FIG. 4 are performed.

  As described above, in the imaging apparatus 1C, the feedback coefficient of the circulation filter unit is set based on the focal length, so that an appropriate feedback coefficient can be easily determined in moving image shooting.

  In the operation of the imaging apparatus 1C shown in the flowchart of FIG. 8, the feedback coefficient of the circulation filter unit 26 may be set to 0 while the focal length is being changed. Accordingly, since the circulation filter process is prohibited during the change of the focal length at which the correlation between the preceding and lower frames becomes low, it is possible to prevent the generation of a moving image with image disturbance due to the circulation filter process.

  Note that it is not essential to set the feedback coefficient to 0 during the change of the focal length, and the filtering process may be prohibited by turning off the function of the cyclic filter processing unit.

<Fourth embodiment>
The imaging device 1D according to the fourth embodiment of the present invention has a configuration similar to that of the imaging device 1A according to the first embodiment, as shown in FIGS. Is different.

  That is, in the circulation filter unit 26 of the image pickup apparatus 1D, a feedback coefficient corresponding to the shutter speed is set in the correlation detection unit 261 shown in FIG.

  Here, when the AF evaluation value exceeds a certain value, it can be determined that the subject is stationary to some extent. In this case, the feedback coefficient Control shall be performed. For example, SS is less than 1 / 250s, 1 / 250-1 / 100s, and 1 / 100-1 / 30s under each SS setting condition, the feedback amplifier unit 264 has a feedback coefficient of 0.2, 0.5, and It is set to 0.8 and appropriate cyclic filter processing is performed. Regarding the setting of the feedback coefficient, since the correlation between frame images becomes small when SS is short, that is, at high shutter speed, if the feedback coefficient is increased, an afterimage effect occurs, and an unnatural image is generated by cyclic filter processing. The feedback coefficient is set smaller as SS is shorter. On the other hand, for SS (exposure time) of 1/30 s or more, priority is given to multi-frame shooting, and the circulation filter processing is prohibited because there is a high possibility of causing camera shake.

  The operation of the circulation filter process (NR1) of the imaging apparatus 1D having such a circulation filter unit 26 will be described below.

<About circulation filter processing (NR1)>
FIG. 9 is a flowchart showing the operation of the circulating filter process in the imaging apparatus 1D. This operation is executed by the camera control unit 40D.

  In steps S81 to S86, operations similar to those in steps S2 to S7 shown in the flowchart of FIG. 4 are performed.

  In step S87, it is determined whether the calculated value of SS (shutter speed) is smaller than 1/30 s. Here, if it is smaller than 1/30 s, the process proceeds to step S88, and if it is 1/30 s or more, the process proceeds to step S94 where the cyclic filter process is not performed.

  Only when the condition that the calculated value of SS is smaller than 1/30 s is satisfied by the operation of step S87, in other words, the circulation filter unit 26 operates (active) according to the condition relating to the exposure time (first imaging condition). Noise processing is performed. Thereby, a circulation filter process can be performed appropriately.

  In step S88, it is determined whether the calculated value of SS is smaller than 1 / 250s. If it is smaller than 1 / 250s, the process proceeds to step S90, and if it is 1 / 250s or more, the process proceeds to step S89.

  In step S89, it is determined whether the calculated value of SS is smaller than 1/100 s. If it is less than 1/100 s, the process proceeds to step S91. If it is 1/100 s or more, the process proceeds to step S92.

  In step S90, the feedback coefficient of the feedback amplifier unit 264 is set to 0.2.

  In step S91, the feedback coefficient of the feedback amplifier unit 264 is set to 0.5.

  In step S92, the feedback coefficient of the feedback amplifier unit 264 is set to 0.8.

  In step S93, the same operation as step S12 shown in the flowchart of FIG. 4 is performed.

  In step S94, the feedback coefficient of the feedback amplifier unit 264 is set to zero. That is, when the set value of SS is equal to or longer than 1/30 s of one frame time, priority is given to multi-frame exposure, and cyclic filter processing is prohibited.

  In steps S95 to S98, operations similar to those in steps S14 to S17 shown in the flowchart of FIG. 4 are performed.

  As described above, in the imaging apparatus 1D, the feedback coefficient of the circulation filter unit is set based on the shutter speed, which is control information related to shooting, so that an appropriate feedback coefficient can be easily determined in moving image shooting.

<Fifth Embodiment>
An imaging apparatus 1E according to the fifth embodiment of the present invention has a configuration similar to that of the imaging apparatus 1A according to the first embodiment as shown in FIGS. 1 and 2, but the camera control unit is different. Yes.

  That is, in the camera control unit 40E of the imaging apparatus 1E, a program for performing an operation of switching between the cyclic filter processing based on the AF evaluation value and the multiframe exposure described in the first embodiment in accordance with the movement of the subject is stored in the memory. Has been. This operation will be described below.

<Operation of Imaging Device 1E>
FIG. 10 is a flowchart showing the basic operation of the imaging apparatus 1E. This operation is executed by the camera control unit 40E.

  In steps S101 to S107a and S107b, operations similar to those in steps S21 to S27a and S27b shown in the flowchart of FIG. 5 are performed.

  In step S108, it is determined whether a moving object has been detected. Specifically, the distance calculation / motion detection unit 36 determines whether or not movement of a main subject such as a person has been detected.

  The reason for determining the detection of a moving object is to select a circulation filter process that is strong against the movement of the subject in the case of a moving object, and to select multi-frame exposure if it is not a moving object, that is, a stationary subject. As a result, it is possible to reduce noise caused by blurring due to movement of the subject during moving image shooting.

  In this step S108, when a moving body is detected, it progresses to step S110, and when a moving body is not detected, it progresses to step S111. Thereby, one of the circular filter processing and the multi-frame exposure is selected according to the movement of the subject.

  In steps S109 to S115, operations similar to those in steps S29 to S35 shown in the flowchart of FIG. 5 are performed.

  As described above, in the imaging apparatus 1E, the circular filter process or the multi-frame exposure is selected according to the movement of the subject (detection of moving object), so that the optimum noise reduction process can be performed while preventing the occurrence of image disturbance. It becomes. Further, since the feedback coefficient of the circulation filter unit is set based on the AF evaluation value, an appropriate feedback coefficient can be easily determined in moving image shooting as in the first embodiment.

<Sixth Embodiment>
An imaging apparatus 1F according to the sixth embodiment of the present invention has a configuration similar to that of the imaging apparatus 1A of the first embodiment as shown in FIGS. 1 and 2, but the camera control unit is different. Yes.

  That is, in the camera control unit 40F of the imaging apparatus 1F, a program for appropriately selecting two types of noise reduction processing depending on the shooting mode is stored in the memory.

  For the two types of noise reduction processing, the cyclic filter processing based on the AF evaluation value described in the first embodiment and the noise reduction processing by multi-frame exposure can be selected. In the imaging apparatus 1F, the above two types of noise processing can be switched in accordance with the setting state of the shooting mode (sport mode, night view mode, macro mode).

  With respect to the above-described shooting modes, for example, on the menu screen displayed on the LCD monitor 42, each shooting mode of “sport mode”, “night view mode”, and “macro mode” is operated by the photographer's operation on the frame advance / zoom switch 15. Can be set. That is, the photographer can selectively designate a plurality of shooting modes by operating the frame advance / zoom switch 15.

<Operation of Imaging Device 1F>
FIG. 11 is a flowchart showing the basic operation of the imaging apparatus 1F. This operation is executed by the camera control unit 40F.

  In steps S121 to S127a and S127b, operations similar to those in steps S21 to S27a and S27b shown in the flowchart of FIG. 5 are performed.

  In step S128, it is determined whether the sport mode is set. Specifically, it is determined whether or not “sport mode” is selected on the menu screen displayed on the LCD monitor 42.

  Here, the setting of the sport mode is determined because the high-speed shutter is prioritized when the sport mode is selected, so that the exposure time is determined by the cyclic filter process without performing multi-frame exposure over a plurality of frames. This is for noise removal.

  If the sport mode is set in step S128, the process proceeds to step S132, and if the sport mode is not set, the process proceeds to step S129.

  In step S129, it is determined whether the night view mode is set. Specifically, it is determined whether or not “night scene mode” is selected on the menu screen displayed on the LCD monitor 42.

  Here, the setting of the night view mode is generally determined when a night view mode in which camera shake is likely to occur is selected, since a tripod is assumed to be used, noise removal is performed by multi-frame exposure instead of cyclic filter processing. This is because it is preferable.

  In step S129, if the night view mode is set, the process proceeds to step S133. If the night view mode is not set, the process proceeds to step S130.

  In step S130, it is determined whether the macro mode is set. Specifically, it is determined whether “macro mode” is selected on the menu screen displayed on the LCD monitor 42.

  Here, the setting of the macro mode is generally determined when a tripod is used when a macro mode that is likely to cause camera shake is selected. Therefore, noise removal is performed by multi-frame exposure instead of cyclic filter processing. This is because it is preferable.

  In step S130, if the macro mode is set, the process proceeds to step S133. If the macro mode is not set, the process proceeds to step S132.

  Through the operations in steps S128 to S130 described above, one of cyclic filter processing and multiframe exposure is selected according to the designated photographing mode.

  In steps S131 to S137, operations similar to those in steps S29 to S35 shown in the flowchart of FIG. 5 are performed.

  As described above, in the imaging apparatus 1F, the circular filter process or the multi-frame exposure is selected according to the set shooting mode, so that it is possible to perform an optimal noise reduction process while preventing the occurrence of image distortion. Further, since the feedback coefficient of the circulation filter unit is set based on the AF evaluation value, an appropriate feedback coefficient can be easily determined in moving image shooting as in the first embodiment.

<Modification>
In each of the above-described embodiments, it is not essential to perform the cyclic filter processing when the AGC setting gain satisfies a low-illuminance shooting condition (first shooting condition) that is a predetermined value (Ref1) or more, and subject brightness In the case of a low-illuminance shooting condition in which is less than or equal to a predetermined value, the circulation filter process may be performed.

  In the fifth and sixth embodiments described above, it is not essential to set a feedback coefficient based on the AF evaluation value described in the first embodiment. The camera shake detection value described in the second embodiment, The feedback coefficient may be set based on the focal length described in the third embodiment and the shutter speed described in the fourth embodiment.

It is a figure which shows the principal part structure of 1 A of imaging devices which concern on 1st Embodiment of this invention. It is a figure which shows the functional block of 1 A of imaging devices. It is a figure for demonstrating the circulation filter process in 1 A of imaging devices. It is a flowchart which shows the operation | movement of the circulation filter process in 1 A of imaging devices. It is a flowchart which shows basic operation | movement of 1 A of imaging devices. It is a flowchart which shows other operation | movement of 1 A of imaging devices. It is a flowchart which shows the operation | movement of the circulation filter process in the imaging device 1B which concerns on 2nd Embodiment of this invention. It is a flowchart which shows operation | movement of the circulation filter process in 1 C of imaging devices which concern on 3rd Embodiment of this invention. It is a flowchart which shows the operation | movement of the circulation filter process in imaging device 1D which concerns on 4th Embodiment of this invention. It is a flowchart which shows the basic operation | movement of the imaging device 1E which concerns on 5th Embodiment of this invention. It is a flowchart which shows the basic operation | movement of the imaging device 1F which concerns on 6th Embodiment of this invention. It is a block diagram for demonstrating the circulation filter process which concerns on a prior art.

Explanation of symbols

1A to 1F Imaging device 2 Signal processing unit 3 Image processing unit 10 Shooting lens 14 Camera shake correction switch 15 Frame advance / zoom switch 16 Image sensor 17 NR (noise reduction) mode selection switch 26, 82 Circulating filter unit 36 Distance calculation / Motion detection units 40A to 40F Camera control unit 42 LCD monitor 48 Camera shake correction actuator driver 49 Camera shake sensor 51 Tripod detection unit 261 Correlation detection unit

Claims (8)

An imaging device,
(a) moving image capturing means for capturing an image of a subject and sequentially generating frames constituting the moving image;
(b) cyclic filter means for performing noise processing by multiplying a frame image sequentially generated by the moving image photographing means by a feedback coefficient;
(c) processing means for activating the circulation filter means to perform the noise processing according to a first imaging condition;
With
The circulation filter means includes
(b-1) determining means for determining the feedback coefficient according to the second imaging condition;
An imaging device comprising: The imaging device according to claim 1,
The imaging apparatus according to claim 1, wherein the first imaging condition is a condition related to an exposure time. In the imaging device according to claim 1 or 2,
(d) evaluation information generating means for generating evaluation information for evaluating the in-focus state based on the image data of the frame;
Further comprising
The imaging apparatus according to claim 2, wherein the second imaging condition is a condition related to the evaluation information. The imaging apparatus according to any one of claims 1 to 3,
(e) focal length changing means for changing the focal length of the taking lens;
Further comprising
The imaging apparatus according to claim 2, wherein the second imaging condition is a condition related to the focal length. The imaging apparatus according to claim 4,
The processing means includes
(c-1) means for prohibiting the noise processing during the change of the focal length by the focal length changing means,
An imaging device comprising: The imaging device according to any one of claims 1 to 5,
(f) measurement information generating means for measuring the amount of camera shake related to the imaging device and generating measurement information;
Further comprising
The imaging apparatus according to claim 2, wherein the second imaging condition is a condition relating to the measurement information. The imaging device according to claim 6,
(g) camera shake correction means for performing camera shake correction based on the measurement information;
Further comprising
The processing means includes
(c-2) means for prohibiting the noise processing when the camera shake correction means is disabled and the camera shake correction is not performed,
An imaging device comprising: A video noise processing method,
(a) a moving image shooting process for capturing an image of a subject and sequentially generating frames constituting the moving image;
(b) a cyclic filter step of performing noise processing by multiplying a frame image sequentially generated in the moving image shooting step by a feedback coefficient;
(c) when the first imaging condition is satisfied, a processing step for performing the noise processing;
With
The moving image noise processing method, wherein the feedback coefficient is set based on a second shooting condition.

Images