JPH07107366A - Image processor - Google Patents

Abstract

PURPOSE:To prevent an image disturbed instead due to a correction from being photographed and recorded by properly performing the correction of the blurring of an image. CONSTITUTION:A motion vector is detected from an image signal (S202) and the correction values HOSEIX and HOSEIY of the shake of an image are calculated based on the integrated value of the detected motion vector (S208). When the correction value HOSEIX or HOSEIY is larger than a threshold TH1 or TH2, correction impossible state is judged because the shake of the image is large and a correction is interrupted (S210, S214). During the interruption of the correction, the storage location on a memory of a correction object screen is moved to a location which is suitable for the correction.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus,
In particular, the present invention relates to an image processing device that corrects a blur of an image captured by a video camera.

[0002]

2. Description of the Related Art In recent years, a camera-integrated VTR has been reduced in size by the rear focus of a lens, the parts used have been downsized, and high-density mounting has been achieved. As the magnification increases, the frequency of handheld and high-magnification photography without using a tripod is increasing.

On the other hand, the photographed image is recorded as an image accompanied by a large screen blur even with slight camera shake during use.

There is known an optical device having a function of preventing the image blur caused by the camera shake or the like. As an example of the optical device, a correction optical system is movably supported by a gimbal mechanism to correct the optical shake. An inertial pendulum type image blur prevention device (U.S. Pat. Nos. 2959088 and 2829557, etc.) for preventing image blur due to its inertia has been conventionally known.

Further, a variable apex angle type image blur preventing device for preventing image blur by arranging a variable apex angle prism in front of the front lens of the video lens and controlling the apex angle of the prism by a sensor output for detecting vibration, The image signal output from the image sensor is stored in an image memory, etc., the motion is detected from the information, and the read address of the image memory is shifted according to the amount of motion to correct the image blur. A preventive device (Japanese Patent Laid-Open No. 61-248681) has already been proposed.

Particularly, the pure electronic image blur prevention device does not require a special mechanical mechanism for correcting the image blur, and due to the rapid progress of semiconductor technology, a large-scale electric circuit can be accommodated in an extremely small package to reduce the size and weight. In recent years, it has attracted attention because of its low cost.

[0007]

However, in the conventional pure electronic image blur prevention device, the correction limit area is automatically determined by the image sensor and the memory, and therefore, when the blur correction value exceeds the correction limit area. However, there has been a problem that further correction is impossible and the image is disturbed.

The present invention has been made to solve this problem, and it is possible to appropriately correct image blur, and to prevent image capturing and recording of an image that is disturbed by the correction. The purpose is to provide a device.

[0009]

In order to achieve the above object, the present invention provides an image processing apparatus which detects a motion vector in an image signal and corrects an image blur in real time. Detecting means for detecting the motion vector between the images by performing the correlation calculation of, the calculating means for adding the motion vectors obtained by the detecting means to obtain the absolute deviation from the reference point of the image, and the absolute deviation When the state in which the absolute deviation is larger than the predetermined correction amount continues for a predetermined time or more, it is determined that the blurring of the image exceeds the correctable range, An interruption means for interrupting the shake correction is provided.

Further, it is preferable to include a storage means for storing the image signal, and to move a storage position of the surface to be corrected in the storage means to a position suitable for correction during the correction interruption.

In order to achieve the same object, the present invention is, in the image processing apparatus, in place of the interruption means, when the state in which the absolute deviation is larger than a predetermined correction amount continues for a predetermined time or more, the image blurring is caused. Is provided with a warning means for warning the photographer that it has exceeded the correctable range.

[0012]

When the state in which the absolute deviation of the image from the reference point is larger than the predetermined correction amount continues for the predetermined time or longer, the correction of the image blur is interrupted or the photographer is warned.

During the interruption of the image blur correction, the storage position of the correction target screen is moved to a position suitable for the correction.

[0014]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram of the essential parts of a camera-integrated VTR (video tape recorder) incorporating an image processing apparatus according to the first embodiment of the present invention.

In FIG. 1, 100 is a focus lens group for normal focusing, 101 is a zoom lens group that changes the focal length, 102 is a lens group of a correction system that corrects the focus plane shift that has changed due to the operation of the zoom lens, and 103. Is an aperture, 104 is an image pickup device including, for example, a two-dimensional CCD, 16 is a sample hold (S / H) circuit for sampling an electric signal obtained from the image pickup device, and 18
Is an automatic gain control (AGC) circuit for keeping the signal level substantially constant, and 20 is an analog-
The digital (A / D) converter 22 delays the color difference line sequential signal from the image sensor by one or two horizontal scanning periods (hereinafter, the horizontal scanning period is referred to as “H”), and a 1H delay signal,
A circuit for outputting a signal obtained by adding a 0H delay signal and a 2H delay signal, a circuit 24 for generating a color signal C, a low-pass filter (LPF) 26 for removing a color signal mixed in a luminance signal Y, and 28 a An enhancer for emphasizing high-frequency components, 30 a gamma correction circuit, 32 a two-dimensional bandpass filter (BPF), 34 a motion vector detection circuit for detecting a motion vector from an image signal, 36 and 38 field memories, and 120 various types. A logic control device for signal processing, 42 a memory read control circuit, 40 an electronic zoom circuit for electronically enlarging or reducing an image read from the memory, 44 a digital-analog (D / A) converter, 48
Are signal output terminals for the color signal C and the luminance signal Y.

Next, the operation of the apparatus shown in FIG. 1 will be described. The subject 10 includes lens groups 100, 101, 102 and a diaphragm 103.
An image is formed on the image sensor 104 through the optical path and is photoelectrically converted. The S / H circuit 16 holds the output signal of the image sensor 104, and the subsequent AGC circuit 18 automatically controls the gain so that the output signal level becomes substantially constant. A
The / D converter 20 performs analog-digital conversion on the output signal of the AGC circuit 18. 2 horizontal scanning period delay circuit 22
Separates the color difference line sequential signal converted into a digital signal into a 1H delay signal and a (0H + 2H) delay signal, and a luminance signal processing unit (26 and later) and a color signal processing unit (24 and later), respectively.
Send to. A color signal is generated in the color signal processing circuit 24 and written in the field memory 38.

On the other hand, the signal sent to the luminance signal processing section (from 26 onward) is first input to the LPF 26. LPF26
Removes the carrier component from the color difference line sequential signal and performs luminance signal separation. The enhancer 28 performs a process of emphasizing a high frequency component such as an edge of a subject in order to improve image quality. Usually, a secondary differential signal of the video signal is added to the original signal. The gamma correction circuit 30 performs gamma correction to prevent saturation in the highlight portion and widen the dynamic range. BP
F32 extracts a spatial frequency component effective for detecting the motion vector. Generally, the low frequency component and the high frequency component of the image signal are not suitable for detecting the motion vector,
Preliminarily removed by BPF32. In this embodiment, only the sign bit of the output of the BPF 32 is output. This means that the luminance signal is binarized using the DC level as a threshold. Therefore, the luminance signal after the BPF 32 is a 1-bit binary signal. The motion vector detection circuit 34 is a circuit for detecting a motion vector by a matching calculation, and in this embodiment, it is necessary to adopt a detection method capable of real-time processing. Memory 36
Is a delay circuit that delays the luminance signal output from the BPF 32 by a predetermined time (one field time in this embodiment). The luminance signal of one field before is stored and compared with the luminance signal of the current field to obtain an image. Direction of displacement of
Enables matching operation to detect size. The logic control circuit 120 calculates the deviation from the reference position of the image at that moment from the output signal (each horizontal and vertical component of the motion vector) from the motion vector detection circuit 34 according to the flowchart shown in FIG. To do. The memory reading control circuit 42 controls the reading position of the field memory 38 so that the deviation position calculated by the logic control device 120 becomes the center, that is, the motion vector is canceled, and the electronic zoom circuit 40 linearly controls. It is enlarged and reduced to a desired size by interpolation and converted to a normal angle of view. The image signal created in this way is converted into an analog signal by the D / A converter as a signal in which image blur is corrected,
The signal is output from the signal output terminal 48.

FIG. 2 is a flow chart showing the processing contents in the logic control device 120 of FIG. 1, and the operation of the logic control device 120 will be described in detail with reference to this.

In FIG. 2, in step S202, output signals from the motion vector detection circuit 34 (horizontal and vertical components of the motion vector at a predetermined screen position) are fetched for each field.

In step S204, step S202
The motion vector detected at the predetermined screen position captured in step 3 is integrated to obtain the deviation from the reference position of the screen to obtain the image blur correction signal. Also, here, the reliability of each motion vector is also evaluated and reflected in the integration operation,
Seeking a more accurate deviation.

In step S206, step S20
2, based on the spatial distribution of the motion vector and the motion vector integrated value obtained in S204, or temporal variation,
The area in which the motion exists is determined, and the correction target area is determined.

In step S208, step S206
The motion vector integral value in the correction target area determined in step S3 is extracted to obtain the final image blur correction value.

In step S210, step S208
On the basis of the image blur correction value obtained in step S1, it is determined whether or not the current image blur exceeds the correctable area on the field memory 38, for example. If it is determined that the correction is possible, the correction is interrupted (step S214). If not, the process proceeds to step S212 to perform the image blur correction. Step S
The routine of 210 and the routine of step S214 form the essential part of the present invention, and will be described in detail later.

In step S212, step S210
The image blur correction value obtained in step S1 is converted into a read address of the field memory 38, and a command for actually controlling the memory is issued. Further, a command for giving a desired enlargement or reduction ratio to the electronic zoom circuit 40 is also issued.

Next, the contents of the image blur determination routine of step S210 and the correction interruption routine of step S214 will be described with reference to FIGS.

FIG. 3 is a flow chart showing the details of the image blur determination routine.

In step S302, step S in FIG.
X-direction shake correction value HOSEIX, Y obtained in 208
It is determined whether or not the direction blur correction value HOSEIY exceeds the predetermined threshold values TH1 and TH2. Here, the correction values HOSEIX and HOSEYY are, for example, as shown in FIG. 5, obtained by averaging the motion vector integral values in the correction target area determined in step S206 of FIG. Further, the threshold values TH1 and TH2 are values corresponding to offsets in the X direction and the Y direction from the center, that is, image blur, when cutting an image with a predetermined reduction ratio from the field memory 38 as shown in FIG. Set the limit value that can be corrected.

In step S302, the shake correction value HOSEI
X or HOSEIY is the predetermined threshold value TH1 or TH
When it exceeds 2, the time counter is incremented by 1 (step S304). This time counter
It is a counter that counts how many times in the past the shake correction value has exceeded a predetermined threshold value. On the other hand, when the shake correction value does not exceed the predetermined threshold value, the time counter is reset (step S306).

In step S308, it is finally determined whether or not image blur correction is possible, specifically, whether or not the count value of the time counter exceeds a predetermined number TH3. That is, when the count value exceeds the predetermined number of times TH3,
The correction has already largely exceeded the correction limit range (see FIG. 6), and it is judged that further correction is impossible. If not, it is determined that the correction is still possible (normal image blur).

Further, in the present embodiment, when the image blur determination routine (step S210) determines that the image cannot be corrected, the image is not corrected and the image is corrected by the following correction interruption routine (step S214). Is suspended for a certain period of time and is waiting.

FIG. 4 is a flow chart showing the details of the processing in step S214.

In step S402, after the image blur determination routine (S210) determines that the image blur is large,
As shown in FIG. 7, the read address of the feed memory 38 is controlled to move the corrected image to the center of the screen over a predetermined time. Here, the control is stabilized by not performing any other shake correction control during movement.

As described above, in the present embodiment, even when an image blur that exceeds the image blur correction area occurs, it is surely identified, and in that case, the blur correction control is not performed. By prohibiting the correction control within the predetermined time and moving the correction image to the center of the screen, the photographer does not feel uncomfortable and the disturbance of the correction image can be minimized.

FIG. 9 is a block diagram of the essential parts of a camera-integrated VTR according to the second embodiment of the present invention.

The configuration of FIG. 9 has a warning generation circuit 130, an electric viewfinder (hereinafter referred to as “EVF”) 134 for displaying an image being captured or a reproduced image, and an EV.
The configuration is different from that of FIG. 1 in that a display circuit 132 for controlling the display of F is provided, but otherwise the configuration is the same as that of FIG.

The warning generation circuit 130 is used in the logic control unit 12.
0 is a circuit for displaying a warning on the EVF 134 when it is determined that the image blur is large.

FIG. 10 shows the logic control circuit 1 in this embodiment.
20 is a flowchart showing the processing contents in step 20. Step S214 of the flowchart of FIG.
14a. The steps other than step S214a are the same as those in FIG.

In the present embodiment, when it is determined that the image blur is large and the image cannot be corrected, a command is output to the warning generation circuit 130 to display a warning display as shown in FIG. When the warning generation circuit 130 receives this instruction, it controls the EVF display circuit 132 to control the EV.
A warning is displayed in F134.

According to the present embodiment, even when an image blur that exceeds the image blur correction area occurs, it is surely identified, and in that case, a warning is displayed on the EVF to alert the photographer. It is possible to prevent problems such as shooting an image with a lot of camera shake.

In the above-described embodiment, when the final blur correction value is obtained, the average of the motion vector integral values in the correction target area is averaged (see FIG. 5), but the present invention is not limited to this and, for example, As shown in FIG. 8, the median (median value) of the motion vector in the correction target area may be taken.

In the second embodiment described above, the EVF
Although the warning is displayed in 134, the present invention is not limited to this, and a light emitting diode or the like for warning may be separately turned on, for example.

[0043]

As described above in detail, according to the image processing apparatus of the first aspect, when the state where the absolute deviation of the image from the reference point is larger than the predetermined correction amount continues for a predetermined time or more, the image blur correction is performed. Since the process is interrupted, it is possible to prevent an image which is disturbed by the correction from being recorded.

According to the image processing apparatus of the second aspect, since the storage position of the correction target screen is moved to a position suitable for the correction while the image blur correction is suspended, the photographer does not feel uncomfortable.
It is possible to minimize the disturbance of the corrected image.

According to the image processing apparatus of the third aspect, when the state in which the absolute deviation of the image from the reference point is larger than the predetermined correction amount continues for a predetermined time or more, a warning is given to the photographer. It is possible to prevent the shooting of many images.

[Brief description of drawings]

FIG. 1 is a camera-integrated VT according to a first embodiment of the present invention.
It is a block diagram which shows the structure of the principal part of R.

FIG. 2 is a flowchart showing processing contents in the logic control device of FIG.

FIG. 3 is a flowchart showing a part of the processing of FIG. 2 in detail.

FIG. 4 is a flowchart showing a part of the processing of FIG. 2 in detail.

FIG. 5 is a diagram for explaining a method of calculating correction values (HOSEIX, HOSEYY).

FIG. 6 is a diagram for explaining a correction limit region.

FIG. 7 is a diagram for explaining movement of a correction screen.

FIG. 8 is a diagram for explaining a method of calculating correction values (HOSEIX, HOSEII).

FIG. 9 is a camera-integrated VT according to a second embodiment of the present invention.
It is a block diagram which shows the structure of the principal part of R.

10 is a flowchart showing processing contents in the logic control device of FIG.

FIG. 11 is a diagram showing an example of a warning display.

[Explanation of symbols]

 32 band pass filter 34 motion vector detection circuit 36, 38 field memory 40 electronic zoom circuit 42 memory reading circuit 120 logic control device

Claims (3)

[Claims] 1. A motion vector is detected from an image signal,
In an image processing device that corrects image blur in real time,
Detecting means for detecting a motion vector between the images by performing correlation calculation between the images which are continuous in time series, and operation for adding the motion vector obtained by the detecting means to obtain an absolute deviation from the reference point of the image. Means, comparing means for comparing the absolute deviation with a predetermined correction amount, and when the state in which the absolute deviation is larger than the predetermined correction amount continues for a predetermined time or more, it is determined that the image blur exceeds the correctable range. However, the image processing apparatus is provided with an interruption unit that interrupts the correction of the image blur. 2. A storage unit for storing an image signal, wherein the storage position of the correction target screen in the storage unit is moved to a position suitable for correction during the correction interruption. Image processing device. 3. A motion vector is detected from the image signal,
In an image processing apparatus having a correction unit that corrects image blur in real time, a detection unit that detects a motion vector between images by performing a correlation calculation between images that are continuous in time series, and a detection unit that is obtained by the detection unit. The motion vector is added to calculate the absolute deviation from the reference point of the image, the comparing means for comparing the absolute deviation with a predetermined correction amount, and the absolute deviation is larger than the predetermined correction amount. An image processing apparatus comprising: a warning unit that warns a photographer when it is determined that an image blur exceeds a correctable range when it continues for a predetermined time or more.