JPH07115583A - Video camera with hand-shake correction device - Google Patents

Abstract

PURPOSE:To detect the amount of hand-shake more accurately in which the resultant motion vector is more accurately obtained. CONSTITUTION:When a microcomputer 26 discriminates the presence of a valid area in a detection area, the microcomputer 26 calculates the degree of divergence. The divergence coefficient Hk is set stepwise in response to the degree of divergence, and the resultant motion vector is determined by calculating the arithmetic mean of partial motion vectors of a valid area XHk. When the divergence is a predetermined value or over, a minimum motion vector is selected from four partial motion vectors and the selected motion vector is used for the resultant motion vector. When no valid area is in existence in the detection area, the coefficient smaller than the resultant motion vector of the preceding field X1 is used as the resultant motion vector of a current field.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video camera having a camera shake correction device, and more particularly to a video camera having a camera shake correction device which is used, for example, as a VTR with a built-in camera for consumer use.

[0002]

2. Description of the Related Art An example of a method for detecting a shake component of an image pickup apparatus was announced by Matsushita Electric Industrial Co., Ltd. at the 20th Imaging Engineering Conference in 1989. This method
JP-A-61-201581 [H04N 7/137]
The motion vector obtained from the representative point matching method described in the publication is used to detect the shake component of the imaging device from the image information. In this presentation, camera shake correction is performed based on the motion vector obtained from the image information.

That is, in this case, four detection areas of the screen are arranged, and four partial motion vectors are obtained from one screen. Then, the arithmetic mean of the partial motion vectors of these four detection areas is used as the overall motion vector of the screen,
Of the four partial motion vectors, the camera shake is corrected by using the arithmetic mean of the middle two partial motion vectors as the overall motion vector of the screen.

[0004]

However, when there is a moving object in two areas of the four detection areas, such as the left half or the right half of the screen, that is, an object that moves only in a part of the screen. Is present, then
Due to the moving object, the entire motion vector cannot be accurately detected, and the amount of camera shake may be erroneously detected.

Therefore, a main object of the present invention is to provide a video camera having a camera shake correction device, which can detect a camera shake amount more accurately.

[0006]

SUMMARY OF THE INVENTION The present invention is a video camera having a camera shake correction device for obtaining an overall motion vector by using a partial motion vector of each of a plurality of detection areas formed in a screen. An averaging means for obtaining an average vector which is the average of the partial motion vectors, and a divergence calculation for obtaining the divergence by adding the absolute values of the differences between each partial motion vector and the average vector and dividing the addition result by the number of addition A video camera having a camera shake correction device, comprising: a means, a comparing means for comparing the divergence degree with a threshold value, and an overall motion vector detecting means for detecting an overall motion vector by a different method according to a comparison result by the comparing means.

[0007]

The divergence calculating means calculates the divergence using the average vector calculated by the averaging means and the partial motion vector of each detection area. A large divergence indicates that there are large variations in each partial motion vector. This tendency tends to occur when there is a moving object in only a part of the screen. on the other hand,
The small divergence indicates that the variation of each partial motion vector is small. This tendency occurs when there is no moving object on the screen or when there is a moving object on the entire screen.

Therefore, if the overall motion vector is obtained by using the partial motion vector when the degree of divergence is large, an unreliable overall motion vector is obtained. On the other hand, when the overall motion vector is obtained using the partial motion vector when the divergence is small, a highly reliable overall motion vector is obtained. Therefore, when the degree of divergence is large, the average of the partial motion vectors is multiplied by a small coefficient to reduce the value of the overall motion vector. When the divergence is small, for example, the average of the partial motion vectors is multiplied by a large coefficient to increase the value of the overall motion vector. That is, the global motion vector when the divergence is large is used as little as possible, and the global motion vector when the divergence is small is used as much as possible. The degree of divergence is determined by the comparison means, and the overall motion vector detection means detects the overall motion vector according to the comparison result.

[0009]

According to the present invention, by calculating the divergence, the state of the screen is grasped and the whole motion vector is detected by a method according to the divergence, so that the whole motion vector can be detected more accurately. Therefore, the amount of camera shake can be detected more accurately. The above-mentioned objects, other objects, features and advantages of the present invention will become more apparent from the following detailed description of the embodiments with reference to the drawings.

[0010]

DESCRIPTION OF THE PREFERRED EMBODIMENTS A video camera 10 of this embodiment shown in FIG.
Includes a solid-state image sensor 12 such as a CCD that converts an optical signal from a subject (not shown) input from the lens 14 into an electric signal. The electric signal from the solid-state image sensor 12 is input to the camera circuit 16. As is well known, the camera circuit 16 includes a sample hold circuit, and includes the solid-state image sensor 1
Sample and hold the electrical signal from 2. The level of the sampled and held electric signal is adjusted by the AGC, and the synchronizing signal is further added by the synchronizing signal adding circuit. In this way, the camera circuit 16 converts the image signal from the solid-state image sensor 12 into an analog video signal. This analog video signal is
It is converted into a digital video signal by the / D converter 18. The digital video signal is supplied to the motion detection circuit 20. As the motion detection circuit 20, for example, an LSI "L7A0948" manufactured by Sanyo Electric Co., Ltd. is used.
Under the control of the memory control circuit 22 included in the same LSI configuring the motion detection circuit 20, the digital video signals are field-sequentially written in the field memory 24.

The motion detection circuit 20 gives the highest degree of correlation (minimum correlation value) to each of the four detection areas A, B, C and D shown in FIG. 3 by using, for example, a well-known representative point matching method. The positions of one point and four points around it and the respective correlation values are calculated. The position data and the correlation value data from the motion detection circuit 20 are stored in the microcomputer 26.
Given to.

That is, referring to FIG. 2, motion detection circuit 20 includes an input end 28 for receiving the digital video signal from A / D converter 18, and the digital video signal input from this input end 28 is filtered. It is given to the representative point memory 32 and the subtraction circuit 34 through 30. The filter 30 is a kind of digital low-pass filter,
It is used to improve the S / N ratio and secure sufficient detection accuracy with a small number of representative points. The representative point memory 32 extracts a plurality of representative points within the range of each of the detection areas A to D shown in FIG. 3 (in this embodiment, 30 of each of the detection areas A to D is extracted).
Therefore, 30 representative points are divided, and the position data and luminance data are stored. Each detection area 42 (FIG. 4) formed by dividing into 30 is composed of, for example, 32 pixels × 16 rows.

The subtraction circuit 34 receives the luminance data of the representative point of the previous field given from the representative point memory 32 and the input terminal 2.
The luminance data of all the pixels of the current field given from 8 is subtracted, and the absolute value is taken. That is, the brightness difference between the brightness data of the current field and the brightness data of the previous field is obtained. The obtained brightness difference is given to the cumulative addition circuit 36. The cumulative addition circuit 36 cumulatively adds (30 in this embodiment) the brightness differences obtained for the pixels at the same position in the respective detection areas 42 in the same detection area, and outputs it as correlation value data. The correlation value data is given to the arithmetic circuit 38,
The arithmetic circuit 38 calculates the minimum correlation value and the average correlation value for each detection area A-D, and also obtains the position data of the pixel indicating the minimum correlation value for each detection area AD. The minimum correlation value, the average correlation value and the position data thus obtained are given from the output end 40 to the microcomputer 26. However, such calculation of the correlation value is executed by the above-described LSI "L7A0948".

Then, in the microcomputer 26,
Based on the position data and the correlation value data, the motion vector of the entire screen, that is, the image field 44 (FIG. 3) (hereinafter, simply referred to as “whole motion vector”) is calculated. First, based on the position data of the pixel having the minimum correlation value, the shift of the pixel having the minimum correlation value with respect to the representative point is obtained, and the shift is set as the partial motion vector. In order to improve the detection accuracy of the partial motion vector, interpolation interpolation is performed using the correlation values of the four pixels surrounding the pixel having the minimum correlation value,
Compute the position data for the pixel with the smallest correlation value.

Then, the microcomputer 26 determines whether a value obtained by dividing the average correlation value by the minimum correlation value is larger than a certain threshold value, whether the average correlation value is a predetermined value or more, and a slope X described later. Whether or not it is equal to or more than a predetermined value P is detected for each detection area A-D, and whether the partial motion vector from each detection area A-D is reliable without being erroneously detected by a moving object other than camera shake That is, it is determined whether or not each detection area A-D is a valid area. Here, the microcomputer 26 detects whether or not an inclination X, which will be described later, is a predetermined value P or more for each of the detection areas A to D, as one method of detecting a subject having a repetitive pattern. If the value obtained by dividing the average correlation value by the minimum correlation value is larger than a certain threshold value, the average correlation value is a predetermined value or more, and the slope X is a predetermined value P or more,
The detection area is determined to be the effective area.

Specifically, it is determined as follows whether or not the area is the effective area. First, when the screen contrast is low, the brightness difference is small, so the correlation value is small. For example, the correlation value is small when the entire screen is white. In such a case, the reliability is lost, so the average correlation value ≧
When it is a predetermined value, it is determined to be valid. The predetermined value is determined by experiment. In this way, it is determined from the average correlation value whether or not the contrast is low.

When there is a moving object in the detection area, the correlation value differs between the part occupied by the moving object and the part not occupied, and the part occupied by the moving object has various correlation values, and the correlation value is generally Is relatively large (the degree of correlation is low). Therefore, when there is a moving object in the detection area, the minimum correlation value is likely to be large, and the partial motion vector in the detection area may be erroneously detected. If the partial motion vector is erroneously detected, the whole motion vector is erroneously detected. However, when the average correlation value is large, it is reliable even if the minimum correlation value is large to some extent. On the other hand, when the average correlation value is small, the minimum correlation value must be smaller to be reliable. Therefore, specifically, (average correlation value)
When / (minimum correlation value)> 7, it is determined to be valid, and the partial motion vector of the detection area that does not satisfy this condition is not used to prevent the above-mentioned adverse effects due to false detection. In this way, (average correlation value) / (minimum correlation value) is calculated,
Determine if there is a moving object.

Further, in the microcomputer 26,
In order to detect a subject having a repeating pattern (for example, a striped image), the correlation value of one point having the minimum correlation value and the four surrounding points are used. That is, as shown in FIG.
Assuming that the minimum correlation value is M and the correlation values at the four points on the left, right, top, and bottom are L, R, U, and D, respectively, the difference between the correlation values, that is, LM, RM, UM, and DM. And calculate
The minimum value among them is the slope X. The inclination X is compared with a predetermined value P (P = 4 in this embodiment) obtained by the field test. If the inclination X is equal to or larger than the predetermined value P, the detection area is determined to be a valid area, and if it is smaller than the predetermined value P, the detection area is determined to be an invalid area.

Based on these three conditions, it is judged whether or not the detection area is the effective area. Then, the microcomputer 26 obtains the average of the four partial motion vectors (average vector) and the average of the partial motion vectors of the effective area. Further, the divergence is obtained by the microcomputer 26. The degree of divergence evaluates the degree of variation of the partial motion vector in each detection region. The divergence is represented by equation 1.

[0020]

## EQU1 ## Divergence in X direction = .SIGMA. (| X direction partial motion vector of each detection area−X direction average vector |) / number of areas Y direction divergence = .SIGMA. (| Y direction partial motion vector in each detection area) -Y direction average vector |) / number of regions Here, the X direction partial motion vector indicates the X direction component of the partial motion vector, and the Y direction partial motion vector indicates the Y direction component of the partial motion vector. The X-direction average vector indicates the X-direction component of the average vector, and the Y-direction average vector indicates the Y-direction component of the average vector.

According to this divergence degree, the divergence coefficient Hk as shown in equation 2 is multiplied by the average of the partial motion vectors in the effective area to obtain the entire motion vector of the screen. The overall motion vector represents the amount of motion between fields and its direction.

[0022]

[Equation 2] 1) When 0 ≦ divergence <8 Hk = 1 2) When 8 ≦ divergence <16 Hk = 0.75 3) 16 ≦ When divergence <24 Hk = 0.5 4) 24 ≤ Divergence <32 Hk = 0.25 When the divergence is 32 or more, the smallest one of the four partial motion vectors is set as the overall motion vector.

As described above, when there is an effective area, the entire motion vector is obtained by multiplying the average of partial motion vectors of the effective area by the divergence coefficient Hk. The value of (total motion vector of current field × coefficient of 1 or less) can be used. Here, as a coefficient of 1 or less, "0.97"
Is preferred.

The reason why the divergence coefficient is set stepwise according to the degree of divergence is as follows. A large divergence indicates that there are large variations in each partial motion vector. This tendency tends to occur when there is a moving object in only a part of the screen. On the other hand, the small divergence indicates that the variation of each partial motion vector is small. This tendency occurs when there is no moving object on the screen or when there is a moving object on the entire screen.

Therefore, if the overall motion vector is obtained by using the partial motion vector when the divergence is large, the overall motion vector with low reliability is obtained. On the other hand, when the overall motion vector is obtained using the partial motion vector when the divergence is small, a highly reliable overall motion vector is obtained. Therefore, when the divergence is large, the average of the partial motion vectors is multiplied by the small divergence coefficient Hk to reduce the value of the overall motion vector. When the degree of divergence is small, for example, the average of the partial motion vectors is multiplied by a large divergence coefficient Hk to increase the value of the overall motion vector. In other words, by using the whole motion vector when the divergence is large as much as possible, on the contrary, by using the whole motion vector when the divergence is small as much as possible,
This is because a highly reliable global motion vector can be obtained as a whole.

The overall motion vector thus obtained is applied to the memory control circuit 22. Then, the memory control circuit 22 determines the read start address of the field memory 24 based on the overall motion vector, and reads the digital video signal stored in the field memory 24 from the address. That is, the memory control circuit 22 has the extraction area 46 formed by the digital video signal of the field memory 24 according to the overall motion vector calculated by the microcomputer 26.
Move (Fig. 6).

However, if the digital video signal read from the field memory 24 is as it is, the extraction area 4
Since 6 cannot be moved, electronic zoom circuit 48 (FIG. 1) is used. Referring to FIG. 6, the electronic zoom circuit 48 sets the extraction area 46 in which the image is enlarged according to the zoom magnification with respect to the size of the image field 44. The position of the extraction area 46 can be freely moved within the range of the image field 44 by changing the read start address of the field memory 24. And
In order to obtain the video signal of the entire image field 44 based on the extracted digital video signal, the image is enlarged by using the interpolation method based on the digital video signal read from the field memory 24.

In this way, the image field 44
By electronically zooming the image of an arbitrary extraction area 46 in the electronic zoom circuit 48, a correctable range 50 corresponding to the difference between the image field 44 and the extraction area 46 can be formed. When the video camera 10 is shaken due to the vibration of the hand of the person who operates the video camera 10 as shown in FIG. 7, the image from the video camera 10 is shaken, resulting in the image field 44.
If there is a target person in the lower left of the inside (Fig. 7 top),
This may be the case, for example, when the target person exists in the upper right of the image field 44 (bottom of FIG. 7). Therefore, the extraction area 46 is set for each field and the microcomputer 26
By moving in accordance with the overall motion vector calculated by, the target person will fit into the extraction area 46 as shown in the right side of FIG.

In this way, the digital video signal output from the electronic zoom circuit 48 is converted into an analog signal by the D / A converter 52 and output from the output terminal 54. The main operation of the microcomputer 26 of the video camera 10 will be described with reference to FIG. First, in step S1, each of the four detection areas (A-
It is determined whether D) is a valid area or an invalid area. At this time, as described above, if the value obtained by dividing the average correlation value by the minimum correlation value is larger than a fixed threshold value, the average correlation value is a predetermined value or more, and the slope X is a predetermined P or more, the detection area is valid. Judge as an area. If any one of these conditions is not satisfied, the detection area is determined to be an invalid area.

Then, in step S3, it is determined whether or not there is a valid area, and if there is even one valid area, the process proceeds to step S5. In step S5, the divergence is calculated. The divergence is calculated independently of the divergence in the X direction and the divergence in the Y direction, as shown in Formula 1. If 0 ≦ divergence degree <8 in step S7, step S7
In step 9, the divergence coefficient Hk is set to 1, and the process proceeds to step S11. If 0 ≦ divergence <8 in step S7,
It proceeds to step S13. If 8 ≦ divergence degree <16 in step S13, the divergence coefficient Hk = 0.75 is set in step S15, and the process proceeds to step S11. If 8 ≦ divergence degree <16 in step S13, the process proceeds to step S17. In step S17, 16 ≦ divergence degree <
If it is 24, the divergence coefficient Hk =
It is set to 0.5 and the process proceeds to step S11. If 16 ≦ divergence degree <24 in step S19, step S21
Proceed to. If 24 ≦ divergence degree <32 in step S21, divergence coefficient Hk = 0.2 in step S23.
5, and the process proceeds to step S11. And step S1
1, the arithmetic mean of the partial motion vectors in the effective area ×
Find Hk. The process of step S11 is performed independently for each of the X-direction partial motion vector and the Y-direction partial motion vector, and is performed for the X direction.
The result of step 11 and the result of step S11 performed in the Y direction are combined to obtain the overall motion vector, and the process ends.

On the other hand, in step S21, if 24 ≦ divergence degree <32, the process proceeds to step S25. In step S25, the smallest one is selected from the four partial motion vectors, and this is set as the overall motion vector, and the process ends. On the other hand, if there is no effective area in step S3, the process proceeds to step S27. In step S27, the whole motion vector of the previous field multiplied by a coefficient of 1 or less is ended as the whole motion vector of the current field. Here, "0.97" can be preferably used as the coefficient of 1 or less.

According to such a video camera 10, even if a moving object exists in two of the four detection areas, it is less likely to be affected by the moving object, and the amount of camera shake is more accurate. Can be detected. Regardless of whether it is the effective area or the invalid area, the average of the partial motion vectors of each detection area may be multiplied by the divergence coefficient Hk to be the entire motion vector. In particular, it is effective when it is not determined whether the area is valid or invalid.

Needless to say, the value to be compared with the divergence is not limited to that of the equation 2, and the divergence coefficient Hk is not limited to that of the equation 2.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention.

FIG. 2 is a block diagram showing a motion detection circuit of the embodiment in FIG.

FIG. 3 is an illustrative view showing a principle of electronic zoom and showing a detection region in an image field.

FIG. 4 is an illustrative view showing a principle of electronic zoom and showing representative points and sampling points in a detection area.

FIG. 5 is an illustrative view for explaining a method of detecting a subject having a repetitive pattern by using a pixel having a minimum correlation value and four pixels around the pixel.

FIG. 6 is an illustrative view showing the principle of camera shake correction.

FIG. 7 is an illustrative view showing each block in an image field to which the representative point matching method is applied.

FIG. 8 is a flowchart showing main operations of this embodiment.

[Explanation of symbols]

 10 ... Video camera 20 ... Motion detection circuit 22 ... Memory control circuit 24 ... Field memory 26 ... Microcomputer 44 ... Image field 46 ... Extraction area 48 ... Electronic zoom circuit

Claims (1)

[Claims] 1. A video camera having a camera shake correction device that obtains an overall motion vector by using partial motion vectors of each of a plurality of detection areas formed on a screen, the video camera being an average of the partial motion vectors. Averaging means for obtaining an average vector, divergence calculating means for obtaining the divergence by adding absolute values of differences between the respective partial motion vectors and the average vector, and dividing the addition result by the number of additions, the divergence And a global motion vector detecting means for detecting the global motion vector by a different method according to a comparison result by the comparing means.
A video camera having an image stabilizer.