JP2735264B2 - Motion compensated imaging device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion-corrected image pickup apparatus that performs camera shake correction.

2. Description of the Related Art As a conventional motion-compensated imaging device, for example,
There is one disclosed in -143330. FIG. 8 shows a block diagram of a conventional motion-compensated imaging apparatus. 8, 81 is a lens, 82 is an image sensor, 83 is a drive circuit for the image sensor 82, 84 is a drive control circuit for controlling the drive circuit 83, 85 is a signal processing circuit including an image memory, 86 is a lens 81, A motion detection circuit 87 detects a rapid motion of the image sensor 82, and a motion correction control circuit 87 performs a motion correction control in response to a signal from the motion detection circuit 86.

In the conventional motion-compensated imaging apparatus configured as described above, the motion detection circuit 86 detects the motion of the lens 81 and the image sensor 82, and the motion-correction control circuit 87 controls the signal processing circuit 85 to perform motion-free motion. Obtain the corrected video signal.

The operation of the signal processing circuit 85 at this time will be described with reference to FIG. In FIG. 9, the entire memory area indicated by the solid line is the area of the image memory in the signal processing circuit 85 in the vertical direction V ₀ × horizontal direction H ₀ , which corresponds to the photoelectric conversion area of the image sensor 82. The memory area read as a video signal is:
For example, there may be a motion compensation memory area I [V ₁ × H ₁ ] or a motion compensation memory area II [V ₂ × H ₂ ]. The motion component caused by the camera shake or the like is corrected by changing the read address from the memory to obtain a video signal without the camera shake.

However, in the above-described configuration, in order to perform the motion correction, the imaging signal is once fetched into the memory, and thereafter, the motion correction control circuit uses the signal of the motion detection circuit to perform the memory read control. Is performed to obtain a motion-compensated video signal.However, if the lens is composed of a zoom lens and the motion detection circuit is a sensor, the actual blur amount may vary depending on the zoom amount even if the sensor output value is the same. Had the problem of being different.

An object of the present invention is to solve such a problem, and an object of the present invention is to provide a motion-corrected imaging device capable of sufficiently satisfying motion correction even when an imaging device having a zoom function is used.

Means for Solving the Problems To solve the above problems, the present invention provides an optical unit including a zoom lens, a zoom position detection circuit that detects a zoom ratio from a zoom position signal obtained from the optical unit, and an image sensor. A drive circuit for the image sensor, a motion detection sensor circuit for detecting motion of the optical unit and the image sensor, and a signal for detecting motion of the optical unit and the image sensor based on a change in image data from an output signal of the image sensor. A processing motion detection circuit;
At least a memory circuit to which a signal from the image sensor is provided and having a capacity of at least one field, and a memory control circuit that changes a read address of the memory circuit or a drive control circuit that changes image sensor drive of the drive circuit Either one, a zoom ratio signal obtained from the zoom position detection circuit, a first motion signal from the motion detection sensor circuit, and a motion correction signal from a second motion signal from the signal processing motion detection circuit. And a motion correction circuit for correcting at least one of the memory control circuit or the drive control circuit with the obtained motion correction signal,
The motion correction circuit corrects the first motion signal using the zoom ratio signal, and uses the corrected first motion signal and the second motion signal to improve the reliability of the second motion signal. If the value is equal to or less than the reference value and if the reliability of the second motion signal is equal to or more than the reference value and the difference between the two does not exist within a certain range, a motion correction signal is obtained from the corrected first motion signal. 2
If the reliability of the motion signal is equal to or more than the reference value and the difference between the two is within a certain range, a motion correction signal is obtained from the second motion signal, or the motion direction from the corrected first motion signal is obtained. The motion compensation signal is obtained from the information and the motion amount information from the second motion signal.

Further, an optical unit that forms an image of light from a subject, an image sensor, a drive circuit of the image sensor, a motion detection sensor circuit that detects movement of the optical unit and the image sensor, and an output signal of the image sensor. A signal processing motion detection circuit for detecting the motion of the optical unit and the image sensor based on a change in image data; a memory circuit provided with a signal from the image sensor and having a capacity of at least one field; A memory control circuit for changing an address, an electronic zoom control circuit for changing a memory read area in the memory control circuit in accordance with an electronic zoom setting value, a first motion signal from the motion detection sensor circuit, and the signal A motion compensating circuit for obtaining a motion compensating signal from the second motion signal from the processing motion detecting circuit; Using the signal and the second motion signal, when the reliability of the second motion signal is equal to or less than the reference value, and when the reliability of the second motion signal is equal to or more than the reference value, and the difference between the two is within a certain range. If not, a motion compensation signal is obtained from the first motion signal. On the other hand, if the reliability of the second motion signal is greater than or equal to a reference value and the difference between the two is within a certain range, the second motion signal is obtained.
Or a motion correction signal is obtained from the motion direction information from the first motion signal and the motion amount information from the second motion signal.
From a read area corresponding to an electronic zoom position signal obtained from the electronic zoom control circuit and a motion correction signal obtained from the motion correction circuit, at least one of a read address of the memory circuit or a read position of the image sensor is determined. It is configured to control.

According to the above configuration, the first motion signal from the motion detection sensor circuit is corrected using the zoom ratio signal from the zoom position detection circuit, and an accurate motion signal is obtained from the motion detection sensor circuit even during the zoom operation. Using the first motion signal and the second motion signal from the signal processing motion detection circuit for detecting motion by signal processing, the high reliability of the motion detection sensor circuit and the high reliability of the signal processing motion detection circuit due to image detection. By obtaining an accurate motion compensation signal, an accurate motion compensation signal is used to control at least one of a drive circuit for controlling the driving of the image sensor and a memory circuit for holding the output signal of the image sensor, thereby obtaining an accurate motion compensation signal. Perform motion compensation.

Further, by using the first motion signal from the motion detection sensor circuit and the second motion signal from the signal processing motion detection circuit, high reliability of the motion detection sensor circuit and image detection by the signal processing motion detection circuit are achieved. The memory control circuit obtains a high-precision motion compensation signal and further obtains control data for simultaneously implementing the electronic zoom and the motion compensation using the motion compensation signal and the electronic zoom data. The removal was performed by the readout area control or the motion compensation circuit was removed by the drive control that changes the image sensor drive in the drive circuit, and the memory control circuit obtained the electronically zoomed video signal from the memory circuit to correct the motion. Obtain an electronic zoom image.

Further, even when an electronic zoom is used in place of the zoom mechanism, similar motion correction is performed based on the zoom ratio obtained from the electronic zoom control circuit, and an image signal without blur is obtained.

An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 shows a block diagram of a motion-corrected imaging apparatus according to a first embodiment of the present invention. In FIG. 1, reference numeral 11 denotes a zoom mechanism which is an optical unit including a zoom lens, reference numeral 12 denotes a zoom drive circuit, reference numeral 13 denotes a zoom position detection circuit which detects a zoom position of the zoom mechanism 11, reference numeral 14 denotes an image sensor, and reference numeral 15 denotes an image sensor. An element drive circuit, 16 is an element drive control circuit that controls the element drive circuit 15, 17 is a zoom mechanism 11, a motion detection sensor circuit that detects the movement of the image sensor 14 in the X and Y directions, 18 is a memory circuit,
Reference numeral 19 denotes a memory control circuit for changing a read address from the memory circuit 18, and reference numeral 20 denotes a motion for correcting the motion signal of the motion detection sensor circuit 17 based on the output signal of the zoom position detection circuit and correcting the control by the memory control circuit 19. Correction circuit, 31
Is a signal processing circuit.

The operation of the motion-compensated imaging apparatus according to the first embodiment thus configured will be described below. Zoom drive circuit
The zoom mechanism 11 is set to a predetermined zoom amount by 12, and the zoom position is detected by the zoom position detection circuit 13. Further, the amount of motion is detected from the motion detection sensor circuit 17, and the amount of motion is corrected by the output signal of the zoom position detection circuit 13. Based on the data, the motion correction circuit 20 controls the memory control circuit 19 to control the memory circuit. By changing the read address from 18, a video signal from which the blur component has been removed can be obtained.

Next, the operation of the motion correction circuit 20 will be described. FIG. 2 is a diagram for explaining the operation of the memory circuit of the first embodiment. In Figure 2, the size of the entire memory area A ₀ is _[V0 ×
H ₀ ], which corresponds to the photoelectric conversion region of the image sensor 14. The memory area read as a video signal during the motion correction control is, for example, a motion correction memory area (I)
A ₁ , size [V ₁ × H ₁ ] or motion compensation memory area (I
I) A ₂ , size [V ₂ × H ₂ ], etc. These A ₁ and A
₂ region is less than the entire memory area A _0, and thus the motion component caused by camera shake are corrected in the memory area.

However, to change the zoom amount when the motion correction control, motion compensation areas (I) A ₁ and motion compensation region (II) when the zoom amount at the time of photographing of A ₂ are different, for example, A ₁ region when changed from TELE to WIDE of occurs when a subject region is equal to a ₁₂ region, which is part of the a ₂ regions, and the motion amount and motion amount in the memory circuit 18 from the motion detection sensor circuit 17 is different.

Therefore, the motion correction circuit 20 uses the output signals of the motion detection sensor circuit 17 and the zoom position detection circuit 13 to generate a memory circuit.
The amount of motion on 18 is obtained, and a memory control circuit 19 is controlled to extract a motion-corrected signal without blur from the memory circuit 18.

At this time, the motion correction data of the motion correction circuit 20 is obtained by, for example, inputting the motion amount data from the motion detection sensor circuit 17 to the variable amplification factor amplification circuit, and using the zoom position as a signal for controlling the variable amplification factor amplification circuit. The zoom amount data from the detection circuit 13 is used. Using the x1 signal of the zoom amount data as a reference signal, the gain at this time is set to 1 times, and the gain is set to 2 times for the x1 signal, ie, from WIDE to TELE, and to 3 times for the x3 signal. In this way, the motion amount data is corrected with the zoom amount data to obtain the motion correction data.

As described above, according to the first embodiment, the zoom amount is obtained by obtaining the motion correction data from the motion amount data due to the shake from the motion detection sensor circuit 17 and the zoom amount data from the zoom position detection circuit 13. A good image without blur can be obtained even when changing the shooting.

FIG. 3 is a block diagram of a motion-compensated imaging apparatus according to a second embodiment of the present invention. In FIG. 3, the blocks 31 to 41 are the same as the blocks 11 to 21 of the first embodiment, and the difference is that a signal processing motion detecting circuit 42 is provided. The signal processing motion detection circuit 42 detects the motion of the zoom mechanism 31 and the image pickup device 34 from the image information in the same manner as the motion detection sensor circuit 37, and obtains a motion vector in the x and y directions. This is, for example, "screen shake correction device"
(Published in the Technical Report of the Institute of Television Engineers of Japan on May 28, 1987). Further, as described later, the motion correction circuit 40 obtains a more accurate motion amount based on the signals of both the motion detection sensor circuit 37 and the signal processing motion detection circuit 42, and detects the motion when the zoom ratio changes. The output of the sensor circuit 37 is converted into a motion amount changed according to the zoom ratio, and the motion amount is corrected based on this signal and the output of the signal processing motion detection circuit 42 to be provided to the memory control circuit 69.

Hereinafter, points different from the first embodiment will be mainly described. 4th
FIG. 3 shows an example of the configuration of the signal processing motion detection circuit 42 in FIG. 3, and FIG. 5 is an explanatory diagram of an input signal to the signal processing motion detection circuit 42. 4 and 5, reference numeral 51 denotes a representative point memory circuit for storing a signal of a representative point (R ₁₁ , R ₁₂ , R ₁₃ ...) Among input signals, and 52 denotes a current input signal (S ₁₁ , S _12). ..) And a representative point signal before a predetermined time, 53 is a detection memory circuit that stores the calculation result, and 54 is a motion amount calculation circuit that obtains a motion amount from the output of the detection memory circuit 53. The absolute value of the difference between the representative point R _ij and the signals S _{i + x} , _{j + y} having a positional relationship separated by the horizontal direction x and the vertical direction y is defined as Δx _i y _j . This difference is added to xy having the same positional relationship for each representative point to obtain _Dxy .

D _xy = Σ _ij Δx _i y _j, and the minimum values x and y in D _xy are set as horizontal and vertical movement amounts.

The operation of the second embodiment thus configured will be described below. In the second embodiment, (data I) from the motion detection sensor circuit 37 and (data II) from the signal processing motion detection circuit 42 can be obtained as motion amount data. Therefore, the motion correction circuit 40 uses both data to obtain a more accurate motion amount. For example, (a) compare both data and use (data II) if the difference between both data is within a certain range, or (b) obtain the direction from (data I) and (data In II), only the amount of movement of the directional component is obtained from calculation.

However, when the zoom amount of the zoom mechanism 31 changes during this motion correction control, (data I) from the motion detection sensor circuit 37 differs from the motion amount on the memory circuit as in the first embodiment. Therefore, the motion compensating circuit 40 includes a zoom position detecting circuit 33.
When the zoom amount changes, (Data I) is corrected using the zoom amount data, and motion correction data is obtained from the corrected signal and (Data II). Under control, a motion-corrected signal without blur is extracted from the memory circuit.

Next, a method of obtaining motion correction data from a signal obtained by correcting (data I) using the zoom amount data and (data II) will be described. A motion amount detection device using a signal processing method tends to generate an error in motion detection for a special subject image. Therefore, the reliability of the motion detection data is determined by signal processing, for example, from the distribution of the minimum value, the distribution of the minimum value and the second smallest value, or the ratio of the minimum value to the average value. When the value is equal to or less than the value, the motion correction data is obtained from the signal obtained by correcting the (data I) using the zoom amount data without using the (data II) according to the signal processing method. The motion correction data is obtained from the signal obtained by correcting (data I) and the data (data II), and the memory control circuit 39 is controlled to control the memory circuit 38
A motion-corrected signal without blur is extracted from the signal.

As described above, according to the second embodiment, the motion detection sensor circuit 37, the signal processing motion detection circuit 42, and the zoom position detection circuit
33, when the zoom function is not operating, highly accurate and highly reliable motion amount data is obtained by using (data I) from the motion detection sensor circuit 37 and (data II) from the signal processing motion detection circuit 42. In addition, when operating the zoom function,
A motion amount corresponding to the zoom amount is obtained by using (Data I) and the zoom amount from the zoom position detection circuit 33, and motion correction data is obtained from the motion amount and (Data II). It is possible to obtain a good image without blurring for a special subject image in which the image is likely to occur.

FIG. 6 is a block diagram of a motion-compensated imaging apparatus according to a third embodiment of the present invention. In FIG. 6, the blocks 64 to 71 are the same as the blocks 14 to 21 of the first embodiment, respectively, except that a lens 72 as an optical unit and an electronic zoom control circuit 73 are provided. Hereinafter, the points different from the first and second embodiments will be mainly described. As shown in FIG. 6, the output signal of the image sensor 64 is once stored in the memory circuit 68,
In the case of a configuration in which video signals are read out according to the memory control circuit 69, the memory control circuit 69 and the signal processing circuit 71 can have a zoom function by a reading method from the memory circuit 68 and signal processing such as interpolation. This will be described with reference to FIG. Entire memory area A ₀ in FIG. 7 corresponds to the photoelectric conversion region of the imaging element. The motion compensation memory areas A _M1 and A _M2 are smaller than A ₀ to compensate for a camera shake component, and the memory area A _M1 when no motion is compensated when the zoom function is not operated is located at the center of the entire memory area A _0. I have. Also shows the A ₁ and A ₂ as zoom function memory area using the memory (I) and (II). In this case, the zoom ratio is twice. The A ₁ and A ₂ in the horizontal direction to half the frequency of the read clock, the vertical direction by reading the NTSC signal by performing line interpolation is has a substantially zoom function.

As described above, using the signal from the electronic zoom control circuit 73, the memory control circuit 69 reads the signal of the zoom function memory area from the memory circuit 68, and obtains the zoomed video signal via the signal processing circuit 71. In the imaging device having the zoom function, the motion correction is performed by the motion correction circuit 70 using the motion amount data from the motion detection sensor circuit 67, and the zoom position and the zoom position obtained from the motion correction data and the electronic zoom control circuit 73. Using the data of the zoom ratio, the memory control circuit 69 reads out the data from the memory circuit 68, passes through the signal processing circuit 71, and obtains a zoomed and motion-compensated signal.

For example, in FIG. 7, when the zoom function is not operating,
In the motion detection sensor circuit motion compensation circuit 70 using the motion amount data obtains a motion compensation data from the 67, the memory control circuit 69 reads the start position, for example from the memory circuit 69 is P ₁ or P ₂ in accordance with the motion compensation data A certain motion compensation memory area A _M1 or A _M2 is read to perform motion compensation. Next, during the operation of the zoom function, when no motion is corrected, the memory control circuit 69 determines the read start position Q1, the read clock frequency, and the interpolation method using the zoom amount data from the electronic zoom control circuit 73, and the memory circuit 68 The zoom function memory area A1 is read from and the zoom is performed. At this time, if further motion compensation is to be performed, the motion compensation data obtained by using the motion amount data from the motion detection circuit 70 and the zoom amount data from the electronic zoom control circuit 73 are added, and the memory control circuit 69 determines the read start position Q ₂ to which was interpolation method and motion correction and the read clock frequency, reads a zoom function memory area a ₂ from the memory circuit 68, performs a predetermined zoom and motion compensation.

As described above, according to the third embodiment, the memory control circuit 69 obtains a zoomed video signal from the memory circuit 68 using the zoom data of the electronic zoom control circuit 73. Using the motion amount data of 67, the memory control circuit 69 obtains a favorable video signal from the memory circuit 68 on which zooming and motion correction have been performed. In addition, since there are many common parts in the circuit configuration, it can be realized with a small circuit scale.

As the motion detection sensor circuits 17, 37, and 67 provided in the first, second, and third embodiments, a method of detecting the amount of motion by a mechanical gyro is conceivable. Since there is no description, the description is omitted.

Next, in the first embodiment, as a method of obtaining the motion correction data from the zoom amount data and the motion amount data, the case where the motion amount data is corrected using the zoom amount data by the variable amplification factor amplifying circuit has been described. It is not limited to. Then, when the zoom amount data changes, it is conceivable that the motion compensation is stopped and the motion compensation data is changed to the non-compensation state, or the motion compensation data immediately before is used again, or the motion compensation data is switched to the average fixed data. However, in a broad sense, it is considered that the motion correction data is obtained from the zoom amount data and the motion amount data.

In the second embodiment, the motion detection signal processing circuit is not limited to this, and any other method or configuration may be used as long as the circuit obtains the amount of motion by a signal processing method. The method of synthesizing the two pieces of motion amount data and the zoom amount data and the method of determination performed in step 2 are not limited to those described here.

Further, in the first, second, and third embodiments, a method of controlling a drive control circuit of an image sensor can be considered as a method of obtaining a video signal without blur from the obtained motion correction data.

Further, in the third embodiment, the case of the motion detection sensor circuit for detecting the motion of the optical unit and the imaging device has been described. However, as described in the second embodiment, a motion detection signal processing circuit may be used. It is clear that the configuration may be made of both.

Further, in the first and second embodiments, the case where the zoom circuit is driven by the zoom drive circuit has been described, but a manual zoom circuit configuration may be used. It is apparent that the zoom drive circuit may be controlled by the zoom drive control circuit, and the zoom position detection circuit may use a signal from the zoom drive control circuit.

As described above, according to the present invention, in the imaging apparatus having the zoom circuit, the first motion signal from the motion detection sensor circuit is corrected using the zoom ratio signal from the position detection circuit, and even when the zoom operation is performed. A motion detection sensor circuit that obtains an accurate motion signal from the motion detection sensor circuit, and uses the corrected first motion signal and a second motion signal from a signal processing motion detection circuit that detects motion by signal processing; Thus, accurate motion compensation can be performed by obtaining a highly reliable motion compensation signal based on high reliability and image detection of the signal processing motion detection circuit.

Further, in an image pickup apparatus having an electronic zoom function, high reliability of the motion detection sensor circuit is obtained by using the first motion signal from the motion detection sensor circuit and the second motion signal from the signal processing motion detection circuit. By obtaining a high-precision motion compensation signal based on the image detection of the signal processing motion detection circuit, and further obtaining control data for simultaneously implementing electronic zoom and motion compensation using the motion compensation signal and the electronic zoom data, the motion compensation is performed. The obtained electronic zoom image can be obtained, and the practical effect is great.

[Brief description of the drawings]

FIG. 1 is a block diagram of a motion-corrected imaging apparatus according to a first embodiment of the present invention, FIG. 2 is an explanatory diagram of the operation of a memory circuit according to the first embodiment, and FIG. 3 is a second embodiment of the present invention. 4, FIG. 4 is a block diagram of a signal processing motion detection circuit of the second embodiment, FIG. 5 is an operation explanatory diagram of the signal processing motion detection circuit of the second embodiment, and FIG. FIG. 7 is a block diagram of the third embodiment, FIG. 7 is an operation explanatory diagram of the memory circuit of the third embodiment,
FIG. 8 is a block diagram of a conventional motion-compensated imaging apparatus, and FIG. 9 is an operation explanatory diagram of a conventional memory circuit. 11,31 …… Zoom mechanism, 12,32 …… Zoom drive circuit, 13,3
3… Zoom position detection circuit, 14,34 …… Image sensor, 15,35
…… Element drive circuit, 17,37 …… Motion detection sensor circuit, 18,
38 …… Memory circuit, 19,39 …… Memory control circuit, 20,40…
... Motion correction circuits, 21, 41 ... Signal processing circuits, 42 ... Signal processing motion detection circuits, 64 ... Imaging elements, 65 ... Element drive circuits, 67 ... Motion detection sensor circuits, 68 ... Memory circuits, 69
... A memory control circuit, 70 a motion compensation circuit, 71 a signal processing circuit, 72 a lens, 73 an electronic zoom control circuit.

Claims (2)

(57) [Claims] An optical unit including a zoom lens; a zoom position detection circuit for detecting a zoom ratio from a zoom position signal obtained from the optical unit; an image sensor; a drive circuit for the image sensor; A motion detection sensor circuit for detecting a motion of the element, a signal processing motion detection circuit for detecting a motion of the optical unit and the image sensor based on a change in image data from an output signal of the image sensor, and a signal from the image sensor. A memory circuit having a given capacity of at least one field, at least one of a memory control circuit for changing a read address of the memory circuit, and / or a drive control circuit for changing image sensor driving of the drive circuit; A zoom ratio signal obtained from a position detection circuit, a first motion signal from the motion detection sensor circuit,
A motion correction circuit for obtaining a motion correction signal from the second motion signal from the signal processing motion detection circuit, and correcting at least one of the memory control circuit or the drive control circuit with the obtained motion correction signal; The motion correction circuit corrects the first motion signal using the zoom ratio signal, and uses the corrected first motion signal and the second motion signal to improve the reliability of the second motion signal. If the value is equal to or less than the reference value and if the reliability of the second motion signal is equal to or more than the reference value and the difference between the two does not exist within a certain range, a motion correction signal is obtained from the corrected first motion signal. If the reliability of the second motion signal is greater than or equal to the reference value and the difference between the two is within a certain range, a motion correction signal is obtained from the second motion signal, or the motion from the corrected first motion signal is obtained. Direction information and motion from the second motion signal Motion compensation image pickup device to obtain a motion compensation signal from the amount information. An optical unit for forming an image of light from a subject, an image sensor, a drive circuit for the image sensor, a motion detection sensor circuit for detecting the movement of the optical unit and the image sensor, A signal processing motion detection circuit for detecting a motion of the optical unit and the image sensor based on a change in image data from an output signal; a memory circuit to which a signal from the image sensor is provided and having a capacity of at least one field; A memory control circuit for changing a read address of a circuit, an electronic zoom control circuit for changing a memory read area in the memory control circuit in accordance with an electronic zoom set value, and a first motion signal from the motion detection sensor circuit And a motion compensation circuit for obtaining a motion compensation signal from the second motion signal from the signal processing motion detection circuit. The reliability of the second motion signal is less than or equal to the reference value, and the reliability of the second motion signal is greater than or equal to the reference value, and the difference between the two is within a certain range. , The motion compensation signal is obtained from the first motion signal. On the other hand, if the reliability of the second motion signal is equal to or more than the reference value and the difference between the two is within a certain range, the second motion signal is obtained. The motion compensation signal from
A motion correction signal is obtained from the motion direction information from the motion signal and the motion amount information from the second motion signal, and the memory control circuit reads out a read area corresponding to an electronic zoom position signal obtained from the electronic zoom control circuit. And a motion correction signal obtained from the motion correction circuit and controlling at least one of a read address of the memory circuit and a read position of the image sensor.