JP3116422B2 - Camera control device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera control device suitable for application to, for example, a video camera or a VTR integrated with a camera.

[0002]

2. Description of the Related Art Conventionally, a CCD (Charge-Coupled
2. Description of the Related Art A video camera using a device, an image pickup tube, or the like is widely used in general and business use. Such a video camera uses the light of the subject captured by the optical system as CC.
A photoelectric conversion is performed by a D element, an image pickup tube, or the like, and predetermined signal processing is performed to obtain a color video signal of a system such as NTSC, SECAM, and PAL. Then, the video signal obtained by this imaging is supplied to a monitor and projected on the screen thereof, or supplied to a recording device and recorded on a recording medium used in the recording device. It has been done. Such a video camera shoots an image while viewing the image with a viewfinder called a viewfinder or the like.

In general, when photographing using such a video camera, the photographing is performed by focusing on the subject to be photographed. This focusing includes manual focusing and automatic focusing. Manual focusing is performed by manually changing the position of the lens. On the other hand, automatic focusing is generally called autofocus, and conventionally, high-frequency components and contrast are detected from a video signal obtained by imaging, and a lens is moved so as to improve the detection result to focus. I try to match.

[0004] As an example of the auto focus for automatically adjusting the focus, for example, the high frequency power P
The ratio of _H and a low power P _L is a method using a principle such that the maximum.

[0005] Figure 11 the ratio and the vertical axis P _H / P _L, as x (lens position) on the horizontal axis, indicating the state of focus and out-of-focus. FIG. 1 shows a state in which a subject is moved from a focused state to an out-of-focus state by moving a camera.
It is shown in FIG. FIG 12 is a graph showing the ratio of the vertical axis similarly to FIG. 11 described above P _H / P _L, the horizontal axis t (time).

In FIG. 12, at t1, when the lens starts to shift from the in-focus state to the out-of-focus state, at t2, the lens is moved when the lens deviates from the in-focus state. , T3 indicates the case where the movement is performed in the opposite direction to the case of t2.

At this time, for example, the direction in which the lens is moved (front and rear) cannot be immediately determined only from such a signal. This is because, as is clear from FIG. 11, the graph is symmetrical about a certain position in the focused state. That is,
This is because the ratio P _H / P _L becomes the same regardless of the direction in which the position deviates from the in-focus position, both in the forward direction and in the backward direction.

In this case, as described above, the lens is once moved in a certain direction, and when the ratio P _H / P _L is increased, the lens is further moved so as to be in focus in that direction. When the ratio P _H / P _L decreases, the lens is moved so as to be in focus in a direction opposite to the above-described direction.

In recent years, a subject is irradiated with infrared light, reflected light from the subject is received, a distance between the camera and the subject is measured, and a lens is moved based on the measured distance. Cameras and video cameras adopting a method of focusing on a camera are widely used in general.

[0010]

Meanwhile [0007] As described above, when shifted from the focusing can be reduced even if the relative distance between the camera and the subject becomes larger, the ratio P _H shown in FIG. 11 since / P _L drops in the same way, to detect the high-frequency component and the contrast from the video signal obtained by imaging, it has been to focus by moving the lens so the detection result is improved. Therefore, such a camera control device has an unnatural and poor followability of the autofocus operation. Therefore, for example, a video camera or the like equipped with such a camera control device has a disadvantage that the user feels poor in usability.

As described above, when the subject is irradiated with infrared light, the reflected light is received to obtain the distance between the subject and the camera, and the lens is moved to perform focusing. However, there has been a problem that a dedicated sensor or the like is newly mounted on the camera control device, and the configuration becomes complicated.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and makes it possible to naturally and favorably follow an autofocus operation with a simple configuration without newly mounting a dedicated sensor or the like. When the present invention is applied to a camera control device, it is intended to propose a camera control device capable of improving a feeling of use for a user.

[0013]

As shown in FIGS. 1 to 5, for example, as shown in FIGS. 1 to 5, when the contrast of a video signal from an image pickup means 1 for picking up a subject is small, at least two screens of an image are controlled. Processing means 3, 4, and 5 for performing a closed-loop integration of a motion vector from a signal; obtaining perspective data of a subject based on the processing results from the processing means 3, 4, and 5; A control unit 6 that determines the direction and amount of movement of the lens and controls the focus of the imaging unit 1 on the subject.

[0014]

According to the present invention described above, when the contrast of a video signal is low, motion vectors are closed-circuit integrated from video signals of at least two screens, and based on the perspective data of the object obtained based on this processing result. In addition, since the moving direction and the amount of the lens of the image pickup means 1 are determined and the focus of the image pickup means 1 on the subject is controlled, the auto focus can be performed with a simple configuration without newly mounting a dedicated sensor or the like. When the present invention is applied to a video camera or the like, the followability of the operation is made natural and good.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the camera control apparatus according to the present invention will be described below in detail with reference to FIG.

In FIG. 1, reference numeral 1 denotes a video camera using, for example, a CCD element, which photoelectrically converts light from a subject by an optical system of the video camera 1, and stores, for example, a color video signal of, eg, an NTSC system obtained in a memory. 3, the memory 4, the monitor 7, and the recording unit 8, respectively.

The memory 3 and the memory 4 store, for example, the whole or a part of the image data (for example, one frame). In this example, for example, the memory 3 stores the image data of I (t), and the memory 4 stores the image data of I (t−1). In addition, many memories other than the memories 3 and 4 may be used to store image data up to I (t−n).

The processing unit 5 detects the movement of a set of a group of pixels, which are part of pixel points or blocks, from the image data I (t) and I (t-1) from the memories 3 and 4. A signal M (t) is generated, and the signal M (t) is supplied to the determination unit 6. At the same time, the processing unit 5 generates a signal P (t) necessary for determining the focus by calculating, for example, the ratio of the high-frequency component and the low-frequency component of the image data. (T) is supplied to the determination unit 6.

The judging section 6 outputs a signal M (t) from the processing section 5.
And P (t), a control signal for the video camera 1 necessary for focusing is generated, and the generated control signal is supplied to the video camera 1.

The video camera 1 drives, for example, a lens based on the control signal from the determination unit 6 to bring the lens into a focused state.

In this embodiment, as shown by two-dot chain lines in FIG. 1, the memories 3 and 4, the processing unit 5, and the judgment unit 6 are formed into a so-called one-chip, that is, an integrated circuit. Further, the processing unit 5 and the determination unit 6 may be configured as a microcomputer.

To obtain the above-described detection signal M (t), that is, to obtain a motion vector at a corresponding point between the image data I (t-1) and the image data I (t) is called an optical flow. The method used is Hereinafter, this optical flow will be described.

The optical flow is a trajectory corresponding to pixels having invariant luminance in the time direction, that is, I (x, y,
Assuming that t) is luminance, a locus (x, y) that satisfies the following equation 1 corresponds to a two-dimensional vector.

(Equation 1) Normally, it is assumed that u≡∂x / ∂t and v≡∂y / ∂t change smoothly in the (x, y) space, and u and v that minimize the following Expression 2 are obtained.

(Equation 2) _{However, I x = ∂I / ∂x,} I y = ∂I / ∂y, I t = ∂I
/ ∂t, u _x = ∂u / ∂x, u _y = ∂u / ∂y, v _x = ∂
Let v / ∂x, v _y = ∂v / ∂y. In addition, a so-called line process may be introduced so that calculation can be performed correctly even at discontinuous boundaries. Now, as a method of calculating the movement of the target 9 for the obtained flow (u, v) in each of (x, y), for example, averaging in a specified area or averaging an area surrounded by a line process There are methods.

In order to minimize the above-mentioned equation (2), the following equations (3) and (4), ie, two linear equations for u and v are obtained.

(Equation 3)

(Equation 4) Then, the thus obtained {u (x, y), v
For (x, y)}, the motion vector of the designated area is represented by the following equation (5) (an example by averaging).

(Equation 5) Now, as shown in FIGS. 2A and 2B, the image data I (t−
1) and image data I (t) are considered. At this time, I
The position of each pixel or a set of pixels in a block within (t-1) moves to a certain position in I (t). The vector connecting the corresponding points is the motion vector shown in FIG. 2C.

As shown in FIG. 2C, the motion vectorm
(X, y) is obtained, and as shown in FIG.
In a region where the vector is continuous, a closed line is
Assume. The closed line may be arbitrary, but it should be large within the area.
Shall be. Along this closed curve as shown
Lever curve normal unit vectorn(X, y) (traveling direction
The normal to the right) and the motion vectorm(X, y)
Consider taking the inner product and integrating around once. This is the next number
6 can be represented.

(Equation 6) Here, if the subject is moving in the direction away from the camera (if the camera is moving in the direction away from the subject), as shown in m (x, y) shown in FIG. , M in Equation 6 is a positive value.

On the other hand, if the subject is moving in the direction approaching the camera (if the camera is moving in the direction approaching the subject), the outside of the entire area is opposite to m (x, y) shown in FIG. Therefore, M in Expression 6 becomes a negative value.

Therefore, the judgment unit 6 shown in FIG.
From the signal (t), it can be determined whether the distance between the subject and the camera is short or long.

Now, the signal M (t) in the processing unit 5
Will be described in more detail with reference to FIGS. A basic expression for obtaining M (t) can be expressed by the following equation (7).

(Equation 7) The following equation 8 can be derived from the equation 7 and Gauss's theorem.

(Equation 8) Here, a case where M (t) is obtained by the above equation 7 will be described. First, as shown in FIG. 7, a closed circuit C is assumed, and as shown in FIG. 8, a vector of a black point on the closed circuit C is set to m (x, y).

The normal vector n (x, y) of the closed circuit C shown in FIG. 8 is taken outside the closed loop, that is, on the right side in the traveling direction with respect to the counterclockwise line integration.

Generally, n (x, y) · m (x, y)
= Nxmx + nymy. Where n (x, y) =
(Nx, ny), m (x, y) = (mx, my). However, in this example, n (x, y) is (1,0),
Since (0, 1), (-1, 0) or (0, -1), n (x, y) · m (x, y) is mx, my,
Since it becomes either -mx or -my, and the integration is an addition, it can be expressed by the following Expression 9.

(Equation 9) However, it represents number k = 1, 2, 3 in FIG. 8, the vector of the point corresponding to ... 14 m i, is m i = (mxk, myk) with.

Next, the case where M (t) is obtained from Expression 8 will be described. In Equation 8, ▽ · m is the following Equation 1
It becomes 0.

(Equation 10) Equation 11 is obtained by transforming Equation 10 into a difference form.

[Equation 11] Here, if the integration is converted to addition, the following equation 12 is obtained.

(Equation 12) However it becomes as shown in this case number 1 3.

(Equation 13) Therefore, Equation 12 is eventually the same as Equation 9.

Next, a modified example of Expression 7 will be described. This modified example shows that even when M (t) is obtained by Expression 7, it can be reduced to a partial local calculation algorithm.

As shown in FIG. 9, assuming a large closed loop C, calculating the above equation 7 is the same as described above. However, as shown in FIG.
It is known that similar results can be obtained by addition of 1, C2, C3,... Cn. This can be shown by Equation 14.

[Equation 14] FIG. 10 shows the discrete form of FIG. 9 described above. In this case, the integral may be defined as in the following Expression 15.

(Equation 15) This selection method is a selection method in which common points are canceled when values of adjacent loops are added. Therefore, the following equation 16 may be calculated.

(Equation 16) These methods are effective when a parallel processing device is assumed.

Next, a method for obtaining the signal P (t) will be described with reference to FIG. FIG. 3 shows a circuit of the processing unit 5 shown in FIG. 1 for obtaining the signal P (t).

In FIG. 3, reference numeral 9 denotes an input terminal through which the image data I shown in FIG.
(T) is a high-pass filter 10 and a power detection circuit 11
Respectively.

The high-pass component of the image data I (t) is extracted by the high-pass filter 10, and the extracted high-frequency component is supplied to the power detection circuit 12.

The power detection circuit 11 squares the image data I (t) from the input terminal 9, performs integration (spatial integration), and supplies the result of this processing to the division circuit 13.

The power detection circuit 12 is a high-pass filter 1
The high-frequency component of the image data I (t) from 0 is squared, integrated (spatial integrated), and the result of this processing is supplied to the division circuit 13.

The division circuit 13 calculates P _H / (P−P _H ) based on the signal P from the power detection circuit 11 and the signal P _H from the power detection circuit 12 to obtain P (t). The signal P (t) is supplied to the determination unit 6 shown in FIG.

This P (t) is subjected to a Fourier transform of, for example, image data I (t), and the power P _L of the low-frequency component is obtained as follows:
Seeking by the ratio P _H / P _L of the high frequency component P _H is the same.

Next, the function of the judgment unit 6 shown in FIG. 1 will be described. The determination unit 6 generates a control signal for the video camera 1 to perform autofocus based on the signals P (t) and M (t) described above.

First, ΔP (t) = P (t) −P (t−
As 1), a combination of P (t-1), P (t) and M (t) is considered. In this case, as combination 1, ΔP (t)
<0, M (t) <0, ΔP (t) <
0, M (t) ≧ 0, as combination 3, ΔP (t)> 0,
M (t) <0, as combination 4 ΔP (t)> 0, M
Four combinations of (t) ≧ 0 are considered (ΔP (t) =
When it is 0, it is assumed that the lens is in focus).

As described above, if M (t) <0, the distance between the subject and the camera 1 can be determined to be short, and if M (t)> 0, the distance between the subject and the camera 1 can be determined to be long. Thus, an optimum control signal can be generated for the combinations 1 to 4.

FIG. 5 shows the video camera 1 and a subject. Auto-focusing by the judgment unit 6 will be described with reference to FIG.

In FIG. 5, for example, the video camera 1
The case where the light detection unit 1a (CCD element) is moved by the above-described control signal to perform auto-focusing is shown.
FIG. 5A shows a state in which the subject is in focus, FIG. 5B shows a state in which the distance between the subject and the light detection unit 1a of the video camera 1 is short,
FIG. 5C shows a state in which the distance between the subject and the light detection unit 1a of the video camera 1 is long.

The case of combination 1 described above corresponds to FIG. 5B. In this case, as shown in FIG. 5B, the light detection unit 1a is moved in the plus direction (rearward) by the control signal, and 1a 'indicated by a broken line. And focus.

The case of the combination 2 corresponds to FIG. 5C. In this case, as shown in FIG. 5C, the control signal moves the light detecting section 1a in the minus direction (forward), and Position and focus.

It is determined that each of the combinations 3 and 4 is in focus, and the light detection unit 1a is controlled by a control signal.
May not be moved. For example, in the case of the combination 3, it is determined that focusing is in progress because the distance between the subject and the light detection unit 1a of the video camera 1 is short, and furthermore, as shown in FIG. 5B As described above, the light detection unit 1a may be moved at a high speed in the plus direction by the control signal so as to focus. In the case of the combination 4, it is determined that the subject is in focus because the distance between the subject and the light detection unit 1a of the video camera 1 is long, and further, as shown in FIG. 5C,
In response to the control signal, the light detection unit 1a may be moved at a high speed in the minus direction to focus.

Next, referring to the flowchart of FIG.
A processing operation for autofocus by the processing unit 5 and the determination unit 6 will be described.

First, in step 100, the image data I
Enter (t). That is, the image data I (t) is read from the memory 3 shown in FIG.
(T) is supplied to the processing unit 5. And step 110
Move to

In step 110, the motion vector m (i, t) is calculated by the above-described optical flow, and P (t) as contrast information is calculated.
Then, control goes to a step 120.

In step 120, the integration path or the area corresponding to the area is determined. Then, control goes to a step 130. Then, control goes to a step 130.

In step 130, the size M
Calculate (t). Then, control goes to a step 140.

In step 140, the above-mentioned contrast ΔP (t) is calculated by ΔP (t) = P (t) -P (t-1)
When ΔP (t) <0, the routine proceeds to step 160, and when ΔP (t) ≧ 0, the routine proceeds to step 150.

In step 150, the control amount is set to "0". This is because when .DELTA.P (t) is "0", the object is in focus, and when .DELTA.P (t) exceeds "0", the contrast is improved due to illumination. That is, an increase in P (t) in a focused state means that illumination or the like has improved. Then, control goes to a step 160.

In step 160, the control amount M
(T) Calculate ΔP (t). Then, control goes to a step 170.

In step 170, the control amount data is supplied to the video camera 1. And step 180
Move to

At step 180, "1" is added to t. Then, control goes to a step 190.

In step 190, it is determined whether or not the processing is to be terminated.
If “YES”, the process ends. If “NO”, the process returns to step 100. The determination as to whether or not to end may be based on, for example, pressing any switch of the video camera 1 or, for example, switching from automatic to manual.

As described above, in the present example, the size of the image data of at least two
(T) and the contrast difference P between the two screens
(T) is determined, and only when ΔP (t) is smaller than “0”, the control amount is determined based on M (t) and ΔP (t), and the focus of the video camera 1 is controlled based on the control amount. Therefore, the auto-focusing operation can be naturally and well followed by a simple configuration without newly installing a dedicated sensor or the like. The feeling of use can be improved.

[0061] In the example described above, the processing unit 5 an image data I (t) and I (t-1) from the motion vector m (x, y) has been determined and further the m (x, y) M
The values at (x, y, t) and t can be easily used for estimating m (x, y) = m (x, y, t + 1) at the next time t + 1, or for filtering.

The method of obtaining the motion vector m (x, y) is not limited to the Area-Based optical flow, but may be, for example, an optical flow that focuses on edges, or a method of searching for corresponding points by blocking using a correlation method. Etc.

The method of selecting the line integration path of the above equation (6) as a boundary (contour) is effective.

Further, as described above, the judgment unit 6 judges only plus and minus. However, assuming that the control signal itself is a continuous value, for example, it is considered that the control signal itself is not only plus and minus. May be.

Further, the present invention is not limited to the above-described embodiment, and may take various other configurations without departing from the gist of the present invention.

[0066]

According to the present invention described above, when the contrast of the video signal is low, the motion vector is closed-circuit integrated from the video signal of at least two screens, and the perspective data of the subject obtained based on the processing result is obtained. Based on the above, the moving direction and amount of the lens of the image pickup means are determined and the focus of the image pickup means on the subject is controlled, so that the auto focus can be performed with a simple configuration without newly installing a dedicated sensor or the like. When applied to a video camera or the like, for example, there is an advantage that the usability for the user can be improved when the follow-up of the operation is made natural and good.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing an embodiment of a camera control device of the present invention.

FIG. 2 is an explanatory diagram showing functions of a processing unit for explaining an embodiment of the camera control device of the present invention.

FIG. 3 is an explanatory diagram showing functions of a processing unit for explaining an embodiment of the camera control device of the present invention.

FIG. 4 is a diagram for describing an embodiment of a camera control device according to the present invention;

FIG. 5 is an explanatory diagram showing functions of a determination unit for explaining an embodiment of the camera control device of the present invention.

FIG. 6 is a flowchart for explaining one embodiment of the camera control device of the present invention.

FIG. 7 is an explanatory diagram of camera control for explaining an embodiment of the camera control device of the present invention.

FIG. 8 is an explanatory diagram of camera control for explaining an embodiment of the camera control device of the present invention.

FIG. 9 is an explanatory diagram of camera control for explaining an embodiment of the camera control device of the present invention.

FIG. 10 is an explanatory diagram of camera control for explaining an embodiment of the camera control device of the present invention.

FIG. 11 is a graph for explaining a conventional auto focus.

FIG. 12 is a graph for explaining a conventional auto focus.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Video camera 3 Memory 4 Memory 5 Processing part 6 Judgment part 7 Monitor 8 Recording part

──────────────────────────────────────────────────続 き Continued on front page (58) Field surveyed (Int.Cl. ⁷ , DB name) H04N 5/232 G02B 7/28

Claims (1)

(57) [Claims] A processing means for performing a closed-loop integration of a motion vector from at least two screens of video signals when the contrast of a video signal from an imaging means for imaging a subject is small;
The perspective data of the subject is obtained based on the processing result from the processing unit, and the moving direction and amount of the lens of the imaging unit are determined based on the perspective data to control the focus of the imaging unit on the subject. A camera control device comprising: