KR101592125B1 - Target tracing method and apparatus - Google Patents

Abstract

Disclosed are a method and an apparatus for tracing a target. The method for tracing a target comprises: (a) a step of using an optical flow with respect to a previous frame and a current frame to calculate a motion vector of a target; (b) a step of using an error between a forward motion vector and a backward motion vector with respect to the motion vectors to remove a first outlier motion vector among the motion vectors; (c) a step of using a median value with respect to an X-axis change amount and a Y-axis change amount of the motion vectors from which the first outlier motion vector is removed to remove a second outlier motion vector to derive an inlier motion vector; (d) a step of using the inlier motion vector to calculate a target size change rate; and (e) a step of using the target size change rate to estimate the size of the target.

Description

[0001] TARGET TRACKING METHOD AND APPARATUS [0002]

The present invention relates to a target tracking method and apparatus capable of robustly estimating a target size using a motion vector.

In object tracking, estimating the size of the target is one of the most important problems for extracting the characteristic information of the target, excluding the background as much as possible. Generally, a target tracking method based on a motion vector estimates the size of a target using motion vectors. In a motion vector calculated by an optical flow or the like, an ideal motion vector is included stochastically to estimate a target size A filtering process is required to properly remove the motion vector beforehand.

However, in the case of rapid target motion or sudden illumination change, the abnormal motion vector may increase greatly. If the abnormal motion vector is not sufficiently filtered in the filtering process, the error is accumulated in the target size estimation, There is a problem to be done.

The present invention is intended to provide a target tracking method and apparatus capable of effectively removing outlier motion vectors even in the case of rapid target motion or sudden illumination change.

Accordingly, it is an object of the present invention to provide a target tracking method and apparatus capable of accurately estimating a target size even in the case of rapid target movement or sudden illumination change.

According to an aspect of the present invention, there is provided a target tracking method capable of effectively removing an outlier motion vector even in the case of rapid target movement or sudden illumination change.

According to an embodiment of the present invention, there is provided a method of generating a motion vector, comprising: (a) calculating a motion vector of a target using an optical flow for a previous frame and a current frame; (b) removing an outlier motion vector of the motion vector using an error between the forward and backward motion vectors for the motion vectors; (c) deriving an inlier motion vector by removing a second-order or higher-order motion vector using an intermediate value of the X-axis variation and the Y-axis variation of the motion vector with the first-order and higher-order motion vectors removed; (d) calculating a target size change rate using the fit motion vector; And (e) estimating the size of the target using the target size change rate.

The step (c) may include calculating an intermediate value of the X-axis variation amount and the Y-axis variation amount for each of the motion vectors from which the first-order and higher-order motion vectors have been removed, Setting a reference vector using the calculated intermediate value of the X-axis variation and the Y-axis variation; Determining a motion vector having a difference of more than a predetermined magnitude from the reference vector among the motion vectors having the first-order and higher-order motion vectors removed; And deriving the motion vector from which the secondary abnormal motion vector has been removed as a suitable motion vector.

The removing of the secondary abnormal motion vector may include removing the secondary abnormal motion vector using a Median Absolute Deviation filter (MAD) based on the calculated intermediate value between the X-axis change amount and the Y- Can be removed.

In the step (d), the target size change rate may be calculated using the distance between the start point coordinate and the end point coordinate of the adaptive motion vector.

(E) applying the target size of the previous frame to the current frame if the target size change rate is greater than or equal to a threshold value; And estimating a target size for the current frame using the target size change rate if the target size change rate is less than the threshold.

(B) comprises: calculating a backward motion vector for each of the motion vectors; And removing the motion vector having a difference between the end point coordinates of the backward motion vector and the start point coordinates of each motion vector by a second or more motion vector.

In step (b), the step of retrieving a boundary box (patch) of the target of the previous frame and a boundary box (patch) most similar to the current frame, before removing the outlier motion vector, The motion vector included in the most similar bounding box may be removed from the motion vector of the first order or higher.

According to another aspect of the present invention, there is provided an apparatus capable of effectively removing an outlier motion vector even in the case of rapid target motion or sudden illumination change.

According to an embodiment of the present invention, a motion vector calculation unit calculates a motion vector of a target using an optical flow for a previous frame and a current frame; A first filter for removing an outlier motion vector of the motion vector using an error between forward and backward motion vectors for the motion vectors; A second filter unit for deriving an inlier motion vector by removing a second-order or higher-order motion vector using an intermediate value of the X-axis change amount and the Y-axis change amount of the motion vector with the first-order and higher-order motion vectors removed; And an estimator for estimating the target size using the target magnitude change rate by calculating the target magnitude change rate using the adaptive motion vector.

Wherein the second filter calculates an intermediate value of the X-axis variation amount and the Y-axis variation amount for each of the motion vectors from which the first-order and higher-order motion vectors have been removed, calculates an intermediate value between the calculated X- A motion vector having a difference of a predetermined magnitude or more from the reference vector is determined as a second-order motion vector or more, and the resultant motion vector is derived as a suitable motion vector .

The estimator may calculate the target size change rate using the distance between the start point coordinate and the end point coordinate of the adaptive motion vector.

Wherein the estimating unit applies the target size of the previous frame to the current frame if the target size change rate is equal to or greater than a threshold value and if the target size change rate is less than the threshold, Size can be estimated.

By providing the target tracking method and apparatus according to an embodiment of the present invention, there is an advantage that the outlier motion vector can be effectively removed even in the case of rapid target movement or sudden illumination change.

Therefore, the present invention has an advantage of accurately estimating the target size even in the case of rapid target movement or sudden illumination change.

1 is a flowchart showing a target tracking method according to an embodiment of the present invention;
FIG. 2 is a view for explaining a target tracking process according to an embodiment of the present invention; FIG.
FIG. 3 and FIG. 4 are diagrams showing images obtained by comparing results obtained according to the conventional and the target tracking using motion vectors according to an embodiment of the present invention. FIG.
FIG. 5 schematically illustrates an internal configuration of a target tracking apparatus according to an embodiment of the present invention; FIG.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. In addition, numerals (e.g., first, second, etc.) used in the description of the present invention are merely an identifier for distinguishing one component from another.

Also, in this specification, when an element is referred to as being "connected" or "connected" with another element, the element may be directly connected or directly connected to the other element, It should be understood that, unless an opposite description is present, it may be connected or connected via another element in the middle.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a flowchart illustrating a target tracking method according to an embodiment of the present invention. FIG. 2 is a view for explaining a target tracking process according to an embodiment of the present invention. FIG. 6 is a graph illustrating a result of a target tracking using a motion vector according to an exemplary embodiment of the present invention.

In step 110, the target tracking device 100 calculates the motion vector within the bounding box of the target using an optical flow.

The present invention is for accurately estimating a target size by effectively removing an abnormal motion vector in a target tracking. Therefore, a method of deriving a motion vector of each pixel using an optical flow in a video sequence is a well-known technique, and a description of a method of calculating an optical flow-based motion vector, which may obscure the present invention, It will be omitted.

In FIG. 2, a result of estimating a motion vector for each pixel in a boundary box of a target is shown using an optical flow based on a previous frame and a current frame.

As shown in 210 of FIG. 2, the estimated motion vectors based on the optical flows are very diverse, and the direction and magnitude of the motion vectors are estimated to be very various. Due to the optical flow characteristics, when there is a large change in motion of the target in the previous frame and the current frame, or when the illumination change is abrupt, the abnormal motion vector included in the estimated motion vector is further increased.

If the target is estimated correctly, the motion vectors for the target calculated using the optical flow will have a certain tendency. Therefore, the motion vector that violates the tendency among the calculated motion vectors causes errors in the target tracking, and therefore must be effectively removed.

Then, in step 115, the target tracking apparatus 100 removes the abnormal motion vector from the motion vector calculated using NCC (Normalized Cross Correlation) and FB (Forward and Backward) filters.

For example, the target tracking apparatus 100 can identify a patch (boundary box) that is most similar to a patch of a previous frame including a motion vector calculated using an NCC & FB filter, in the current frame. The target tracking device 100 may then remove the abnormal motion vector among the calculated motion vectors in the specified patch.

In order to remove the ideal motion vector, the target tracking device 100 may calculate the motion vector in the reverse direction based on the end point of each motion vector in the specified patch.

In order to facilitate understanding and explanation, each motion vector in a specified patch (i.e., between a previous frame and a current frame) is referred to as a forward motion vector, and a motion vector calculated based on the end point of the forward motion vector is referred to as a backward motion Quot; vector ".

The target tracking apparatus 100 compares the forward motion vector with the backward motion vector, and when an error above the reference value occurs, the target motion vector is determined to be an abnormal motion vector and removed.

For example, assuming that the starting point of the motion vector is A and the ending point is B, as will be described in more detail with reference to FIG. If the error between the calculated motion vector end point (C) and the start point (A) of the motion vector is more than the reference value as a result of calculating the motion vector again based on the end point (B) of the motion vector, Can be removed.

The result of removing the abnormal motion vector in the motion vector by applying the NCC & FB filter is illustrated in 220 of FIG.

The NCC & FB filter may be adjusted to determine the ratio of the motion vector to be removed (i.e., the ratio of the motion vector to be remained) to the motion vector.

As shown in 220 of FIG. 2, by applying the NCC & FB filter, it is possible to effectively remove about 50% of the motion vector having the abnormal motion in the motion vector.

In step 120, the target tracking apparatus 100 finally removes the abnormal motion vector using a median absolute deviation (MAD) filter.

The median absolute deviation (MAD) is a robust measure of the daily sample volatility of quantitative data and can be derived using Equation (1).

는 MAD 함수를 나타내고,

는 양적 자료의 요소(즉, 모션 벡터 요소)를 나타내고,

는 중간값(median) 함수를 나타낸다. 또한,

은

의 개수를 나타낸다.here,

Represents a MAD function,

Represents an element of quantitative data (i.e., a motion vector element)

Represents a median function. Also,

silver

≪ / RTI >

According to Equation (1), the median absolute deviation can be defined as an intermediate value of the absolute deviation generated as the median of the motion vector set.

Based on the above-described median absolute deviation (MAD), the MAD filter can be expressed by Equation (2).

는 MAD 필터를 나타내고,

는 NCC & FB 필터로 필터링된 모션 벡터를 나타낸다.here,

Represents a MAD filter,

Represents the motion vector filtered by the NCC & FB filter.

이 1보다 작거나 같으면, MAD 필터는 해당 모션 벡터를 적합(inlier) 모션 벡터로 분류할 수 있다. 그러나 만일

이 1 보다 크면, MAD 필터는 해당 모션 벡터를 이상 모션 벡터로 분류하여 제거할 수 있다.E.g,

Is less than or equal to 1, the MAD filter can classify the motion vector into an inlier motion vector. However,

Is greater than 1, the MAD filter can classify the motion vector into an abnormal motion vector and remove the motion vector.

In this manner, the MAD filter is used to filter again the motion vector filtered by the NCC & FB filter (i.e., the residual motion vector after the first-order motion vector is removed) based on the median value of the absolute deviation generated as the median It is possible to effectively remove an abnormal motion vector.

As shown in FIG. 2, only the reference vector having the intermediate value of the X-axis and Y-axis variation among the motion vectors having the first-order and higher-order motion vectors removed and the motion vector having a small error can be selectively classified into the suitable motion vectors. have.

There is an advantage in that the target can be accurately tracked by using only such a suitable motion vector for the target size estimation.

In step 125, the target tracking device 100 calculates a target size change rate using the suitable motion vectors from which the abnormal motion vector has been removed. The remaining motion vectors (i.e., the suitable motion vectors) after the second-order or higher-order motion vector removal may be at least two.

In the present specification, the adaptive motion vector refers to a motion vector remaining after the abnormal motion vector is removed by the NCC & FB filter and the MAD filter among the motion vectors calculated using the optical flow in the patch (boundary box) of the target.

The target tracking apparatus 100 can calculate the target size change rate using the distance between the start point coordinate and the end point coordinate of the suitable motion vector.

For example, the target tracking apparatus 100 may calculate the target size change rate using the following equation (3).

는 적합 모션 벡터간에 계산된 표적 크기 변화율을 나타내고,

,

는 각각 적합 모션 벡터를 나타내고,

는

의 시작점을 나타내고,

는

의 끝점을 나타내며,

는

의 시작점을 나타내고,

는

의 끝점을 나타낸다.here,

Represents the calculated target size change rate between the best fit motion vectors,

,

Respectively represent a suitable motion vector,

The

, ≪ / RTI >

The

≪ / RTI >

The

, ≪ / RTI >

The

.

는 거리 함수로, 두 모션 벡터 좌표간의 거리는 하기 수학식 4를 이용하여 각각 계산될 수 있다.

Is a distance function, and the distance between two motion vector coordinates can be calculated using Equation (4) below.

는

의 좌표를 나타내며,

는

의 좌표를 나타낸다.here,

The

Lt; / RTI >

The

.

In step 130, the target tracking device 100 estimates the target size using the calculated target size change rate.

For example, the target tracking apparatus 100 may verify the reliability of the target size change rate, and if the reliability is higher than the reference value, the target size of the previous frame may be used as it is in the current frame.

However, if the reliability verification result of the target size change rate is less than the reference value, the target tracking apparatus 100 can estimate the target by calculating the target size for the current frame using the target size change rate.

For example, the target tracking apparatus 100 may verify the reliability of the target size change rate using Equation (5).

는 검증 함수를 나타내며, T는 기준치를 의미한다.here,

Denotes a verification function, and T denotes a reference value.

In one embodiment of the present invention, if the reference value is set to 0.1, and thus the confidence level of the target size change rate is smaller than 1, the target tracking apparatus 100 can calculate the target size for the current frame using the target- have.

However, if the reliability verification result for the target size change rate is greater than or equal to 1, the estimated target size change rate changes very rapidly, so that the target tracking device 100 determines that the calculated target size change rate is unreliable The target size of the previous frame is applied to the current frame as it is.

FIG. 3 compares the result of estimating the target size based on the motion vector according to the conventional Kalal and the present invention.

FIG. 3 shows a result of estimating a target size based on a motion vector using a conventional Kalal method, and 320 shows a result of estimating a target size based on a motion vector according to an embodiment of the present invention will be.

The example of FIG. 3 compares the case where the motion of the inter-frame target occurs rapidly, and compares the result of tracking some of the biker sequences and tracking the target.

The bounding box in FIG. 3 is the head of the biker, and both the Kalal method and the proposed method show images of traces of frames 86, 89, and 92. FIG.

The reason why the bounding box indicating the biker's head position and size in frame 86 differs slightly from the Kalal and proposed method is due to the difference in the calculated results as the tracking progresses from the first frame to the frame 86.

When the target tracking is progressed to the frame 89 and the frame 92, the Kalal method shows a problem that the error of the motion vector is accumulated and the frame 92 estimates a much larger range than the initial target as the target size.

On the other hand, the target tracking method according to an embodiment of the present invention can accurately estimate the head size of the biker stably even in the frames 89 and 92.

FIG. 4 is a view illustrating an image obtained by estimating the size of a target using a motion vector according to an embodiment of the present invention, in a frame in which a sudden illumination change occurs.

In Fig. 4, the head of the singer is set as the target, and Kalal's method and the proposed method are compared with the images of the frames 57 and 58, respectively.

In the frame 58, both the Kalal method and the proposed method show that the abnormal motion vector is generated more than the calculated motion vector due to the illumination change.

However, Kalal's method estimates a size much larger than the size of the target due to the ideal vector, whereas the proposed method estimates that the estimated target size keeps the size of the frame 57, .

As described above, it can be seen that the target tracking method according to the embodiment of the present invention can effectively track the target even when the movement of the target is large or sudden illumination change occurs.

5 is a schematic view illustrating an internal configuration of a target tracking apparatus according to an embodiment of the present invention.

5, a target tracking apparatus 100 according to an embodiment of the present invention includes a calculation unit 510, a first filter 515, a second filter 520, an estimation unit 525, a memory 530 And a processor 535.

The calculation unit 510 calculates the motion vector for the target using the optical flow for the previous frame and the current frame.

Since the method of calculating the target motion vector based on the optical flow itself is already obvious to those skilled in the art, a detailed description thereof will be omitted.

The first filter 515 functions to remove an abnormal motion vector among the motion vectors.

For example, the first filter 515 may be an NCC and an FB filter.

That is, the first filter 515 finds the patch most similar to the boundary box (patch) of the target of the previous frame in the current frame, and detects the motion of the abnormal frame by using the error between the forward and backward motion vectors, You can remove the vector.

In one embodiment of the present invention, the forward motion vector refers to each motion vector derived by the calculation unit 510. On the other hand, the backward motion vector refers to the motion vector calculated by going back to the previous frame in the current frame based on the end point coordinates of each forward motion vector.

Accordingly, when the difference between the start point of the forward motion vector and the end point of the backward motion vector is greater than or equal to the reference value, the first filter 515 determines that the motion vector is abnormal and removes the motion vector as an abnormal motion vector .

About 50% of the motion vector can be removed by the first filter 515 with the abnormal motion vector.

The second filter 520 removes the abnormal motion vector based on the intermediate value of the X-axis and Y-axis variation among the motion vectors from which the abnormal motion vector has been removed by the first filter 515.

That is, by using the second filter 520, the abnormal motion vector for the motion vector can be removed secondarily.

For example, the second filter 520 may be a MAD filter. Since the MAD filter is the same as that described in detail with reference to FIG. 1, a duplicate description will be omitted.

In addition, the second motion vector is removed by the second filter 520, and the remaining motion vector is referred to as a suitable motion vector.

The estimator 525 calculates a target size change rate using a suitable motion vector, and estimates a target size using the calculated target size change rate. This is the same as that described in detail with reference to FIG. 1, so that redundant description will be omitted.

The memory 530 stores various algorithms necessary for performing the method of estimating the target size by removing the abnormal motion vectors of the motion vectors over two steps according to an embodiment of the present invention, Function.

Processor 535 may be coupled to internal components (e.g., computation unit 510, first filter 515, second filter 520, and subtractor) of target tracking apparatus 100 in accordance with an embodiment of the present invention. The memory 525, the memory 530, and the like).

The target tracking method according to the present invention described above can be implemented as a computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording media storing data that can be decoded by a computer system. For example, it may be a ROM (Read Only Memory), a RAM (Random Access Memory), a magnetic tape, a magnetic disk, a flash memory, an optical data storage device, or the like. In addition, the computer-readable recording medium may be distributed and executed in a computer system connected to a computer network, and may be stored and executed as a code readable in a distributed manner.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the following claims And changes may be made without departing from the spirit and scope of the invention.

100: Target tracking device
510:
515: first filter
520: second filter
525:
530: Memory
535: Processor

Claims (12)

(a) calculating motion vectors of a target using an optical flow for a previous frame and a current frame;
(b) removing an outlier motion vector of the motion vectors using an error between forward and backward motion vectors for each of the motion vectors;
(c) deriving an inlier motion vector by removing a second-order or higher-order motion vector by using an intermediate value of the X-axis change amount and the Y-axis change amount of the motion vectors with the first-order and higher-order motion vectors removed;
(d) calculating a target size change rate using the fit motion vector; And
(e) estimating the size of the target using the target size change rate.
The method according to claim 1,
The step (c)
Calculating an intermediate value of an X-axis variation amount and a Y-axis variation amount for each of the motion vectors from which the first-order and higher-order motion vectors have been removed;
Setting a reference vector using the calculated intermediate value of the X-axis variation and the Y-axis variation;
Determining a motion vector having a difference of more than a predetermined magnitude from the reference vector among the motion vectors having the first-order and higher-order motion vectors removed as a second-order motion vector and removing the motion vector; And
And deriving a motion vector from which the secondary ideal motion vector is removed as a suitable motion vector.
3. The method of claim 2,
The step of deriving the inlier motion vector by removing the secondary ideal motion vector comprises:
Wherein the second over-the-order motion vector is removed using a Median Absolute Deviation filter (MAD) based on the calculated intermediate value between the X-axis change amount and the Y-axis change amount.
The method according to claim 1,
The step (d)
Wherein the target size change rate is calculated using the distance between the start point coordinate and the end point coordinate of the adaptive motion vector.
The method according to claim 1,
The step (e)
Applying a target size of the previous frame to the current frame if the target size change rate is greater than or equal to a threshold; And
And estimating a target size for the current frame using the target size change rate if the target size change rate is less than the threshold.
The method according to claim 1,
The step (b)
Calculating a backward motion vector for each of the motion vectors,
Wherein the step (c) includes the step of removing a motion vector having a difference between the end point coordinates of the backward motion vector and the start point coordinates of each of the motion vectors to a second or more motion vector.
delete A computer-readable recording medium on which program codes for carrying out the method according to any one of claims 1 to 6 are recorded.
A motion vector calculation unit for calculating target motion vectors using an optical flow for a previous frame and a current frame;
A first filter that removes a first outlier motion vector of the motion vector using an error between forward and backward motion vectors for each of the motion vectors;
A second filter unit for deriving an inlier motion vector by removing a second-order or higher-order motion vector by using an intermediate value of the X-axis change amount and the Y-axis change amount of the motion vectors with the first-order and higher-order motion vectors removed; And
And an estimator for estimating the target size using the target magnitude change rate, calculating a target magnitude change rate using the suitable motion vector, and estimating the target magnitude using the target magnitude change rate.
10. The method of claim 9,
The second filter unit includes:
Calculating a median value of the X-axis variation amount and a Y-axis variation amount for each of the motion vectors with the first-order and higher-order motion vectors removed, and calculating a median value of the Y-axis variation using the median value of the calculated X- A motion vector having a difference of a predetermined magnitude or more from the reference vector among the motion vectors having the first order or higher order motion vectors removed is determined as a second order or higher order motion vector and is derived as a suitable motion vector. Target tracking device.
10. The method of claim 9,
Wherein the estimating unit comprises:
And calculates the target size change rate using the distance between the start point coordinate and the end point coordinate of the adaptive motion vector.
10. The method of claim 9,
Wherein the estimating unit comprises:
Applying the target size of the previous frame to the current frame if the target size change rate is greater than or equal to a threshold,
And when the target size change rate is less than the threshold value, estimates the target size for the current frame using the target size change rate.

Images