JPH0528269A - Hourly change extraction method for area shape - Google Patents

Abstract

PURPOSE:To supply the hourly change extraction method of an area shape, which can roughly capture the deformation of an object in a moving picture, especially the movement of a part constituting a shape without being affected by the micro change of an outline owing to noise or the like at the time of inputting the moving picture. CONSTITUTION:A thinning processing means 2 generates a thinning picture by cutting the shape outline of a closed area in a moving picture frame outputted from a picture input device 1 from a periphery. A characteristic point extraction means 3 extracts a characteristic point on a thin line thinned by the thinning processing means 2. A characteristic point corresponding means 4 executes correspondence on the characteristic point between two frames extracted in the characteristic point extraction means 3. A calculation means calculates the moving amount of the characteristic point based on the characteristic point which is made correspondence and calculates the moving amount of the point on the thin line. When a known point exists in the closed area, the moving amount of the other point is considered to be equal to the moving amount of the point on the thin line, which can arrive at a shortest distance within the closed area so as to calculate it.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for extracting a change in the shape of a region for extracting the amount of deformation of a closed region existing in a moving image for each frame.

A field of application of this method is an apparatus for recognizing an object in a moving image. The recognition of an object in an image can be performed by comparing the shape feature of the model representing a region representing the object, and the feature of the region shape extracted by this method is used to identify the object and the model. By making a comparison, more accurate object recognition becomes possible.

[0003]

2. Description of the Related Art Conventionally, extraction of a deformation amount between frames in a closed region has been performed by obtaining the deformation of the contour of the closed region. In this change amount extraction method, the contour line of the closed region is first extracted for each frame, and the feature points of this contour line are extracted.
As the characteristic points, for example, vertices when the contour line is approximated by a polygonal line, points of maximum curvature, non-differentiable points, etc. are used.

Then, feature points are associated with each other between two adjacent frames in the moving image, and the amount of movement of the corresponding feature points between the two frames is calculated. Since the movement amount of the points adjacent to each other on the contour line should change smoothly, the movement amount of the points existing on the contour line between the feature points is determined by interpolating the feature point movement amounts of both end points. The amount of deformation of the area is
It is defined by the amount of movement of points on this contour line.

[0005]

However, in the above-described conventional deformation amount extraction method, since the processing is performed only near the shape contour, it is sensitive to the shape deformation due to noise or the like. I will end up. Further, in object recognition in a moving image, it is more important to roughly capture the movement of a part forming a shape than the minute temporal deformation of the contour of the shape. Therefore, the conventional method is not suitable for recognizing object deformation in moving images.

An object of the present invention is to be able to roughly capture the deformation of an object in a moving image, particularly the movement of a part constituting a shape, without being affected by a minute change in a contour line due to noise or the like when a moving image is input. An object of the present invention is to provide a method for extracting a change in area shape over time.

[0007]

In order to solve the above-mentioned problems, the present invention has the following configuration as a method for extracting a change in area shape over time. FIG. 1 is a flowchart of the principle of the present invention, and FIG. 2 is a device configuration diagram for realizing the present invention. The present invention will be described with reference to FIGS.

The image input device 1 outputs a binarized moving image for each frame to the thinning processing means 2. The thinning processing unit 2 creates a thin line image by cutting the shape contour of the closed region in the moving image frame input from the image input unit 1 from the periphery.

The feature point extraction means 3 is the thinning processing means 2
The feature points on the thin line thinned by are extracted. The feature point correspondence means 4 associates the feature points between the two frames extracted by the feature point extraction means 3.

The calculation means 5 calculates the movement amount of the feature points and the movement amount of the points on the thin line based on the feature points associated by the feature point correspondence means 4, and the movement amount is calculated in the closed area. When a known point exists, the movement amount of the other point is estimated and calculated as being equal to the movement amount of the point on the thin line that can be reached in the shortest distance along the closed region.

Further, the calculation means 5 calculates the amount of movement of the point inside the closed region and outputs this amount of movement to the image output device 6 as the amount of deformation of the region part. In addition, the calculation means 5 is 2
When the correspondence of the feature points between the frames is known, the correspondence between the frames on the thin line section sandwiched by the feature points is estimated, and the movement amount of the points on the thin line section is calculated based on this estimation result.

[0012]

According to the present invention, the following effects are exhibited. FIG. 3 is a diagram for explaining the operation of the present invention. The processing target of the present invention is a moving image in which the closed region is binarized to 1 and the background to 0 in each frame as shown in FIG. 3A, and the closed region in the moving image is temporally transformed. In order to extract, the thin line of the shape of the closed region is used instead of using the contour of the region shape.

By the thinning processing means 1, the pixels forming the closed region are sequentially scraped off from the contour of the shape, and FIG.
As shown in, a linear thin line with a pixel width of 1 is created. The feature point extracting means 3 extracts feature points such as end points of fine lines, branches and intersections from the area shape thinned for each frame as shown in FIG. The feature points are associated between two adjacent frames.

Next, the movement amount calculating means 5 calculates as follows. As shown in FIG. 3D, of the points in the closed region, the movement amount of the feature point is calculated based on the association result. The amount of movement of points on the thin line section sandwiched by the characteristic points is calculated based on the correspondence result of the characteristic points as shown in FIG. As shown in FIG. 2F, the amount of movement between two frames of a point that constitutes the closed area and does not exist on the thin line has a path that passes through that point and the inside of the closed area, and the path length is Since it is determined to be equal to the movement amount of the point on the thin line that is the minimum, and the movement amount inside the closed area is calculated, the moving image is not affected by the minute change of the contour line due to noise etc. at the time of input to the moving image, It is possible to roughly capture the deformation of an object in an image, in particular, the movement of a part forming a shape.

Further, the amount of movement of the points forming the closed region between two frames is output to the image output device as the amount of deformation of the region in the moving image, and an image in which a color is assigned to a region part according to the amount of deformation. By creating and displaying a color image, the amount of deformation can be identified by color.

Further, when the correspondence of the feature points between the two frames is known by the calculation means 5, the correspondence between the frames of the points on the thin line section sandwiched by the feature points is estimated, and the thin line section is based on this estimation result. Since the movement amount of the upper point is calculated, the movement amount of the point on the closed region can be easily calculated.

[0017]

Embodiments of the present invention will be described in detail with reference to the drawings. FIG. 4 is a flowchart in the embodiment of the present invention, and FIG. 5 is a device configuration diagram in the embodiment. The present apparatus includes an image input unit 1 that outputs a moving image for each frame, and a region shape that extracts a moving amount of a region as a time change amount inside a closed region based on a moving image for each frame input from the image input unit 1. It is composed of a time change extraction unit 7 and an image display unit 6 that displays the amount of movement of a part as an image.

The image input section 1 sequentially creates digital imaged frames at a time (t = 1, 2 ...) At which a moving object is photographed by a moving image input device 11 such as a camera. In this frame, a binary image B (t) in which the pixel corresponding to the background has a pixel value of 0 and the pixel corresponding to the object shape has a pixel value of 1 by the background removing unit 12 including a digital filter, for example.
(See FIG. 3A) and sent to the region shape time change extraction unit 7.

The area shape time change extraction unit 7 includes a thinning processing unit 2, a feature point extraction unit 3, a feature point correspondence unit 4, a movement amount calculation unit 5, and an image display unit 6. The thinning processing unit 2 performs thinning processing on a region having a pixel value of 1 of the binary image B (t) input from the background removing unit 12, and calculates pixel values in the linear region after the thinning processing. 1 and 0 for other pixel values
The thin line image S (t) is created (for example, the thin line images S (t-1) and S (t) shown in FIG. 3B). The thinning process is a process in which the region contour having a pixel value of 1 in the binary image is cut off from the surroundings to extract the center line having a width of 1 in the shape. Fine line image S after processing
(t) is sequentially sent to the feature point extraction unit 3.

The feature point extraction unit 3 determines the connection state of the thin lines in the thin line image S (t) by the number of intersections of the points, and extracts the feature points in the shape of the closed region (for example, in FIG. 3 (c)). Characteristic point a
(t-1), b (t-1), a (t), b (t)). The number of intersections of points is the number of transitions from the background (pixel value = 0) to the inside of the area (pixel value = 1) when one round is made around eight points. Fine line image S
On (t), the number of intersections is 1 (end point), 3 points (branch point),
The point 4 (intersection) is determined as a feature point of the shape of the closed region, and the label pt (pt = 1, ..., Pt) is added. The two-dimensional coordinates (Xt [pt], Yt [pt]) of the feature point are extracted and sent to the feature point correspondence unit 4.

The feature point correspondence unit 4 is composed of coordinate storage units 411 and 412 and a correspondence processing unit 42 as shown in FIG.
When t = 1, the feature point coordinate values sent from the feature point extraction unit 3 are stored in the coordinate storage unit 411. At this time, the handling processing unit 42 is not activated. When t ≠ 1, the feature point extraction unit 3
When the coordinate value of the characteristic point at time t is sent from the coordinate storage unit 4,
The coordinate value at time (t-1) stored in 11 is moved to the coordinate storage unit 412, the coordinate value at time t is stored in the coordinate storage unit 411, and the correspondence processing unit 42 is activated.

The correspondence processing unit 42 compares the feature point coordinate values at time (t-1) and time t, and associates the ones having the shortest Euclidean distance between the feature points with each other in a one-to-one correspondence (for example, FIG. 3). Feature points a (t-1) and a (t), b (t-1) and b shown in (d)
(t)). The result of this correspondence is the corresponding ordered pair of p (t-1) and pt.
It is stored in the movement amount calculation unit 5 as a set Γt of (p (t−1), pt).

Γt = {(p (t-1), pt) | p (t-1) is a feature point at time (t-1) corresponding to the feature point pt at time t on a one-to-one basis} (1) The movement amount calculation unit 5, as shown in FIG.
X, 51Y, a feature point movement amount calculation unit 52, a fine line movement amount calculation unit 53, and a region movement amount calculation unit 54.

The movement amount storage unit 51X (Y) is a frame memory having the same size as the processed image, and the time ((X, Y)) of the pixel of the coordinate value (X, Y) in the area existing in the thin line image B (t) is stored. t-
It has the amount of movement (unit: pixel) in the X (Y) direction between 1) to t as the value of the address (X, Y). The feature point movement amount calculation unit 52 refers to the contents of the coordinate storage unit 412 and the coordinate storage unit 411, and based on the set Γt, the feature point pt and the feature point (p ′ (t-1) corresponding to pt ), The difference between the X and Y coordinate values is calculated, and this difference value is registered in the movement amount storage units 51X and 51Y as the movement amount of pt between time (t-1) and t.

M51X (Xt [pt], Yt [pt]) = Xt [pt] -X (t-1) [p '(t-1)] M51Y (Xt [pt], Yt [pt]) = Yt [pt] -Y (t-1) [p '(t-1)] ··· (2) where M51X (X, Y) and M51Y (X, Y) are movement amount storage units 51X and 51Y, respectively. Represents the value of the address (X, Y).

The thin line movement amount calculation unit 53 is started after the movement amounts of all pt are calculated, and the end points are determined by referring to the thin line images S (t-1), S (t) and the set Γt. The corresponding thin line section is regarded as corresponding between two frames, and the coordinate sequence (Xs) at the time (t-1) of the corresponding thin line section between the two frames is considered.
₁ [l ₁ ], Ys ₁ [l ₁ ]) (l ₁ = 1,2, ..., L ₁ ) and the coordinate sequence of time t (Xs ₂ [l
₂ ], Ys ₂ [l ₂ ]) (l ₂ = 1,2, ..., L ₂ ) is extracted. Then the time
The points of l ₁ = 1, l ₁ = L ₁ in (t-1) are feature points, and
It corresponds to the feature point where l ₂ = 1, l ₂ = L ₂ at t. Any l ₂
L ₁ corresponding to is estimated so that l ₂ : L ₂ −l ₂ = l ₁ : L ₁ −l ₁ ··· (3) holds, and the point of l _{2 in} the thin line section is estimated based on the estimation result. The amount of movement from time (t-1) to t is calculated by the following formula and registered (for example, c (t-1), c (t) shown in FIG. 3 (e)). M51X (Xs ₂ [l ₂ ], Ys ₂ [l ₂ ]) = Xs ₂ [l ₂ ] -Xs ₁ [int (l ₂ × L ₁ / L ₂ )] M51Y (Xs ₂ [l ₂ ], Ys ₂ [l ₂ ]) = Ys ₂ [l ₂ ] -Ys ₁ [int (l ₂ × L ₁ / L ₂ )] ・ ・ (4) However, int () is a function that rounds the decimal places to an integer. is there.

The movement amount is calculated for all points on the thin line.
After the registration, the area movement amount calculation unit 54 is activated.
The area movement amount calculation unit 54 refers to the contents of the binary image B (t) and the movement amount storage units 51X and 51Y, and extracts the coordinates (Xm, Ym) of a point existing inside the area and having no movement amount registered. . The amount of movement of this point is equal to the amount of movement of a point (coordinate value (Xsm, Ysm)) on the thin line that can be reached by the shortest path along the point of pixel value 1 in B (t) (Fig. 3). (See (f)).

M51X (Xm, Ym) = M51X (Xsm, Ysm) M51Y (Xm, Ym) = M51Y (Xsm, Ysm) (5) At the point where the pixel value is 1 on the binary image B (t) All the corresponding movement amounts are calculated and registered in the movement amount storage units 51X and 51Y.

As described above, for a closed region shape that is continuously formed and changed in a moving image, instead of extracting the deformation by the contour line, the center line has a shape that is hardly affected by the minute deformation of the contour portion. Since the amount of movement inside the closed region is calculated by extracting the deformation using thin lines, the object deformation in the moving image, especially the shape, can be configured without being affected by minute changes in the contour line due to noise etc. at the time of input to the moving image. You can roughly capture the movement of the part you do.

Further, the movement amount calculation unit 5 outputs the movement amount of the points in the movement amount storage units 51X and 51Y to the outside of the apparatus as the deformation amount of the closed area between time (t-1) and t. At the same time, the image display unit 6 displays the deformation amount as an image.

The image display unit 6 includes movement amount storage units 51X and 5X.
It is a device that visualizes and displays the deformation amount of the closed region between time (t-1) and t stored in 1Y as an image, and includes an imaging unit 61 and a display 62.

The imaging unit 61 reads the deformation amount of the point (coordinates (Xc, Yc)) having the pixel value 1 on the binary image B (t) from the movement amount storage units 51X and 51Y, and based on the value. An image having the determined RGB value as the RGB value of the coordinates (Xc, Yc) is created, and this image is transmitted to the display 62 and displayed.
(Here, the background pixel value is 0.) RGB from the movement amount
Although there are various functions for determining the value, the function shown in FIG. 8 is shown as an example. R (Xc, Yc), G (Xc, Yc), and B (Xc, Yc) are RGB values of coordinates (Xc, Yc), and are values of 0 to 1.

[0033]   R (Xc, Yc) is as shown in FIG.               {1−3θ / (2π)} r / D; 0 ≦ θ ≦ 2π / 3   R (Xc, Yc) = {3θ / (2π) −2} r / D; 4π / 3 ≦ θ ≦ 2π               0 ； Other than the above ・ ・ (6-1) Is.

[0034]   G (Xc, Yc) is as shown in FIG.             {2−3θ / (2π)} r / D; 2π / 3 ≦ θ ≦ 4π / 3 G (Xc, Yc) = {3θ / (2π)} r / D; 0 ≦ θ ≦ 2π / 3             0 ； Other than the above ・ ・ (6-2) Is.

[0035]   B (Xc, Yc) is as shown in FIG. 8 (c).             {3−3θ / (2π)} r / D; 4π / 3 ≦ θ ≦ 2π B (Xc, Yc) = {3θ / (2π) −1} r / D; 2π / 3 ≦ θ ≦ 4π / 3             0 ； Other than the above ・ ・ (6-3) Is.

The angle θ is θ = 2 tan ⁻¹ {−ΔX / (r + ΔY)} (unit: rad, 0 ≦ θ ≦ 2
π), and the magnitude r of the amount of deformation is r = sqrt (ΔX ² + ΔY ² ) (unit: pixel).

As shown in FIG. 8D, the X-axis direction deformation amount ΔX is ΔX = M51X (Xc, Yc) and the Y-axis direction deformation amount ΔY ΔY = M51Y (Xc, Yc).

Here, D is a constant for accommodating the RGB value in a value of 0 to 1, and is a value set larger than the maximum value of the size r of the deformation amount expected in the moving image. If you use this function to create an image of the deformation of a region, an image is created that becomes brighter as the region moves greatly with respect to the origin of the image, and is colored in a different color depending on the moving direction. At 62, the deformation of the area can be easily confirmed.

[0039]

As described above, according to the present invention,
For closed region shapes that continuously change in moving images,
Instead of extracting the deformation by the contour line, the deformation is extracted by the thin line which is the center line of the shape that is hard to be affected by the minute deformation of the contour part, so it is not affected by the minute change of the contour line due to noise etc. at the time of moving image input. It is possible to extract a rough deformation of the shape.

[Brief description of drawings]

FIG. 1 is a flowchart showing the principle of the present invention.

FIG. 2 is a device configuration diagram for realizing the present invention.

FIG. 3 is a diagram for explaining the operation of the present invention.

FIG. 4 is a flowchart in an embodiment of the present invention.

FIG. 5 is a device configuration diagram in an embodiment of the present invention.

FIG. 6 is a detailed configuration diagram of a feature point corresponding unit.

FIG. 7 is a detailed configuration diagram of a movement amount calculation unit.

FIG. 8 is a diagram illustrating R for converting a moving amount inside a region into a color image.
It is a GB value explanatory drawing.

[Explanation of symbols]

1 ... Image input section 2 ... Thinning processing unit 3 ... Feature point extraction unit 4 ... Characteristic point corresponding part 5 ... Movement amount calculation unit 6 ... Image display 7 ... Region shape time change extraction unit 11 ... Moving image input device 12 ... Background removal section 42 ... Corresponding processing unit 51X, 51Y ... Moving amount storage 52 .. Feature point movement amount calculation unit 53 .. Thin line movement amount calculation unit 54 ... Area movement amount calculation unit 61 .. Imaging unit 62..Display 411, 412 ... Coordinate storage unit

Claims (3)

[Claims] 1. A region shape temporal change extraction device for extracting a temporal change amount of a shape contour of a closed region between two adjacent frames using an image in a moving image frame output from an image input device (1) ( In 7), a thinning processing unit (2) for creating a thin line image by shaving the shape contour of the closed region in the moving image frame from the periphery, and a feature on the thin line thinned by the thinning processing unit (2) Feature point extraction means (3) for extracting points, feature point correspondence means (4) for associating feature points between the two frames extracted by the feature point extraction means (3), and the feature point correspondence A calculation means (5) for calculating the movement amount of the feature points and the movement amount of the points on the thin line based on the feature points associated by the means (4), wherein the calculation means (5) is closed. If there is a point with a known amount of movement in the area, move another point The time change detection process of a region shape, and calculates estimated as equivalent to the amount of movement of the point on the thin line that can be reached at the shortest distance along the closed area. 2. The calculation means (5) calculates the amount of movement of a point inside the closed region, and outputs the amount of movement to the image output device (6) as the amount of deformation of the region part. The method for extracting a temporal change of a region shape according to claim 1. 3. The calculation means (5) estimates, when the correspondence of feature points between two frames is known, the correspondence between frames of points on a thin line section sandwiched by the feature points, The method for extracting a temporal change of a region shape according to claim 2, wherein the amount of movement of the point on the thin line section is calculated based on the calculated amount.