CN100446544C - A Method for Extracting Outer Boundary of Video Object - Google Patents

Abstract

The invention provides a method for extracting the outer boundary of a video object, which generates a frame difference image by performing a Gaussian noise model seed region growth method on two consecutive gray-scale images after global motion compensation, and calculates the motion after extracting the time-domain change area Vector, to distinguish the moving model area and the failure area, and then extract the moving object and repair it, detect the maximum point of the outer boundary of the airspace, and connect the canny boundary where the maximum point of the airspace is located to obtain the outer boundary of the moving object. By adopting the method of the invention, not only accurate outer boundary positioning information can be obtained, but also the whole method has high robustness, and at the same time, its calculation speed can be applied to real-time systems, and has strong practical application prospects.

Description

A Method for Extracting Outer Boundary of Video Object

technical field

The invention belongs to the technical field of image processing, in particular to the image processing technology of video object segmentation.

Background technique

In order to support content-based interactivity, that is, to support independent encoding and decoding of content, the MPEG-4 video inspection model introduces the concept of Video Object Plane (VOP: Video Object Plane). Video object segmentation refers to dividing an image sequence or video into regions according to a certain standard, with the purpose of separating entities with certain semantics from the video. Such semantic entities are called video objects in digital video. The outer boundary of the video object is the outermost outline of the video object. Therefore, by extracting the information of the outer boundary of the video object, the regional characteristics of the entire video object in the image can be obtained, thereby realizing the segmentation of the video object. Compared with a single picture that only has image information based on the space on its coordinates (that is, image spatial domain information), video images also have time domain information of each frame of image relative to the time correlation between previous and subsequent frames. Therefore, a video image has information in both the spatial domain and the temporal domain.

Video object segmentation is based on the premise of MEPG4 video processing and calculation. It has a wide range of applications in many civilian fields such as computer vision, traffic monitoring, visual early warning, and machine navigation. It also plays an important role in military fields such as shooting range TV measurement and aircraft TV guidance. effect. Video segmentation is a core operation in object-oriented video coding, multimedia description, and intelligent signal processing. But effective segmentation is a very difficult task and challenge in image analysis. (See literature Haritaoglu, I, Harwood, D., Davis, L.S. "W4: real-time surveillance of people and their activities", Pattern Analysis and Machine Intelligence, IEEE Transactions on, Volume: 22, Issue: 8, Aug. 2000 Pages: 809-830.)

From the current information and methods used by researchers, video object segmentation can be divided into three categories: (1) temporal segmentation; (2) spatial segmentation and temporal tracking; (3) combined temporal and spatial segmentation. (See literature Fuhui Long, Dagan Feng, Hanchuan Peng, Wan-Chi Siu, "Extracting semantic video objects", Computer Graphics and Applications, IEEE Vol.21; Jan 2001; pages 48-55)

The first method just uses time-domain information, but because all such methods cannot solve the problem of large calculation and inaccurate positioning, such methods do not get good results. The second method is to perform image segmentation in the spatial domain first, and then track the segmented image regions in the temporal domain. But when faced with most of the current image sequences with complex backgrounds, such methods are usually time-consuming and fail to obtain expected results. The third method has been greatly promoted because it makes full use of both sides of the information, and the method proposed by the present invention belongs to this scheme system. However, because the amount of information in this method is huge, and the time domain and space domain information are generally inconsistent at the same time, that is to say, the obtained time domain result does not match the space domain information in this frame image. Therefore, the methods currently used by most researchers cannot fuse the spatial and temporal information well, which results in too much calculation and cannot obtain the complete information of the video object.

Contents of the invention

The purpose of the present invention is to provide a method for extracting the outer boundary of a video object using time domain and space domain information, which has the characteristics of strong anti-noise, reasonable and fast fusion of space and time domain information, fast calculation speed, and strong robustness.

In order to facilitate the description of the content of the present invention, a definition of terms is first made:

1. Global motion compensation: Global motion is the total motion of the entire image caused by the unavoidable scaling, horizontal motion, vertical motion, and rotational motion of the camera during the recording process of the video stream. Global motion compensation is the compensation of one frame of image relative to the previous frame by the amount of motion calculated by the global motion method, so that the two frames of images can remove the influence of camera motion. (For specific methods, please refer to "Digital Video Processing" Cui Zhiku translation)

2. Seed region growth method: Seed region growth is to gather pixels with similar properties to form a region. Specifically, first find a pixel for each region that needs to be segmented as a seed point for growth, and then merge pixels that have the same or similar properties as the seed pixel in the neighborhood around the seed pixel (judged according to the pre-determined growth or similarity criteria) to the region where the pixel is located. Use these new pixels as new seed pixels to continue the above process until no pixels satisfying the condition are included, so that a region grows.

The regional growth method needs to solve three problems:

A. Select or determine a group of seed pixels that can correctly represent the desired area;

B. Criteria for determining growth;

C. Establish the conditions under which the growth process stops. (For specific methods, please refer to "Machine Vision" edited by Jia Yunde)

3. Background occlusion area: the background area not covered by the moving object in the previous frame, but the background area covered by the moving object in the next frame as the moving object moves. (See "Digital Video Processing" Cui Zhiku translation)

4. Background exposed area: the background area that was covered by the moving object in the previous frame and appears in the next frame with the movement of the moving object. (See "Digital Video Processing" Cui Zhiku translation)

5. Gaussian noise: noise whose statistical characteristics are Gaussian distribution characteristics. In general signal processing, the interfering white noise is regarded as Gaussian noise, and its mean value and method are calculated in the analysis to facilitate processing. (See "Digital Image Processing" Gonzalez)

6. Binarization: use 0, 1 to represent the binary characteristics of the entire area. (See "Digital Image Processing" Gonzalez)

7. Morphological filtering: use the method of mathematical morphology to filter. Usually, based on the morphological dilation operation and erosion operation, combined into an opening operation and a closing operation to remove the noise of different binarized images respectively. (For specific methods, please refer to "Image Processing and Analysis: Mathematical Morphological Methods and Applications" edited by Cui Yi)

8. Structural element: It is a mathematical morphology processing image collection of other images. (For specific methods, please refer to "Image Processing and Analysis: Mathematical Morphological Methods and Applications" edited by Cui Yi)

9. Morphological expansion operation: it is the most basic operator of morphology, and its meaning is: $A\⊕B=\{c\∈{\mathrm{E.}}^{\mathrm{no}}|c=a+b,a\∈A,b\∈B\},$ Among them, A and B are two subsets of n-dimensional Euclidean space E ⁿ . Among them, B is generally a structural element of morphology. (For specific methods, please refer to "Image Processing and Analysis: Mathematical Morphological Methods and Applications" edited by Cui Yi)

10. Morphological corrosion operation: it is the most basic operator of morphology, and its meaning is: $\mathrm{A\ΘB}=\{c\∈{\mathrm{E.}}^{\mathrm{no}}|c+b\∈A,\∀b\∈B\},$ Among them, A and B are two subsets of n-dimensional Euclidean space E ⁿ . Among them, B is generally a structural element of morphology. (For specific methods, please refer to "Image Processing and Analysis: Mathematical Morphological Methods and Applications" edited by Cui Yi)

11. Connectivity: Known pixel p, q∈S, if there is a path from p to q, all the pixels on the path are included in S, then p and q are said to be connected. (See "Digital Image Processing" Gonzalez)

12. Canny boundary: use the canny principle to obtain the binarized image of the image boundary. (See "Digital Image Processing" Gonzalez)

13. Mask: a binarized area, 1 indicates the target, and 0 is an irrelevant area. Using a mask on an image is to retain the value of the image corresponding to the mask image as 1, and change the value of the mask to 0. (See "Digital Image Processing" Gonzalez)

14. Regional connectivity calibration method: It is to perform connectivity calibration on the entire binarized image, and set a flag for each connected area. (For specific methods, please refer to "Machine Vision" edited by Jia Yunde)

15. Phase correlation method: the method of obtaining the optical flow motion vector by using the phase relationship obtained by the Fourier transformation of the corresponding block. (For specific methods, please refer to "Digital Video Processing" Cui Zhiku translation)

16 Outer contour boundary extraction method: extract the outer boundary points of the outermost contour point by point in a clockwise direction. (For specific methods, please refer to "Machine Vision" edited by Jia Yunde)

17. Compensation: The displacement of a pixel according to the compensation definition, and the gray value of the pixel moving from the current frame image to the next frame image remain unchanged, then this displacement is called motion compensation. The movement of pixels according to this displacement is called compensation. (For specific methods, please refer to "Digital Video Processing" Cui Zhiku translation)

18. Matching: The similarity between two pixels is represented by the calculated absolute difference of gray values of two pixels.

19. Spatial gradient: the square difference of the gray value at the top, bottom, left, and right positions of the image pixel.

A kind of video object outer boundary extraction method provided by the present invention, it comprises the following steps (whole process is referring to shown in accompanying drawing 1):

Step 1. For two consecutive grayscale images in the video stream that have undergone global motion compensation, use the seed region growth method conditioned on the Gaussian noise model to perform frame difference processing to obtain a noise-resistant binarized time-domain frame difference Image, where the area with a value of 1 is a time-domain change area;

The specific method is as follows: first calculate the absolute difference on two successive frames of grayscale images that have undergone global motion compensation, then set the absolute difference greater than 40 to 0, and count the mean (expressed by M) and variance (expressed by M) of the remaining absolute differences ( A indicates), and then set a threshold of M+4*A as the seed condition according to the mean M and variance A of the absolute difference image obtained above, and set the interval range of [M+A, M+4*A] as the seed growth condition. Search point by point on the absolute difference images of two consecutive grayscale images that have undergone global motion compensation, and set all the pixels whose absolute difference values are greater than the seed threshold as seed pixels. Then search in the neighborhood of the seed pixel point, and set the pixel point satisfying the seed growth condition as the seed pixel point. After all the absolute difference images are searched, the area where the seed grows is obtained, and all pixels in this area are set to 1, and all pixels outside this area are set to 0, then a binarized time-domain frame difference image is obtained.

Step 2. Perform morphological filtering on the binarized time-domain frame difference image generated by the seed region growth method in step 1 to obtain a time-domain change region image after denoising;

Step 3. For the denoised time-domain change region image obtained in step 2, use the region connectivity calibration method to obtain a time-domain change region image with a sign;

Step 4, take out each sign on the change area image with sign obtained in step 3 one by one, then scan the time domain change area image with sign pixel by pixel, if the sign of this pixel point is the same as the sign taken out, Then the pixel is set to 1, if not the same as 0, the pixel without a mark is set to 0. After scanning, the binarized image of the time-domain single-change area of the mark is formed, and each mark must generate the binarized image of the time-domain single-change area of the mark one by one;

Step 5, take the binarized image of the time-domain single change area of each mark obtained in step 4 as a mask image, scan the two consecutive grayscale images in the video stream in step 1 that have undergone global motion compensation, and keep the mask The gray value of the corresponding position of the pixel point of 1 on the mask image on the corresponding position of two consecutive gray scale images, and the gray value of the corresponding position of the pixel point of 0 on the mask image on the corresponding position of two consecutive gray scale images 0, after the scanning is completed, two time-domain single-change area gray-scale images corresponding to each mark are obtained, which only retain the corresponding time-domain change area gray value;

Step 6, calculation step 4 obtain the coordinate values of the four vertices of the largest circumscribed rectangle of the region whose value is 1 in the binarized image of the single change region in the time domain;

Step 7. According to the coordinate values of the four vertices of the largest circumscribed rectangle of the binarized image of the time-domain single-change area of each sign obtained in step 6, the two time-domain single-change areas of each sign obtained in step 5 are gray In the degree image, the internal image of the largest circumscribed rectangle is extracted to form two local time-domain grayscale images corresponding to each sign;

Step 8. Obtain two local time-domain change area grayscale images corresponding to each sign in step 7, and use the phase correlation method to calculate the actual moving object that produces the local time-domain change area in step 7. The relative motion displacement on the grayscale image of the local time domain change area of the mark;

Step 9, for the two local time-domain change area grayscale images of each sign obtained in step 7, the moving object that produces the change area obtained in step 8 on the two local time-domain change area gray-scale images The relative motion displacement is used as the pixel matching condition, and the seed area growth method is used to remove the background occlusion area and the background exposure area, and generate a single moving object area in time domain.

Specific method: According to the relative motion displacement of the moving object in the time-domain change area corresponding to each sign obtained in step 8 on the grayscale images of two local time-domain change areas, the seed condition is set as follows: if the two parts of the sign The absolute value of the gray-scale amplitude difference of the corresponding pixel point on the gray-scale image of the time-domain change area after compensation according to the relative motion displacement is less than 2, and is smaller than the absolute value of the gray-scale amplitude difference without motion compensation, then the pixel point is The seed pixel. Set the growth condition of the seed as follows: if the absolute value of the gray scale amplitude difference between the corresponding pixels on the gray scale image of the two local time domain change areas of the sign is compensated according to the relative motion displacement, it is greater than 2 and less than 5, and less than that without motion compensation The absolute value of the gray scale difference, then the pixel is the seed growth pixel. Scan step 7 one by one to obtain the pixels on the grayscale image of the two local time-domain change regions corresponding to the sign, and set the pixels satisfying the seed condition as the seed pixels. Then search in the neighborhood of the seed pixel point, and set the point satisfying the seed growth condition as the seed pixel point. After searching all the pixels of the grayscale image in the local time-domain change area, the time-domain single moving object area where the seed grows is obtained, and all pixels in this area are set to 1, and all pixels outside this area are set to 0 , then a binarized time-domain single moving object area is obtained.

Step 10, obtain binarization time domain single moving object area in step 9, use patching method, obtain complete time domain single moving object area; (the above is the processing flow of time domain boundary extraction, referring to shown in Figure 2)

Specific repair method: perform row and column scanning on the single moving object area in the binarization time domain, and remove the pixel points of the object that are not pixels of the moving object and are located between two moving object pixels in the same line (or in the same column) Set it as the pixel of the moving object, and keep the other pixels unchanged to obtain a complete single moving object area in the time domain

Step 11. For the complete time-domain single moving object area generated in step 10, use the outer contour boundary extraction method to obtain the outer boundary contour points of the moving object;

Step 12, for the outer boundary contour points of the moving object obtained in step 11, replace point by point with the maximum value point of the space boundary, and obtain the image of the maximum point of the space boundary of the time domain object;

Specific method: calculate the 8 neighbors of the time-domain boundary contour points for spatial gradient, and then carry out the spatial gradient with the current time-domain boundary point containing the weight W (the weight W is selected as a number greater than 1 to strengthen the time-domain results). In contrast, the point with the largest spatial gradient is selected as the maximum value point of the spatial boundary of the time domain object, and the image of the maximum value point of the spatial boundary of the time domain object is obtained;

Step 13, using the canny boundary extraction method to extract the boundary of the two local time-domain change region grayscale images corresponding to the signs obtained in step 7, to obtain the spatial domain canny boundary image of the local time-domain change region;

Step 14. On the spatial canny boundary image of the local time domain change area in step 13, perform a connection operation according to the maximum value point image of the spatial boundary of the time domain object obtained in step 12, to obtain the outer boundary of the spatial and temporal fusion moving object. (The above is the refinement and connection of the airspace boundary, as shown in Figure 2)

The specific connection operation is: retain the airspace canny boundary of the local time domain change area containing the airspace boundary maximum point, remove the airspace canny boundary of the local time domain change area that does not contain the airspace boundary maximum point, and form the airspace time domain fusion motion The outer boundary of the object, that is, the outer boundary of the video object.

Through the above steps, we can get the outer boundary of the video object.

It should be noted:

(1) What the present invention utilizes is the situation that the moving object in two consecutive video images overlaps to a certain extent in the video stream, and also uses the phase correlation method to calculate the optical flow vector of the moving object simultaneously, so this invention is applicable to moving object speed normal and The case where the moving object is a non-deformable object.

(2) The Gaussian noise model used in step 1 of the present invention is formed based on the theoretical basis of Gaussian noise for frame difference images, which is already a common noise standard in digital images. Therefore, the seed region growth method using this method can remove noise very well and obtain an effective motion change region.

为f₁(x，y)，f₂(x+d₁，y+d₂)的傅立叶变化，为频域因子。这样

的傅立叶变化就存在公式2的关系，它们相位之间存在公式3的关系，因此可以得到一个关于(d₁，d₂)的脉冲，从而计算出(d₁，d₂)，其中

和

分别为

的相位，j为复数单位。(3) There are three kinds of regions in the changed region after the logo is obtained in step 4: moving object, covered background, and exposed background. For these three kinds of regions, in step five, the corresponding change regions in two consecutive frames are used to calculate the corresponding displacements, and then the three kinds of regions can be distinguished. The principle is: because most of the changing areas are moving objects, which have the same motion vector, there is a relationship of formula 1, where (d ₁ , d ₂ ) is the displacement vector of the moving object, f ₁ (x, y), f ₂ (x+d ₁ , y+d ₂ ) represents the corresponding image blocks in the two frames of images,

is the Fourier transform of f ₁ (x, y), f ₂ (x+d ₁ , y+d ₂ ), is the frequency domain factor. so

The Fourier change of , there is a relationship of formula 2, and there is a relationship of formula 3 between their phases, so a pulse about (d ₁ , d ₂ ) can be obtained, and (d ₁ , d ₂ ) can be calculated, where

and

respectively

The phase of j is a complex unit.

f ₁ (x, y) = f ₂ (x+d ₁ , y+d ₂ ) (Formula 1)

(公式2)

(Formula 2)

(Formula 3)

(4) Due to the uncertainty introduced by the displacement calculation and frame difference processing of the time domain object outer boundary extracted in step 11, there will be a result that the precise positioning of the object boundary cannot be obtained, so the present invention obtains accurate positioning by spatial domain processing. The outer boundary of the object.

The essence of the present invention: it generates the frame difference image by performing the seed area growth method of the Gaussian noise model on two successive frames of images after global motion compensation. Then use the phase correlation method to operate on the actual two frames of images in the corresponding change area to obtain the motion vector. The motion vector is used to distinguish the motion model area from the failure area, and then the motion object is extracted by using the seed area growing method in the motion area according to the motion vector. After patching the moving object area, the outer boundary of the airspace is detected. At this point the time domain analysis is complete and then refined using the air domain information. First, replace the outer boundary points of the time domain object with the maximum value points of the spatial gradient, and finally connect the canny boundary where the maximum value points of the space domain are located. Thus the outer boundary of the moving object is obtained.

The method of the present invention has the following three characteristics: one is based on the Gaussian noise model to generate a frame difference image with strong anti-noise, so it is suitable for various complex background situations; two uses the fast phase correlation method to obtain Temporal moving objects can form target object regions with single semantics; three only use the spatial boundary information of temporal object edges to correct moving objects. Therefore, the overall method has an excellent anti-noise method, and the fusion of space and time domain information is reasonably simplified, so that the invention has the characteristics of fast computing speed and strong robustness.

The innovation of the present invention is:

1. The seed region growth method based on the Gaussian noise model to generate a frame difference image with strong anti-noise. Because its denoising theory is reasonable, it is suitable for various complex background situations, thereby improving the robustness of the method and greatly reducing the amount of calculation of the method.

2. Using the fast phase correlation method to obtain the time-domain moving object can form a target object area with a single semantic, so it can achieve the purpose of accuracy while implementing it quickly.

3. Replace the maximum value of the air domain boundary with the obtained time domain object boundary, and finally connect the canny boundary, so as to accurately locate the boundary information on the basis of making full use of the time domain information.

The method for extracting the outer boundary of the video object of the present invention fully utilizes the space and time information of the image sequence. Not only can the accurate positioning information of the outer boundary be obtained, but also the overall method has high robustness, and its calculation speed can be applied to real-time systems, so it has a strong practical application prospect.

Description of drawings

Fig. 1 schematic flow chart of the present invention

Fig. 2 Schematic diagram of time domain boundary extraction processing flow in the present invention

Fig. 3 schematic diagram of airspace boundary refinement and connection processing flow in the present invention

Detailed ways:

A specific implementation example of the present invention is given below. What this implementation example uses is the extraction of vehicles under complex backgrounds.

Step 1 is for two consecutive frames of images in the video stream after global motion compensation, where the current frame is f(x, y, k) and the previous frame is f(x, y, k-1) (in the formula x represents the row coordinates in the image matrix, y represents the column coordinates in the image matrix, k represents the time relative position of this frame in the entire video stream, k-1 represents the time relative position of the previous frame in the entire video stream), Remove the value greater than the threshold 40 in the absolute difference of direct subtraction, and then perform statistics on the remaining absolute value, and calculate the mean M and variance A of the direct difference;

Step 2 takes the pixel greater than M+4×A as the seed point, takes [M+A, M+4×A] as the growth condition, and uses the seed region growth method to get |f(x, y, k)-f (x, y, k-1)|The result is processed to obtain the binarized image of the anti-noise time-domain change area, and the image is represented by h(x, y, k);

Step 3: Filter h(x, y, k) using morphological opening and closing operations to obtain the image of the time-domain change area after denoising, which is represented by o(x, y, k);

Step 4 Carry out connected calibration on o(x, y, k) to obtain a time-domain change area image with a mark, which is represented by l(x, y, k). The specific method is: scan the image, find a point without a mark, Assign it a new label L; recursively assign the label L to the adjacent points of 1 point; stop when there are no unmarked points; return to the beginning, and scan the image again until all 1 points have labels. In this way, images of temporally changing regions with signs can be obtained;

Step 5: Extract the corresponding time-domain images from f(x, y, k) and f(x, y, k-1) for the time-domain change region image of each marker in l(x, y, k) The binarized image of a single change area is represented by ch(x, y, k) and ch(x, y, k-1). At this time, each extracted binarized image of a single change area in the time domain is corresponding to The change caused by the displacement of a single moving object on two consecutive frames of images.

Step 6 uses ch(x, y, k) and ch(x, y, k-1) as mask images, and retaining the mask image as 1 corresponds to f(x, y, k) and f(x, y, The gray value at the position of k-1), the corresponding position of 0 is also set to 0 at f(x, y, k) and f(x, y, k-1), and two corresponding to each sign are obtained A time-domain single-change area gray-scale image that only retains the gray-scale value of the corresponding time-domain change area is represented by gch(x, y, k) and gch(x, y, k-1);

Step 7 Calculate the coordinate values of the four vertices of the largest circumscribed rectangle for the binarized image of the time-domain single change region of each mark in l(x, y, k) corresponding to ch(x, y, k). It is represented by (x ₀ , y ₀ )(x ₁ , y ₁ )(x ₂ , y ₂ )(x ₃ , y ₃ ).

_Step 8 From _ch (x _{, y} , _k ) and _ch (x,y _{, k-} In 1), two local time-domain grayscale images corresponding to each mark are extracted; represented by och(x, y, k) and och(x, y, k-1).

Step 9 for och(x, y, k) and och(x, y, k-1) as f ₁ (x, y), f ₂ (x+d ₁ , y+d ₂ ) in (Formula 1) , get the motion vector (d ₁ , d ₂ ) according to (Formula 2) and (Formula 3);

Step 10 for och(x, y, k) and och(x, y, k-1), with

och(x, y, k)-och(x+d ₁ , y+d ₂ , k-1)<och(x, y, k)-och(x, y, k-1) and

och(x, y, k)-och(x+d ₁ , y+d ₂ , k-1)<Th1 (Th1 is set to 2) is the seed condition, with

och(x, y, k)-och(x+d ₁ , y+d ₂ , k-1)<och(x, y, k)-och(x, y, k-1) and

Th1<och(x, y, k)-och(x+d ₁ , y+d ₂ , k-1)<Th2 (Th2 is set to 5) is the growth condition for seed region growth to obtain the moving object region in the time domain . The specific growth method is: scan the pixel points on och(x, y, k) and och(x, y, k-1) one by one, set the pixel point satisfying the seed condition as the seed pixel point; The neighborhood of the pixel point is searched, and the point that satisfies the seed growth condition is set as the seed pixel point; after searching all the pixels of the gray image in the local time domain change area, the time domain single moving object area of the seed growth is obtained, set All pixels in this area are set to 1, and all pixels outside this area are set to 0, then a binarized time-domain single moving object area is obtained, using to(x, y, k) and to(x, y, k-1) to represent;

Step 11 performs row and column patching on to(x, y, k) and to(x, y, k-1) to obtain a complete time-domain single moving object area, using fto(x, y, k) and fto( x, y, k-1) to represent;

Step 12 extracts the outer boundary contour points of the moving object on fto (x, y, k) and fto (x, y, k-1); the specific method is: (a) scan fto from left to right, from top to bottom ( x, y, k) and fto(x, y, k-1), find the starting point s(k)=(x(k), y(k)) of the region, k=0; (where k is the obtained The sequence value of contour point, x (k), y (k) is the coordinate value of k point, s (k) represents contour k point) (2) represent the pixel point tracked on the current boundary with c, put c= s(k), record the left 4 neighbors of c as b (b is in the connected area); (3) record the 8 neighbors of c starting from b in the counterclockwise direction as n ₁ , n ₂ ..., n ₈ ; (4) find the point that the first n _i belongs to the connected region counterclockwise from b; (5) set c=s(k)=n _i , b=n _i-1 ; (6) repeat steps ( 3)(4)(5), until s(k)=s(0);

Step 13 For the fto(x, y, k) and fto(x, y, k-1) obtained above, the outer boundary contour points of the moving object are replaced point by point with the maximum value points of the space boundary to obtain the space space of the time domain object Boundary maximum point image, represented by mp(x, y, k) and mp(x, y, k-1);

Step 14 For och(x, y, k) and och(x, y, k-1), extract the boundary with the canny boundary extraction method to obtain the spatial canny boundary image of the local time domain change area, use cy(x, y , k) and cy(x, y, k-1) to represent;

Step 15 On the graph of cy(x, y, k) and cy(x, y, k-1), connect cy(x , y, k) and the boundaries on cy(x, y, k-1) to get the final spatial-temporal fusion moving object outer boundary, that is, the video object outer boundary, using tsb(x, y, k) and tsb( x, y, k-1) to represent.

According to the above steps, using MATLAB language programming, the final result can be obtained through computer simulation. Compared with the existing three-frame difference method and other conventional video object extraction methods, it can be seen that the method of the present invention can completely extract the video object, and the calculation speed is fast, the robustness is strong, the processing information is less, and it has high efficiency, wide application range, Adaptable effect.

Claims (1)

1. A video object outer boundary extraction method is characterized in that it comprises the following steps: Step 1. For two consecutive grayscale images in the video stream that have undergone global motion compensation, use the seed region growth method conditioned on the Gaussian noise model to perform frame difference processing to obtain a noise-resistant binarized time-domain frame difference Image, where the area with a value of 1 is a time-domain change area; The specific method is: first calculate the absolute difference on two consecutive grayscale images that have undergone global motion compensation, then set the absolute difference greater than 40 to 0, and count the mean and variance of the remaining absolute difference, and use M to represent the remaining absolute difference The mean value, A represents the variance; then set a threshold of M+4*A as the seed condition according to the mean M and variance A of the absolute difference image obtained above, and set the interval range of [M+A, M+4*A] as the seed Growth condition; search point by point on the absolute difference images of two successive frames of grayscale images that have undergone global motion compensation, and set all pixels whose absolute difference is greater than the seed threshold as seed pixels; then in the seed pixels The neighborhood of the point is searched, and the pixel point that satisfies the seed growth condition is set as the seed pixel point; after searching all the absolute difference images, the area where the seed grows is obtained, and all the pixel points in the area are set to 1, and the area All pixels other than are set to 0, then a binarized time-domain frame difference image is obtained; Step 2. Perform morphological filtering on the binarized time-domain frame difference image generated by the seed region growth method in step 1 to obtain a time-domain change region image after denoising; Step 3. For the denoised time-domain change region image obtained in step 2, use the region connectivity calibration method to obtain a time-domain change region image with a sign; Step 4, take out each sign on the change area image with sign obtained in step 3 one by one, then scan the time domain change area image with sign pixel by pixel, if the sign of this pixel point is the same as the sign taken out, Then the pixel is set to 1, if it is not the same, it is 0, and the pixel without a mark is set to 0; after scanning, the binarized image of the time-domain single change area of the mark is formed, and each mark must be generated one by one. A binarized image of a time-domain single change area of the sign; Step 5, take the binarized image of the time-domain single change area of each mark obtained in step 4 as a mask image, scan the two consecutive grayscale images in the video stream in step 1 that have undergone global motion compensation, and keep the mask The gray value of the corresponding position of the pixel point of 1 on the mask image on the corresponding position of two consecutive gray scale images, and the gray value of the corresponding position of the pixel point of 0 on the mask image on the corresponding position of two consecutive gray scale images 0, after the scanning is completed, two time-domain single-change area gray-scale images corresponding to each mark are obtained, which only retain the corresponding time-domain change area gray value; Step 6, calculation step 4 obtain the coordinate values of the four vertices of the largest circumscribed rectangle of the region whose value is 1 in the binarized image of the single change region in the time domain; Step 7. According to the coordinate values of the four vertices of the largest circumscribed rectangle of the binarized image of the time-domain single-change area of each sign obtained in step 6, the two time-domain single-change areas of each sign obtained in step 5 are gray In the degree image, the internal image of the largest circumscribed rectangle is extracted to form two local time-domain grayscale images corresponding to each sign; Step 8. Obtain two local time-domain change area grayscale images corresponding to each sign in step 7, and use the phase correlation method to calculate the actual moving object that produces the local time-domain change area in step 7. The relative motion displacement on the grayscale image of the local time domain change area of the mark; Step 9, for the two local time-domain change area grayscale images of each sign obtained in step 7, the moving object that produces the change area obtained in step 8 on the two local time-domain change area gray-scale images The relative motion displacement is used as the pixel matching condition, and the seed area growth method is used to remove the background occlusion area and the background exposure area, and generate a single moving object area in the time domain; Specific method: According to the relative motion displacement of the moving object in the time-domain change area corresponding to each sign obtained in step 8 on the grayscale images of two local time-domain change areas, the seed condition is set as follows: if the two parts of the sign The absolute value of the gray-scale amplitude difference of the corresponding pixel point on the gray-scale image of the time-domain change area after compensation according to the relative motion displacement is less than 2, and is smaller than the absolute value of the gray-scale amplitude difference without motion compensation, then the pixel point is Seed pixel point; the growth condition of the seed is set as follows: if the absolute value difference of the gray scale amplitude difference between the corresponding pixel points on the gray scale image of the two local time domain change areas of the sign is greater than 2 and less than 5 after the relative motion displacement is compensated, And less than the absolute value of the gray scale amplitude difference without motion compensation, then this pixel point is a seed growth pixel point; Scan step 7 one by one to obtain the pixel point on the grayscale image of the two local time domain change regions of the corresponding sign , set the pixel point that satisfies the seed condition as the seed pixel point; then search in the neighborhood of the seed pixel point, and set the point that meets the seed growth condition as the seed pixel point; after searching all the local time-domain change regions gray After the pixels of the high-resolution image, the time-domain single-moving object area for seed growth is obtained, and all pixels in this area are set to 1, and all pixels outside this area are set to 0, then a binarized time-domain single motion is obtained object area; Step 10, for the binarized time-domain single moving object area obtained in step 9, use a patching method to obtain a complete time-domain single moving object area; Specific repair method: perform row and column scan on the binarized time-domain single moving object area, set the pixel point of the object that is not the pixel point of the moving object and is located between two moving object pixels in the same row or in the same column as The pixel of the moving object, other pixels remain unchanged, and a complete single moving object area in the time domain is obtained; Step 11. For the complete time-domain single moving object area generated in step 10, use the outer contour boundary extraction method to obtain the outer boundary contour points of the moving object; Step 12, for the outer boundary contour points of the moving object obtained in step 11, replace point by point with the maximum value point of the space boundary, and obtain the image of the maximum point of the space boundary of the time domain object; Specific method: Calculate the 8 neighbors of the time-domain boundary contour points to carry out the spatial gradient, and then compare it with the spatial gradient of the current time-domain boundary point containing the weight W, and the weight W is selected as a number greater than 1 to strengthen For time-domain results, select the point with the largest spatial gradient as the maximum point of the time-domain object’s airspace boundary, and obtain the image of the time-domain object’s airspace boundary maximum point; Step 13, using the canny boundary extraction method to extract the boundary of the two local time-domain change region grayscale images corresponding to the signs obtained in step 7, to obtain the spatial domain canny boundary image of the local time-domain change region; Step 14, on the space canny boundary image of the local time domain change area in step 13, according to the time domain object space boundary maximum value point image obtained in step 12, perform a connection operation to obtain the space and time domain fusion moving object outer boundary; The specific connection operation is: retain the airspace canny boundary of the local time domain change area containing the airspace boundary maximum point, remove the airspace canny boundary of the local time domain change area that does not contain the airspace boundary maximum point, and form the airspace time domain fusion motion Object outer bounds.

Images