CN108280841A - A kind of foreground extracting method based on neighborhood territory pixel intensity correction - Google Patents

Abstract

The present invention provides a kind of foreground extracting method based on neighborhood territory pixel intensity correction, including input video, further comprising the steps of：Initial background is obtained according to the first frame of the frame sequence of the input video；The difference of foreground and background is carried out to the initial background；It sets image difference threshold and carries out stable calculation；Using Otsu threshold come to determine optimal threshold, and target is extracted from background image.The present invention proposes a kind of foreground extracting method based on neighborhood territory pixel intensity correction, and using the foreground detection algorithm based on neighborhood territory pixel intensity correction under dynamic scene, robustness is stronger, is suitable for more complex scene change.

Description

A Foreground Extraction Method Based on Neighborhood Pixel Intensity Correction

technical field

The invention relates to the technical field of machine vision and image target detection, in particular to a foreground extraction method based on neighborhood pixel intensity correction.

Background technique

At present, the automatic processing and prediction of surveillance video information has received great attention in many fields such as information science, computer vision, machine learning, and pattern recognition. How to effectively and quickly extract the foreground target information in the surveillance video is a very important and basic problem. The extraction of the moving target in the video sequence refers to segmenting the moving area from the video sequence, predicting the physical characteristics of the target in the next frame of image by estimating the target’s motion behavior, and performing the target extraction in the image sequence according to these features. Correlation matching to get the trajectory of the moving target. It combines the knowledge of computer image processing, video image processing, pattern recognition, artificial intelligence and automatic control and many other related fields, and is the key technology of intelligent video surveillance.

In terms of moving target extraction, there are mainly background subtraction method, frame difference method and optical flow method. Background subtraction is the most commonly used method. It subtracts the current frame from the extracted background to directly obtain the foreground moving target, which is suitable for the situation where the background is known, and the calculation is simple. However, in practical applications, the background is often not fixed, and the background model needs to be updated dynamically.

In the article "Adaptive background mixture models for real-time tracking" by Stauffer C and Grimson W, a mixed Gaussian distribution is used to model each pixel, and the model is updated using online estimation. The document "Improved adaptive Gaussian mixture model for background subtraction" uses a recursive filter to filter and update the background image, which increases the stability of the background subtraction method. The document "Efficient archical method for background subtraction" combines pixel-based and block-based methods to establish an effective graded background image, and first determines the non-fixed background. The frame difference method is to directly make the difference between the current frame and the previous frame, and the absolute difference between the pixels of the two adjacent images in time can display the position and shape of the target in the image. This method has strong adaptability to the environment, but the extracted objects tend to form holes and blurred edges. The document "A New Method for Moving Target Detection Based on Inter-Frame Difference and Background Difference" combines the background subtraction method and the frame difference method. After detecting the background pixels in the frame based on the frame difference method, the background subtraction method is used to detect the moving target. , to overcome the false detection and void problems. The document "Moving Target Detection Technology in Video Sequence" uses the continuous frame difference method to extract the moving area, and eliminates the noise through mathematical morphological filtering, which improves the extraction effect of the moving area. The optical flow method is a repeated iteration of the difference method between frames, which is similar to the basic principle of the frame difference method, but its error shows a nonlinear increasing trend with the increase of the speed of the moving target. In addition, the calculation of optical flow method is time-consuming, which cannot meet the real-time requirements of many image sequence moving target detection and tracking systems. In response to this shortcoming, the document "Real-time Detection Method of Human Movement Based on Optical Flow" proposed a method based on the absolute value of difference image and SAD combined with optical flow method, which can effectively and accurately detect moving objects. The literature "Regional Optical Flow Analysis Based on Frame Difference and Its Application" analyzes and compares the advantages and disadvantages of frame difference method, background subtraction method and optical flow method, and proposes a regional optical flow analysis method based on joint frame difference to improve The processing speed is improved and the calculation cost of the optical flow method is reduced.

Contents of the invention

In order to solve the above technical problems, the present invention proposes a foreground extraction method based on neighborhood pixel intensity correction, which adopts a foreground detection algorithm based on neighborhood pixel intensity correction in dynamic scenes, which is more robust and suitable for more complex changes Scenes.

The present invention provides a foreground extraction method based on neighborhood pixel intensity correction, including an input video, and further includes the following steps:

Step 1: Acquire the initial background according to the first frame of the frame sequence of the input video;

Step 2: performing a difference between the foreground and the background on the initial background;

Step 3: Set the image difference threshold for stability calculation;

Step 4: Use the Otsu threshold to determine the optimal threshold and extract the target from the background image.

Preferably, B ₁ (x, y)=F ₁ (x, y) is calculated in said step 1, wherein B ₁ (x, y) and F ₁ (x, y) are respectively at the coordinate point (x , the background pixel value at y) and the pixel value of the current frame at this point, x≤P, y≤Q, P and Q represent the width and height of the initial background, respectively.

In any of the above schemes, preferably, for the i-th frame, the background image B _i used for the difference can be obtained from B _i-1 , where,

In any of the above schemes, it is preferred that the step 2 includes calculating the difference D _i between the current background and the current frame as D _i =|F _i (x, y)-B _i-1 (x, y) |, where

Preferably in any of the above schemes, the step 2 also includes separating the image difference D _i from the background image and the moving target, the formula is Among them, τ is the threshold value set by itself.

In any of the above schemes, preferably, the step 3 includes setting the initial value of S ₁ (x, y) to zero, that is, S ₁ (x, y)=0, and calculating the stability S of each pixel (x, y), the calculation formula is Wherein, S _i represents the stability matrix of frame i, whose initial value is zero, that is, S ₁ (x, y)=0.

In any of the solutions above, preferably, the step 2 includes judging whether to correct the pixel value.

In any of the above schemes, preferably, the judging method is that when the minimum value min _{(x, y)} (S _i (x, y)) of the stability is smaller than the threshold δ, then the current background pixel value is Correction; when the minimum value of the stability min _{(x, y)} (S _i (x, y)) is greater than or equal to the threshold δ, the current background pixel value is not corrected.

In any of the above schemes, it is preferable to set D _i (x, y) = 1 when it belongs to the current moving target and there is an unstable value such that S _i (x, y) < 0, re-construct to reduce the amount of calculation , the calculation formula is P _i ={(x, y)|[D _i (x, y)=1]∩[S _i (x, y)<0]}.

Preferably in any of the above schemes, the step 3 includes calculating the standard deviation between corresponding pixel points, and the calculation formula is Among them, n is the size of a square moving window, N= ⁿ² represents the number of pixels, and the mean value μ is calculated by the pixel value of image I, and the formula is

In any of the above schemes it is preferred that, for each pixel of _Pi , two standard deviations are calculated from the background image and the current frame and And calculate the background pixel value B _i (x, y), the formula is:

In any of the above schemes, preferably, the size of the square moving window can change during motion, including (n ₀ +n ₁ ) pixels, where n ₀ and n ₁ represent non-moving pixels and moving pixels respectively. The number of pixels, and the corresponding pixel intensities are g ₀ and g ₁ respectively.

In any of the above schemes, preferably, the calculation formula of the pixel mean value μ becomes

Preferably in any of the above schemes, the formula for calculating the standard deviation becomes:

Wherein, |g ₀ −g ₁ | and (n ₀ +n ₁ ) are constants.

In any of the solutions above, preferably, the step 3 further includes using the detected background for subsequent foreground extraction after pixel value intensity correction, and using the processing result as an input of the subsequent i+1th frame.

In any of the above schemes, preferably, the step 4 also includes obtaining an image-based difference from the current frame and the background image The calculation formula is

Preferably in any of the above schemes, the step 4 also includes finding a reasonable threshold so that the weighted sum of the variances of the two classes is minimized, the formula is where G ₀ is the background region, G ₁ is the foreground object region, ωG ₀ (g) and ωG ₁ (g) denote the class probability at intensity g, and is the class variance.

In any of the above schemes, it is preferred that the step 4 also includes image-based difference Separate the foreground target, use the Otsu threshold method to determine the best threshold, the formula is

In any of the above schemes, preferably, the calculation formula of the image difference D _i becomes:

in, is a grayscale image.

In any of the above-mentioned schemes, it is preferred that the step 4 also includes performing basic erosion and expansion treatment on edge fractures caused by sharp changes in brightness and illumination.

Preferably in any of the above schemes, the effect of the corrosion is to eliminate the boundary points of the object, and the principle of the corrosion is: Among them, A is the target area on the plane (x, y), S is the template element of the specified size and shape, S(x, y) is the area represented by the template element S located on the coordinates (x, y), S (x, y)/A is a difference set, which means a set of elements that belong to S but not to A.

In any of the above schemes, it is preferred that the effect of the expansion is to expand the boundary point of the object, and the principle of the expansion is: Among them, A is the target area in the (x, y) image frame, S is the structural element of the specified size, and the area represented by the structural element S on the coordinate (x, y) is defined as S(x, y), S (x, yv∩A is an intersection, which means a set belonging to both S and A.

The present invention proposes a foreground extraction method based on neighborhood pixel intensity correction, which can accurately extract foreground objects, has clear algorithm ideas, is simple to implement, and can effectively resist changes in scene illumination and effectively eliminate the influence of camera shake on foreground object extraction. It has strong adaptability to dynamic scenes.

Description of drawings

FIG. 1 is a flowchart of a preferred embodiment of a foreground extraction method based on neighborhood pixel intensity correction according to the present invention.

Fig. 2 is an original image before erosion of a preferred embodiment of the foreground extraction method based on neighborhood pixel intensity correction according to the present invention.

FIG. 2A is an S1 template diagram of the embodiment shown in FIG. 2 of the foreground extraction method based on neighborhood pixel intensity correction according to the present invention.

FIG. 2B is an effect diagram after erosion according to the S1 template of the embodiment of the foreground extraction method based on neighborhood pixel intensity correction as shown in FIG. 2 according to the present invention.

FIG. 2C is an S2 template diagram of the embodiment shown in FIG. 2 of the foreground extraction method based on neighborhood pixel intensity correction according to the present invention.

FIG. 2D is an effect diagram after erosion according to the S2 template of the embodiment of the foreground extraction method based on neighborhood pixel intensity correction as shown in FIG. 2 according to the present invention.

Fig. 3 is an original image before dilation of a preferred embodiment of the foreground extraction method based on neighborhood pixel intensity correction according to the present invention.

FIG. 3A is an S3 template diagram of the embodiment shown in FIG. 3 of the foreground extraction method based on neighborhood pixel intensity correction according to the present invention.

FIG. 3B is an effect diagram after dilation according to the S3 template of the embodiment of the foreground extraction method based on neighborhood pixel intensity correction as shown in FIG. 3 according to the present invention.

FIG. 3C is an S4 template diagram of the embodiment shown in FIG. 3 of the foreground extraction method based on neighborhood pixel intensity correction according to the present invention.

FIG. 3D is an effect diagram after template expansion according to S4 of the embodiment of the foreground extraction method based on neighborhood pixel intensity correction as shown in FIG. 3 according to the present invention.

Detailed ways

The present invention will be further elaborated below in conjunction with the accompanying drawings and specific embodiments.

Embodiment one

As shown in FIG. 1 , step 100 is executed to extract a frame sequence. A continuous model based on neighborhood pixel intensity correction extracts moving objects from the background foreground by updating the motion information of the current frame. Step 105 is executed, and the first frame input from the video is used as the initial background, as shown in formula (1).

B ₁ (x, y) = F ₁ (x, y), formula (1)

Where B ₁ (x, y) and F ₁ (x, y) are the background pixel value at the coordinate point (x, y) and the pixel value of the current frame at this point, x≤P, y≤Q, P and Q denote the width and height of the initial background, respectively. for frame i The background image B _i used for the difference can be obtained from B _i-1 . First, the difference D _i between the current background and the current frame is given by the grayscale image calculation, D _i is given by equation (2),

The image difference D _i contains the moving target and noise, so D _i needs to be separated from the background image and the moving target by the threshold τ, as shown in formula (3).

In principle, moving objects are significantly different from noise and shadows, and a higher τ value can remove noise, but some moving pixels will be misjudged as noise points; on the contrary, if τ is too small, noise points will be misjudged It is judged as the moving target point. Therefore, τ will directly affect D _i , and an appropriate value of τ needs to be set in the experimental part. For a moving pixel where D _i =1, in order to reduce the abnormality of the shape as much as possible, we consider the stability S(x, y) of each pixel. S(x,y) is calculated by the number of times the pixel value changes and is updated every frame. If the pixel values in two consecutive frames have changed, the pixel is unstable and the original value should be retained. The stability of each pixel is calculated by formula (4) based on consecutive adjacent frames,

Wherein, S _i represents the stability matrix of frame i, whose initial value is zero, that is, S ₁ (x, y)=0. Through the cumulative calculation of the number of frames, it can be seen that the stable value of the non-moving target pixel is larger than the stable value of the moving target pixel.

In the calculation process, if the current background is close to the real background, the correction of the pixel value will not necessarily reduce the calculation efficiency. The specific judgment method is that if the minimum value of stability is greater than the given stability threshold δ, that is, min _{(x, y)} (S _i (x, y))≥δ, then the process of pixel value correction is no longer considered. So the current background remains until the next frame. For example, B _i =B _i-1 will be assigned to the i+1th frame, and then directly enter the stage of foreground object extraction. On the contrary, if min _{(x, y)} (S _i (x, y))<δ, the current background pixel value is corrected. The specific process can be expressed as follows:

How to correct pixel values when correction is required and both conditions exist.

(1) Set D _i (x, y) = 1 when it belongs to the current moving target;

(2) There is an unstable value such that S _i (x, y)<0, and in this case, formula (6) is used to re-construct to reduce the amount of calculation.

P _i {(x, y)|[D _i (x, y)]∩[S _i (x, y)＜0]}, formula (6)

Wherein, P _i is a series of pixel points to be filtered. It is used for correction by comparing the difference between each pixel of the background image and the current frame image. Then calculate the standard deviation between the corresponding pixel points, if the calculated value is small, it means that the pixel value is close to the average value. On the contrary, the pixel values are scattered. For a square moving window, the standard deviation σ is calculated by Equation (7) below.

Among them, n is the size of a square moving window, N=n ² represents the number of pixels, and the mean value μ is calculated from the pixel values of image I, as shown in formula (8).

For each pixel of _Pi , two standard deviations need to be calculated from the background image and the current frame and according to and These two standard deviations calculate the background pixel value B _i (x, y), as shown in equation (9).

The size (n*n) of the moving window can change during the motion. The size of the moving window affects the result of correcting pixel values. Parameters are set to illustrate the impact result, n ₀ and n ₁ represent the number of non-moving pixels and moving pixels respectively, and the corresponding pixel intensities are g ₀ and g ₁ respectively. Therefore, the entire moving window includes (n ₀ +n ₁ ) pixels. The calculation of the pixel mean is changed from Equation (8) to Equation (10).

The formula for calculating the standard deviation becomes Equation (11).

where |g ₀ -g ₁ | and (n ₀ +n ₁ ) are both constants, so equation (11) is given by to decide. Therefore, the standard deviation is determined by the number of non-moving pixels and moving pixels, and then affects the neighborhood pixel intensity correction model algorithm. Therefore, modifying the standard deviation parameters can adapt to various background scenes, such as low, medium, and high-speed object motion.

After pixel value intensity correction, the neighborhood pixel intensity correction model algorithm uses the detected background for subsequent foreground extraction, and takes the processing result as the input of the subsequent i+1th frame.

Through the background detected above, the next step is to extract the foreground target. The main idea of foreground extraction is based on adaptive background difference algorithm. The difference is obtained from the current frame and the background image through equation (2) to get the following equation (12):

image based diff The foreground object is separated, and the Otsu threshold method is used to determine the best threshold to extract the object from the background image. G ₀ is the background region and G ₁ is the foreground object region; thresholding can reduce classification errors by minimizing the intra-class variance. Find a reasonable threshold to minimize the weighted sum of the variances of the two classes, as shown in equation (13).

ωG ₀ (g) and ωG ₁ (g) denote the class probability at intensity g, and is the class variance. According to literature [13] the threshold is defined as formula (14), as shown below.

Then formula (3) becomes formula (15), as shown below.

produced is a grayscale image. In the foreground, there may be edge breaks caused by sharp changes in brightness and illumination, and basic erosion and expansion can be performed on them.

The role of erosion in mathematical morphology operations is to eliminate the boundary points of objects. In image processing, points smaller than structural elements can be eliminated by erosion operation, that is, the area containing small connected parts in the target area can be segmented by erosion processing.

The principle of corrosion is as follows:

Among them, A is the target area on the plane (x, y), S is the template element of the specified size and shape, S(x, y) is the area represented by the template element S located on the coordinates (x, y), S (x, y)/A is a difference set, which means a set of elements that belong to S but not to A.

In image processing, it can be understood that a template S is defined, and each pixel is scanned in the image from left to right and from top to bottom. When it is located at a certain pixel, the template is completely located in the target pixel, then Keep the current pixel point, otherwise, delete the current point.

The expansion of the image in the morphological operation is to expand the boundary points of the object. Through the expansion operation, some adjacent areas can be connected through specific structural elements. However, the expansion will also enlarge some small noise points and sensitive points. The principle of expansion is as follows:

Among them, A is the target area in the (x, y) image frame, S is the structural element of the specified size, and the area represented by the structural element S on the coordinate (x, y) is defined as S(x, y), S (x, y)∩A is an intersection set, which means a set belonging to both S and A. Starting from the upper left corner of the image, according to the scanning order, when the structural element is located on a certain pixel and there is an intersection between the structural element and the target, then keep this pixel, otherwise delete this pixel.

Embodiment two

The implementation process of corrosion is as follows: the template S is a cross-shaped structural element with a size of 3*3. The corroded target area A(x, y) is an 8*8 binarized image. In the process of scanning and corroding pixel by pixel, if the currently scanned pixel A(x, y) satisfies the pixel value 1 (assuming that 1 represents the pixel value of the black area in the figure), the values of the pixels directly adjacent up, down, left, and right in the eight-neighborhood are 1, and the other pixels with values of 0 are considered to match the template S, A(x , y) is set to 1, otherwise it is set to 0, if the eight neighbors of A(4, 4) are the first pixel that completely matches the value of the template S, then its value is 1 after the erosion operation. Using different templates to corrode images results in different results. Figure 2 shows image A before corrosion, and figure 2A shows template S1. Image 2B is obtained after using template S1 to corrode image A. FIG. 2C is a template S2, and an image 2D is obtained after using the template S2 to corrode the image A.

Embodiment Three

The expansion process of digital image is the same as the image erosion process, and it is also realized with the help of templates. The template is used to represent the shape of the structural elements, and the expansion process can be described as: moving the template in the target area to traverse every pixel of the target image. When the template When it is at the pixel coordinate (x, y), if the pixel corresponding to any position of 1 in the template is 1, then the pixel value at the position corresponding to the coordinate (x, y) after expansion processing is set to 1, otherwise the coordinate (x, y) ) is set to 0 at the corresponding position. Same as image erosion, the size and shape of the templates are different, and the dilated image and the original standard image are also different. Different from erosion, expansion does not necessarily completely contain the original target. If the defined template passes through the template center, that is, the template center is 1, the expanded image is a complete expansion of the original target. If the template is not 1, then The expanded image is not a complete expansion of the original target, but an expansion that corrodes some original pixels. FIG. 3 is the image B before etching, and FIG. 3A is the template S3, and the image 3B is obtained after the image B is etched using the template S1. FIG. 3C is a template S4, which is an image 3D obtained after etching the image B using the template S2.

Claims (10)

1. A foreground extraction method based on neighborhood pixel intensity correction comprises an input video and is characterized by further comprising the following steps:

step 1: acquiring an initial background according to a first frame of the frame sequence of the input video;

step 2: carrying out difference of foreground and background on the initial background;

and step 3: setting an image difference threshold value for stability calculation;

and 4, step 4: the Otsu threshold is used to determine the optimal threshold and to extract the object from the background image.

2. The foreground extraction method based on neighborhood pixel intensity correction as claimed in claim 1, wherein: calculating B in said step 1₁(x，y)＝F₁(x, y) wherein B₁(x, y) and F₁(x, y) are the background pixel value at coordinate point (x, y) and the pixel value of the current frame at that point, respectively, x ≦ P, y ≦ Q, P and Q representing the width and height of the initial background, respectively.

3. The foreground extraction method based on neighborhood pixel intensity correction as claimed in claim 2, wherein: for the ith frame, background image B for the difference_iCan be formed by B_i-1To obtain a mixture of, among others,

4. the foreground extraction method based on neighborhood pixel intensity correction of claim 3, wherein: said step 2 comprises a difference D between the current background and the current frame_iThe calculation formula is D_i＝|F_i(x，y)-B_i-1(x, y) |, wherein,

5. the foreground extraction method based on neighborhood pixel intensity correction of claim 4, wherein: the step 2 further comprises differentiating the image by D_iSeparated from background image and moving object by formulaWhere τ is a self-set threshold.

6. As in claimThe foreground extraction method based on neighborhood pixel intensity correction of claim 5, characterized by: said step 3 comprising setting S₁The initial value of (x, y) is zero, i.e. S₁(x, y) is equal to 0, and the stability S (x, y) of each pixel point is calculated by the formulaWherein S is_iIndicating that frame i is a stable matrix.

7. The foreground extraction method based on neighborhood pixel intensity correction of claim 6, wherein: the step 2 includes determining whether or not to perform correction of the pixel value.

8. The foreground extraction method based on neighborhood pixel intensity correction of claim 7, wherein: the judgment method is that when the minimum value min of the stability is adopted_(x，y)(S_i(x, y)) is smaller than a threshold value delta, then the current background pixel value is corrected; when the minimum value min of the stability is_(x，y)(S_i(x, y)) is equal to or greater than the threshold δ, the current background pixel value is not corrected.

9. The foreground extraction method based on neighborhood pixel intensity correction of claim 8, wherein: setting D when belonging to a currently moving object_i(x, y) is 1 and there is an unstable value such that S_iWhen (x, y) < 0, reconstructing to reduce the calculation amount, wherein the calculation formula is P_i＝{(x，y)|[D_i(x，y)＝1]∩[S_i(x，y)＜0]}。

10. The foreground extraction method based on neighborhood pixel intensity correction of claim 9, wherein: . The step 3 comprises calculating the standard deviation between corresponding pixel points by the formula Where N is the size of a square moving window, N ═ N²Representing the number of pixel points, the mean value μ being calculated from the pixel values of the image I, the formula being