CN115035164B - A moving target recognition method and device - Google Patents

Abstract

The present invention provides a moving target recognition method and device, wherein the moving target recognition method comprises: obtaining two adjacent frames of images of a target scene, wherein the target scene includes a target object; extracting pixel data corresponding to the two adjacent frames of images respectively; inputting the pixel data into an optical flow model, calculating the optical flow energy field corresponding to different weight parameters; classifying the optical flow energy field corresponding to different weight parameters, and determining the target weight parameter of the target scene where the target object is located based on the classification result; and calculating the motion state of the target object based on the optical flow model and the target weight parameter. By realizing flexible adjustment of the weight parameter for different target scenes, on the basis of quickly and accurately determining the target weight parameter, not only the accurate recognition of the motion state of the moving target is realized, but also the efficiency of identifying the moving target is further improved.

Description

A moving target recognition method and device

Technical Field

The present invention relates to the field of image recognition, and in particular to a moving target recognition method and device.

Background Art

Moving object detection plays an important role in video analysis, image processing, micro-nano operation, medical image analysis and other fields. Optical flow method is widely used in scenes such as foreground and obstacle detection, recognition, segmentation, tracking, navigation and shape information recovery of unmanned vehicles due to its advantages of high precision and rich information of motion or scene. Optical flow is the instantaneous speed of pixel movement of a moving object in space on the observation imaging plane. Optical flow method is a method that uses the change of pixels in the time domain in the image sequence and the correlation between adjacent frames to find the corresponding relationship between the previous frame and the current frame, thereby calculating the motion information of objects between adjacent frames.

Generally speaking, the calculation methods of optical flow can be divided into gradient-based methods (represented by HS (Horn Schunck) and LK (Lucas Kanade) and their derivative methods), matching-based methods, energy-based methods, phase-based methods and neural dynamics-based methods. Among them, with the gradual improvement of variational methods and partial differential equation theory, high-precision variational moving target recognition methods based on HS principles and their derivative algorithms have been widely studied. However, this method and its derivative algorithms are greatly affected by the weight coefficient of the smoothing term, and the adjustment process of the weight coefficient is very difficult, requiring multiple manual experiments, resulting in low efficiency in moving target recognition.

Summary of the invention

Therefore, the technical problem to be solved by the present invention is to overcome the defect that the adjustment process of the weight coefficient in the high-precision variational moving target recognition method based on the HS principle in the prior art and its derivative algorithm is very difficult, resulting in low efficiency of moving target recognition, thereby providing a moving target recognition method and device.

According to a first aspect, an embodiment of the present invention provides a moving target recognition method, the method comprising:

Acquire two adjacent frames of images of a target scene, wherein the target scene includes a target object;

Respectively extracting pixel data corresponding to the two adjacent frames of image;

Inputting the pixel data into an optical flow model to calculate the optical flow energy field corresponding to different weight parameters;

Classifying the optical flow energy fields corresponding to different weight parameters, and determining the target weight parameter of the target object in the target scene based on the classification result;

Based on the optical flow model and the target weight parameter, the motion state of the target object is calculated.

Optionally, the classifying the optical flow energy fields corresponding to different weight parameters and determining the target weight parameter of the target object in the target scene based on the classification result includes:

The optical flow energy fields corresponding to different weight parameters are clustered to obtain the rapid decline area, turning area and stable area;

The weight parameter corresponding to the turning zone cluster center is determined as the target weight parameter in the target scene where the target object is located.

Optionally, the calculating the motion state of the target object based on the optical flow model and the target weight parameter includes:

Inputting the pixel data into an optical flow model and performing pyramid filtering to obtain a first filtered image and a second filtered image;

Based on the target weight parameter, the first filtered image and the second filtered image, pyramid layered sampling is performed to obtain a first image and a second image respectively;

Calculate an optical flow vector result based on the target weight coefficient, the first image, and the second image;

The motion state of the target object is determined according to the optical flow vector result.

Optionally, determining the motion state of the target object according to the optical flow vector result includes:

When the optical flow vector result is greater than a preset threshold, determining that the target object is in motion;

When the optical flow vector result is not greater than a preset threshold, it is determined that the target object is in a stationary state.

Optionally, the method further comprises:

When the target object is in motion, all the optical flow vector results reaching a threshold are extracted to obtain pixel extraction data of the target object;

Based on the pixel extraction data of the target object, the moving position of the target object is marked on the original image.

Optionally, extracting data based on pixels of the target object and marking the moving position of the target object on the original image includes:

Obtaining position information of the target object in the original image;

Matching the position information of the original image with the pixel extraction data of the target object to obtain a matching result;

Based on the matching result, the moving position of the target object is marked on the original image.

Optionally, the method further comprises:

Acquire the scene image to be detected;

When the to-be-detected scene image is consistent with the target scene image, the target weight parameter under the target scene is determined as the target weight parameter under the to-be-detected scene.

According to a second aspect, an embodiment of the present invention provides a moving target recognition device, the device comprising:

An acquisition module, used to acquire two adjacent frames of images of a target scene, wherein the target scene includes a target object;

An extraction module, used for respectively extracting pixel data corresponding to the two adjacent frames of images;

A first calculation module, used for inputting the pixel data into an optical flow model to calculate the optical flow energy field corresponding to different weight parameters;

A second calculation module is used to classify the optical flow energy fields corresponding to different weight parameters, and determine the target weight parameter of the target scene where the target object is located based on the classification result;

The third calculation module is used to calculate the motion state of the target object based on the optical flow model and the target weight parameter.

According to a third aspect, an embodiment of the present invention provides an electronic device, including:

A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method described in the first aspect or any optional implementation manner of the first aspect by executing the computer instructions.

According to a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method described in the first aspect or any optional implementation manner of the first aspect.

The technical solution of the present invention has the following advantages:

The moving target recognition method and device provided by the present invention obtain two adjacent frames of images of a target scene, wherein the target scene includes a target object; extract pixel data corresponding to the two adjacent frames of images respectively; input the pixel data into an optical flow model, calculate the optical flow energy field corresponding to different weight parameters; classify the optical flow energy field corresponding to different weight parameters, and determine the target weight parameter in the target scene where the target object is located based on the classification result; and calculate the motion state of the target object based on the optical flow model and the target weight parameter. By inputting the pixel data of the two adjacent frames of images of the target scene into the optical flow model, the optical flow energy field corresponding to different weight parameters and the classification of the optical flow energy field are calculated, and the target weight parameter in the target scene where the target object is located is determined based on the classification result, so as to realize flexible adjustment of the weight parameter for different target scenes, and calculate the motion state of the target object based on the optical flow model and the target weight parameter, and on the basis of quickly and accurately determining the target weight parameter, not only the accurate recognition of the motion state of the moving target is realized, but also the efficiency of identifying the moving target is further improved.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG1 is a flow chart of a moving target recognition method according to an embodiment of the present invention;

FIG2 is a diagram showing a corresponding relationship between an optical flow energy field and weight parameters of a moving target recognition method according to an embodiment of the present invention;

3 is a schematic diagram of a monitoring target object of a moving target recognition method according to an embodiment of the present invention;

FIG4 is a schematic diagram of an optical flow field calculation result of a moving target recognition method according to an embodiment of the present invention;

FIG5 is a diagram showing the calculation results of optical flow vectors of a moving target recognition method according to an embodiment of the present invention;

FIG6 is a schematic diagram of target object cutting of a moving target recognition method according to an embodiment of the present invention;

FIG7 is a flowchart of an algorithm of a moving target recognition method according to an embodiment of the present invention;

FIG8 is a waveform comparison diagram of a moving target recognition method according to an embodiment of the present invention;

FIG9 is a comparison diagram of optical flow algorithm results of a moving target recognition method according to an embodiment of the present invention;

FIG10 is a schematic diagram of the structure of a moving target recognition device according to an embodiment of the present invention;

FIG. 11 is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention.

DETAILED DESCRIPTION

The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, it can also be the internal connection of two components, it can be a wireless connection, or it can be a wired connection. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

The optical flow method is a calculation method that uses the changes in pixels in the time domain in the image sequence and the correlation between adjacent frames to find the corresponding relationship between the previous frame and the current frame, thereby calculating the motion information of objects between adjacent frames. Among the gradient-based optical flow calculation methods, the high-precision variational optical flow calculation method based on the HS principle and its derivative algorithms have the following shortcomings:

1) Solving optical flow is a highly ill-posed problem. Using pure intensity-based constraints usually leads to underdetermined equations and aperture problems.

2) The optical flow method based on HS theory has poor robustness and cannot accurately handle scenes with changing illumination, scenes with large gradient boundary areas, and scenes with large displacement of multiple targets;

3) The optical flow method based on HS theory calculates dense optical flow, which is generally not very efficient;

4) The optical flow method based on HS theory is greatly affected by the weight parameters of the smoothing term and is difficult to adjust. It requires multiple manual experiments and has poor adaptability.

The optical flow variational model based on the HS principle consists of two core parts: data term and smoothing term. The basic model is described as follows:

Among them, E(u,v) is the result of optical flow field calculation; S is the pixel data set corresponding to the entire image; _Ix is the pixel gradient change value in the x direction; _Iy is the pixel gradient change value in the y direction; _It is the pixel gradient change value of the two adjacent frames; u and v are the optical flow vectors in the x and y directions respectively; represents the second-order Laplace operator of u; represents the second-order Laplacian of v.

In view of the above problems, an embodiment of the present invention provides a moving target recognition method for solving the deficiencies in the above-mentioned variational optical flow calculation method, and realizes relatively fast, robust and high-precision optical flow calculation by improving the data terms and smoothing terms in the model.

As shown in FIG1 , the moving target recognition method specifically includes the following steps:

Step S101: acquiring two adjacent frames of images of a target scene, where the target scene includes a target object.

Specifically, in practical applications, the image of the target scene can be acquired through a camera, but the actual situation is not limited to this. The images of the target scene and the target object can be acquired by any image acquisition device, and the target scene and the target object in the image can be separated, thereby laying the foundation for the subsequent accurate identification of the motion state of the target object.

The moving target recognition method provided by the embodiment of the present invention can be applied to scenes such as traffic roads, and provides theoretical and image technology support for obtaining the motion state of the target object.

Step S102: extracting pixel data corresponding to two adjacent frames of images respectively.

Step S103: Input the pixel data into the optical flow model to calculate the optical flow energy field corresponding to different weight parameters.

Specifically, in practical applications, the embodiment of the present invention improves the basic model of the optical flow variational model, and the improved TV-L ¹ optical flow variational model (i.e., optical flow model) is as follows:

Where E(u,v) is the result of optical flow calculation; _Gt is a filter based on trigonometric polynomial expansion; Ω represents the range of the target scene; _I0 and _I1 are the two frames of images before and after the motion respectively; x = ( _px , _py ) ^T is the coordinate of a pixel point on the image; λ is the weight parameter of the smoothing term; That is represents the second-order Laplace operator of u; h(x)＝[u(x),v(x)] ^T is the optical flow vector in the x and y directions to be solved.

Regarding the smoothing term in the basic model: the embodiment of the present invention introduces the K-Means algorithm based on energy field classification, so that the model has the ability to adaptively select the weight coefficient in the smoothing term. Specifically, in practical applications, the K-Means algorithm based on energy field classification in the embodiment of the present invention is used for weight estimation, but the actual situation is not limited to this. The changes in the type and number of algorithms to determine the optimal weight parameters are also within the protection scope of the moving target recognition method provided by the embodiment of the present invention. The use of the K-Means algorithm based on energy field classification for weight estimation can make up for the traditional method of using a large number of experiments to determine the value of the weight parameters, saving a lot of processing time. When selecting weight parameters for unknown scenes, the optical flow energy fields corresponding to different weight parameters are analyzed to quickly determine the weight parameters suitable for the current scene, thereby further improving the efficiency of identifying moving targets on the basis of achieving accurate identification of the motion state of the moving target.

Step S104: classify the optical flow energy fields corresponding to different weight parameters, and determine the target weight parameter of the target scene where the target object is located based on the classification result.

Specifically, in practical applications, the embodiments of the present invention classify the optical flow energy fields of common scenes corresponding to different weight parameters, determine the classification of multiple types of optical flow energy fields based on the optical flow variational model, and compare the target scenes with the common scenes in the model to determine the target weight parameters of the target scene where the target object is located.

The embodiment of the present invention determines the weight parameters suitable for the current common scenes by comparing and matching different weight parameters with the optical flow energy fields of multiple common scenes. After the image data collected by the image acquisition device is input into the optical flow variational model, the target weight parameters can be directly determined by comparing the target scene, target object and the common scenes stored in the model, thereby realizing adaptive adjustment of the weight parameters in unknown scenes.

Step S105: Based on the optical flow model and the target weight parameter, the motion state of the target object is calculated.

Specifically, in practical applications, the embodiments of the present invention calculate the pixel data of the target object based on the optical flow variational model and the target weight parameter, thereby obtaining the motion state of the target object and displaying the motion state of the target object.

By executing the above steps, the moving target recognition method provided by the embodiment of the present invention obtains two adjacent frames of images of the target scene, and the target scene includes the target object; extracts the pixel data corresponding to the two adjacent frames of images respectively; inputs the pixel data into the optical flow model, and calculates the optical flow energy field corresponding to different weight parameters; classifies the optical flow energy field corresponding to different weight parameters, and determines the target weight parameter in the target scene where the target object is located based on the classification result; and calculates the motion state of the target object based on the optical flow model and the target weight parameter. By inputting the pixel data of the two adjacent frames of images of the target scene into the optical flow model, the optical flow energy field corresponding to different weight parameters and the classification of the optical flow energy field are calculated, and the target weight parameter in the target scene where the target object is located is determined based on the classification result, so as to realize flexible adjustment of the weight parameter for different target scenes, and calculate the motion state of the target object based on the optical flow model and the target weight parameter, and on the basis of quickly and accurately determining the target weight parameter, not only the accurate recognition of the motion state of the moving target is realized, but also the efficiency of identifying the moving target is further improved.

Specifically, in one embodiment, the above step S104 classifies the optical flow energy fields corresponding to different weight parameters, and determines the target weight parameter of the target scene where the target object is located based on the classification result, which specifically includes the following steps:

Step S201: clustering the optical flow energy fields corresponding to different weight parameters to obtain a rapid drop area, a turning area and a stable area.

Step S202: Determine the weight parameter corresponding to the turning zone cluster center as the target weight parameter in the target scene where the target object is located.

Specifically, in practical applications, the embodiments of the present invention train and determine the weight parameters based on the optical flow variational model, and adopt a variety of common scenarios to classify the optical flow energy fields of different weight parameters, and determine the target weight parameters of the target object in the target scene based on the classification results, so as to directly compare the scene to be detected with the target scene, thereby greatly shortening the selection and processing time of the weight parameters and laying the foundation for quickly determining the target weight parameters.

Specifically, the K-Means algorithm based on energy field classification is used for weight estimation through three steps: calculation of energy field > screening of effective data of energy field > adaptive selection of weight parameters by K-Means.

Specifically, for different application scenarios, the embodiment of the present invention takes four common scenarios as examples, selects four groups of image sequences, and explains the process of determining the weight parameters of each scene. However, the actual situation is not limited to this. The actual situation can be trained and determined according to this process, which will not be repeated here.

Specifically, the four groups of image sequences in the embodiment of the present invention are "Army-Group1, Mequon-Group2, Evergreen-Group3, Basketball-Group4", and the energy field distribution corresponding to different image sequences at different λ is shown in Figure 2. It can be seen that although the image sequences are different, the energy fields and λ are also different, but the same is the trend of the energy curve. The energy fields corresponding to different λ can be divided into three categories: rapid decline area, turning area and stable area. Because the optical flow field represented by the turning area has a large density and a low noise level, the energy field can be classified by the K-Means method. The specific process is as follows:

① Data normalization: unify the data of different image sequences through preprocessing methods to facilitate subsequent comparison.

② Select the initialized k samples as the initial clustering centers, k is the energy field classification result, that is, k = 3, that is, a _i = a ₁ , a ₂ , a ₃ , where a _i is the i-th clustering center, i = 1, 2, 3.

③ According to the energy field of each λ _i , calculate the distance to the k cluster centers and divide them into the class corresponding to the cluster center with the smallest distance.

④ For each a _i , recalculate the cluster center Among them, x is the sample data of weight parameter λ; _ci is the i-th cluster sample.

Specifically, as shown in FIG2 , _c1 , _c2 , and _c3 respectively represent cluster sets of three types of energy fields corresponding to different λ, namely, a set of sample data of weight parameters in a rapid decline zone, a set of sample data of weight parameters in a turning zone, and a set of sample data of weight parameters in a stable zone. Exemplarily, the embodiment of the present invention takes _c1 as a set of weight parameters in a rapid decline zone, _c2 as a set of weight parameters in a turning zone, and _c3 as a set of weight parameters in a stable zone as an example for explanation, but the actual situation is not limited thereto, and the correspondence between _c1 and the energy field may be changed according to actual needs.

⑤Set the termination condition, which is to limit the number of iterations here, and repeat steps ③-④ until the rapid decline area, turning area and stable area are separated.

⑥ Take the turning zone cluster center as the optimal solution of the turning zone as output.

The embodiment of the present invention can realize the adaptive selection of the weight parameter λ through the processes ①-⑥, and the classification method is highly efficient.

The center of the cluster represents the distribution of data of the same category and is the representative of a type of data. In an embodiment of the present invention, the turning area is the area where the energy characteristics change rapidly, and the cluster center can best represent and reflect the energy characteristics of the image. Using the turning area cluster center value as the weight parameter can fully reflect the energy characteristics of the target scene and the target object, and realize accurate and precise identification of the motion state of the target object.

The clustering method adopted in the embodiment of the present invention refers to the relevant description of the clustering calculation process in the prior art, which will not be described in detail here.

Specifically, in one embodiment, the above step S104 further includes the following steps:

Step S301: Acquire an image of a scene to be detected.

Step S302: When the image of the scene to be detected is consistent with the image of the target scene, the target weight parameters of the target scene are determined as the target weight parameters of the scene to be detected.

Specifically, in practical applications, after the optical flow variational model is trained, after the pixel data of the scene image to be detected is input into the optical flow variational model, the target scene image in the optical flow variational model will be compared first. When the scene image to be detected is consistent with the target scene image, the target weight parameters under the target scene are determined as the target weight parameters under the scene to be detected, thereby avoiding the process of repeatedly calculating the weight parameters and greatly improving the detection efficiency of moving targets.

Specifically, in one embodiment, the above step S105 calculates the motion state of the target object based on the optical flow model and the target weight parameter, and specifically includes the following steps:

Step S401: inputting the pixel data into an optical flow model and performing pyramid filtering to obtain a first filtered image and a second filtered image.

Specifically, in practical applications, the embodiments of the present invention perform pyramid layering processing, namely downsampling processing, by inputting pixel data into an optical flow variational model. The downsampling processing includes filtering and modifying the image size.

Specifically, compared to the traditional pyramid method using Gaussian filter, the embodiment of the present invention uses an optimized bilateral filter to ensure the edge clarity of the target image and the target object. The optimized bilateral filter structure includes a Gaussian filtering process, so that the advantages of the traditional Gaussian filter can be fully retained while further improving the filtering effect. In addition, in order to speed up the calculation speed of the bilateral filtering, the embodiment of the present invention uses an O(1) complexity trigonometric function polynomial approximation to replace the Gaussian filter structure in the bilateral filtering, thereby greatly improving the calculation efficiency and enhancing the edge clarity capability of the target detection.

The optimization of the data items in the basic model in the embodiment of the present invention includes: ① introducing a filter in the outer layer of the integral term, usually a Gaussian filter is selected, and the Gaussian filter is replaced by a trigonometric function polynomial expansion to reduce the filtering complexity from O(n2) to O(1), while improving the model's anti-noise ability and anti-interference ability to illumination changes; ② changing the second-order penalty function in the data item to the first order, so that the optical flow calculation maintains clear edges in places where the gradient changes are large.

Step S402: performing pyramid hierarchical sampling based on the target weight parameter, the first filtered image and the second filtered image to obtain a first image and a second image respectively.

Specifically, regarding the process of modifying the image size, the embodiment of the present invention reduces the length and width of the two input frames of images respectively. Exemplarily, it can be reduced to half of the original size, that is, the overall reduction is 4 times (the number of downsampling layers can be manually set according to the image size, generally 3-5 layers), thereby obtaining a first image of the original size and a reduced second image.

The pyramid layering algorithm can make two frames of images start from the lowest resolution for target detection and then gradually project them to the original resolution. Therefore, even if the initial input is not two adjacent frames of images, for example, the first frame and the fifth frame of images are input, the pyramid layering algorithm can complete the target detection from the low-resolution image and then project it to the high-resolution image to successfully solve the problem of large displacement (i.e., frame skipping detection).

Step S403: Calculate and obtain an optical flow vector result based on the target weight coefficient, the first image, and the second image.

Specifically, in practical applications, the embodiment of the present invention calculates the optical flow vector result of the target object based on formula (2) through the target weight parameter λ, the first image and the second image, wherein the optical flow vector result includes the optical flow direction and the optical flow displacement information.

By optimizing the optical flow algorithm, the embodiment of the present invention adopts trigonometric function polynomials instead of Gaussian filters to reduce the complexity of the algorithm; by effectively determining the target weight parameters, the accuracy of the optical flow field and the optical flow vector calculation results is guaranteed.

Step S404: determining the motion state of the target object according to the optical flow vector result.

Specifically, in practical applications, the embodiments of the present invention effectively display the optical flow vector results on the target object, thereby not only being able to determine the motion state of the target object, but also being able to make certain predictions about the motion trend of the target object, and being more forward-looking.

Specifically, in one embodiment, the above step S404 specifically includes the following steps:

Step S501: When the optical flow vector result is greater than a preset threshold, the target object is determined to be in motion.

Step S502: When the optical flow vector result is not greater than a preset threshold, the target object is determined to be stationary.

Specifically, in practical applications, in order to quickly determine the motion state of the target object, the embodiment of the present invention sets a preset threshold. When the optical flow vector result is greater than the preset threshold, the target object is judged to be in motion; when the optical flow vector result is not greater than the preset threshold, the target object is judged to be stationary. The value of the preset threshold can be set according to actual conditions to meet the usage requirements of different users.

Specifically, in one embodiment, after executing the above step S501 to determine that the target object is in motion, the following steps are further included:

Step S601: when the target object is in motion, all optical flow vector results reaching a threshold are extracted to obtain pixel extraction data of the target object.

Step S602: extracting data based on the pixels of the target object and marking the moving position of the target object on the original image.

Exemplarily, in combination with FIG. 3-FIG. 5, the embodiment of the present invention first obtains two adjacent frames of images through monitoring video; the two frames of images are input into the optical flow variational model to obtain the optical flow field calculation result of the target object in the target scene, as shown in FIG. 4, the horizontal coordinates 0-639 pixels and 640-1280 pixels represent the flow field in the x direction (i.e., u direction) and the y direction (i.e., v direction), respectively; the optical flow vector calculation result of the target object is calculated according to formula (2), and by enlarging the optical flow vector result diagram (as shown in the right figure in FIG. 5), the motion direction and motion displacement of each position coordinate of the target object can be accurately displayed. According to FIG. 5, it can be seen that the larger the target motion amplitude, the higher the vector value; further, as shown in FIG. 6, after determining that the target object is in a moving state, the embodiment of the present invention can also determine the position of the target object in the target scene in the original video image according to the optical flow field calculation result and the optical flow vector calculation result, and segment it from the target scene for marking and display.

The embodiment of the present invention obtains two adjacent frames of images, inputs the pixel data of the images into the optical flow variational model, processes the pixel data through the target weight parameters corresponding to the target scene, and obtains the optical flow field calculation results and the optical flow vector calculation results respectively. The user can intuitively view the movement direction and movement displacement of the target object through the local magnified image of the optical flow vector, which not only greatly improves the recognition of the movement state of the target object (i.e., the moving target), but also can make a certain prediction on the movement trend of the target object to meet the needs of the user. On this basis, the embodiment of the present invention can also segment the target object from the target scene, and emphasize the position of the target object in the target scene, so as to facilitate the user to quickly determine the position of the target object in the original image.

Specifically, in one embodiment, the above step S602 extracts data based on the pixels of the target object and marks the moving position of the target object on the original image, which specifically includes the following steps:

Step S701: Obtain the position information of the target object in the original image.

Step S702: Match the position information of the original image with the pixel extraction data of the target object to obtain a matching result.

Step S703: Based on the matching result, the moving position of the target object is marked on the original image.

Specifically, in practical applications, by obtaining the position information of the target object in the original image and matching it with the pixel extraction data of the target object in the optical flow variational model, the motion position of the target object is marked on the original image based on the matching result, thereby quickly determining the motion state of the target object and facilitating the user to quickly determine the position of the target object in the original image.

By executing the above steps, the moving target recognition method provided by the embodiment of the present invention inputs the pixel data of two adjacent frames of the target scene into the optical flow model, calculates the optical flow energy field corresponding to different weight parameters and the classification of the optical flow energy field, and determines the target weight parameters in the target scene where the target object is located based on the classification result, thereby realizing flexible adjustment of the weight parameters for different target scenes, and calculating the motion state of the target object based on the optical flow model and the target weight parameters. On the basis of quickly and accurately determining the target weight parameters, not only the accurate recognition of the motion state of the moving target is realized, but also the efficiency of identifying the moving target is further improved.

The moving target recognition method provided by the embodiment of the present invention will be described in detail below with reference to specific application examples.

In conjunction with FIG. 1 to FIG. 8 , the embodiments of the present invention optimize the basic model (Formula (1)) respectively, as described in detail as follows:

1) For the data item: ① Introduce a filter in the outer layer of the integral item. Usually, a Gaussian filter is selected, and the trigonometric function polynomial expansion is used to replace the Gaussian filter to reduce the filtering complexity from O(n ² ) to O(1), while improving the model's anti-noise ability and anti-interference ability to illumination changes; ② Change the second-order penalty function in the data item to the first order, so that the optical flow calculation maintains clear edges in places where the gradient changes are large;

2) For smooth terms: the K-Means algorithm based on energy field classification is introduced, so that the model has the ability to adaptively select weight coefficients in smooth terms;

3) For the whole model: introduce the optimized bilateral filtering algorithm and pyramid layering algorithm. When constructing a multi-level resolution image in the pyramid, use the optimized bilateral filter for filtering and then downsample, so that the edge of the object in the solved optical flow field can maintain high-precision optical flow estimation. In addition, the Gaussian term designed in the optimized bilateral filtering algorithm can be directly replaced by the trigonometric function polynomial expansion in 1) to achieve algorithm reuse.

The improved TV- ^L1 optical flow variational model is shown in formula (2) and will not be described in detail here.

The embodiment of the present invention is implemented through three core steps, as shown in FIG7 , which are specifically described as follows:

1) Data item optimization

In the traditional optical flow method based on the HS model, the data item is an ^L2 penalty function, which will cause nonlinear amplification of the error of the energy conservation assumption and affect the edge of the flow field. In order to increase the robustness of the data item flow field estimation, the present invention introduces an ^L1 non-square penalty function in the data item. At the same time, in order to ensure the accuracy of the data item flow field estimation, an optimized filter is added to the outer layer of the integral term. Under normal circumstances, the Gaussian filter is a conventional choice, but in the optical flow estimation method based on the HS model, since HS belongs to the category of dense optical flow calculation, its efficiency itself does not have an advantage. If the traditional filter is still used, its convolution process will cause the program execution time to increase and reduce real-time performance. To this end, the embodiment of the present invention introduces a trigonometric function polynomial expansion to approximate the Gaussian filter, so that it can have similar performance to the Gaussian filter when the order is specified. The expression of the filter is:

Where t is the current time, t∈[-T, T]; λ=π/2T, T is the dynamic range of the pixels in the current frame of the processed image sequence; N is the order of the polynomial, N=2, 3, …, i; i is the total order, i∈[0+∞]. In general, when N is large enough, the following approximation exists:

Where cos(*) is the cosine trigonometric function, which is formally converted into the expression of the Gaussian function, where the domain of cos(*) is [-π/2, π/2]; Among them, σ is the standard deviation of the Gaussian function, and ρ is the scale factor generated during the approximate processing. Figure 8 shows a comparison diagram of the function curve of the trigonometric polynomial and the curve of the Gaussian function. It can be seen that under certain conditions, as the order N increases, the waveform of the dotted line is getting closer and closer to the waveform of the solid line, that is, the waveform of the trigonometric polynomial is getting closer and closer to the Gaussian curve. Experiments have shown that when N=4, the Gaussian curve can be well approximated, thereby achieving replacement. Since the two-dimensional Gaussian function is implemented through a double-layer loop during the encoding process, its complexity is O(n ² ). After being replaced by a trigonometric polynomial, multiplication can be converted into addition, thereby reducing the complexity to O(1). In addition, during the encoding implementation process, for the addition and multiplication of the processed image pixels, the embodiment of the present invention adopts an instruction set method for acceleration, such as the SSE instruction set, which uses a 128-bit register to read 8 pixel values at a time, perform addition or multiplication operations, and then save them, thereby improving execution efficiency.

2) Smoothness Optimization

A key parameter is involved in the smoothing term, namely, λ in formula (2), which represents the weight parameter of the smoothing term. The selection of this parameter will affect the estimation accuracy of the entire flow field. For example, when λ is 50, 100 and 180, there are differences in the optical flow estimation fields in the u and v directions. Therefore, the weight parameter has a greater impact on the accuracy. The traditional method is to determine the value of the weight parameter in the model through prior knowledge or a large number of experiments. These methods are not very applicable in the face of unknown scenes and waste a lot of time. Therefore, the embodiment of the present invention introduces a K-Means algorithm based on energy field classification for weight estimation to solve the above problem.

3) Flow field edge and large displacement optimization

After "1) data term optimization" and "2) smooth term optimization", high-precision estimation of variational optical flow field with micro-displacement at the same scale can be achieved. However, in practical applications, if the object to be measured moves greatly or the illumination changes dramatically at a certain moment, the above "1) data term optimization" and "2) smooth term optimization" processes alone cannot accurately estimate the optical flow field, and the edges of the objects at the gradient changes are relatively blurred, making it impossible to perform accurate numerical calculations of the optical flow field. Therefore, the intervention of flow field edge and large displacement optimization methods is required.

In large displacement calculations, the algorithm based on pyramid layering has been proven to be very effective, but in the process of pyramid construction, Gaussian filtering is required before downsampling between each two layers to prevent the jagged effect of the downsampled image. The general algorithm using Gaussian filtering is very effective, but the estimation of the optical flow field is more sensitive to gradient changes. Small changes will make the optical flow field boundary blurred, resulting in reduced robustness of the calculation. Therefore, the embodiment of the present invention introduces a filter with strong edge retention ability as a processing tool for constructing pyramid inter-layer images. Taking bilateral filtering as an example, if the traditional bilateral filtering algorithm is directly used, a high-precision edge can be obtained, but the algorithm itself is time-consuming. Benefiting from the idea of using trigonometric function polynomial expansion to approximate Gaussian filtering in "1) Data Item Optimization", this operation is introduced into the traditional bilateral filtering, nested with the pyramid layering algorithm and encoded and implemented in conjunction with the instruction set method to form a set of optimization algorithms.

The embodiment of the present invention first acquires two adjacent frames of images of the target scene and extracts the corresponding pixel data, and performs downsampling processing on the two frames of images based on a pyramid layering algorithm, thereby realizing an optimization process for flow field edge and large displacement processing.

By inputting pixel data into the optical flow variational model and performing pyramid layered processing, the optical flow energy field results corresponding to different weight parameters are calculated based on the target scene and target object. By classifying the optical flow energy field results, the target weight parameters of the target object in the target scene are determined. The motion state of the target object is calculated based on the optical flow model and the target weight parameters, and the optical flow vector result of the target object is displayed on the original image.

By completing all the above steps, the optimization estimation based on TV-L ¹ adaptive optical flow field can be fully realized. In order to verify the correctness of the results, the embodiment of the present invention also carried out simulation verification for this process, and the simulation results are shown in Figure 9, wherein the first column is the optical flow field image obtained based on theoretical calculation; the second column is the optical flow field image calculated based on the LK optical flow method; the third column is the optical flow field image calculated based on the HS optical flow method; the fourth column is the optical flow field image calculated based on the block optical flow method; the fifth column is the optical flow field image calculated by the moving target recognition method proposed in the embodiment of the present invention.

Exemplarily, the embodiment of the present invention uses the data of the standard test set from the MIDDLEBURY optical flow database as an example to perform simulation and obtain simulation results.

As can be seen from Figure 9, when processing the two frames of images "Urban3" and "Venus" in the data set, compared with the theoretical optical flow field, the estimation result of the LK optical flow method is relatively sparse, and the description of the object gradient edge optical flow is poor; the HS optical flow method has higher accuracy than the LK, but the flow field estimation in adjacent areas is uneven and is greatly affected by the artificial selection of weight coefficients; the block-based optical flow method has clear boundary optical flow and is suitable for processing high-contrast image sequence flow field estimation, but it is easy to have the phenomenon of erroneous optical flow estimation, making the flow field estimation more chaotic; and the TV-L ¹ adaptive optical flow optimization estimation method proposed in the embodiment of the present invention can realize the selection of adaptive weight coefficients, and the obtained flow field estimation is clearer and more uniform at the edge. The detection performance of the algorithm in "Urban3" with more occlusion is also better than the other three. The method proposed in the embodiment of the present invention is comprehensive and has better universality, while the other three methods have their own outstanding application fields. For example, although the block-based method will estimate more erroneous optical flows, it works very well at the edge with large gradient changes.

In order to quantify the accuracy of the optical flow field estimation of the method proposed in the embodiment of the present invention, the average angle error (AAE), average endpoint error (AEPE) and average standard deviation (ASTD) are used to measure the optical flow field estimation results in Figure 9 as shown in Table 1. It can be seen that under the three error evaluation standards, the error of the method proposed in the embodiment of the present invention is not optimal under a certain indicator, but the overall performance is optimal, the three errors are all maintained at a low level, and the calculation speed is 0.14s and 0.27s respectively, which is 1-2.5 times more efficient than the other three methods.

Table 1 Comparison of algorithm error results

Urban3 LK HS SD The method proposed in the embodiment of the present invention AAE 12.0418 15.1384 29.4743 14.3524 AEPE 1.7547 1.5200 4.7190 0.7857 ASTD 28.9766 31.7956 43.0622 31.0202

Venus LK HS SD The method proposed in the embodiment of the present invention AAE 12.7544 15.4528 12.5920 12.5015 AEPE 0.7837 1.0784 1.1132 0.6928 ASTD 28.4428 27.6540 29.0916 26.4893

In summary, the greatest advantage of the moving target recognition method provided by the embodiment of the present invention is that it provides a performance optimization method from the basic model level, and provides scholars and engineers with a set of solutions that can achieve high precision in variational optical flow calculation, low complexity of dense optical flow field, strong noise resistance, optical flow field boundary preservation, and adaptive weight parameter selection. The performance of the algorithm is verified through simulation experiments, providing an effective reference for R&D and application personnel.

An embodiment of the present invention provides a moving target recognition device, as shown in FIG10 , the moving target recognition device includes:

The acquisition module 101 is used to acquire two adjacent frames of images of a target scene, where the target scene includes a target object. For details, please refer to the relevant description of step S101 in the above method embodiment, which will not be repeated here.

The extraction module 102 is used to extract pixel data corresponding to two adjacent frames of images respectively. For details, please refer to the relevant description of step S102 in the above method embodiment, which will not be repeated here.

The first calculation module 103 is used to input pixel data into the optical flow model and calculate the optical flow energy field corresponding to different weight parameters. For details, please refer to the relevant description of step S103 in the above method embodiment, which will not be repeated here.

The second calculation module 104 is used to classify the optical flow energy fields corresponding to different weight parameters, and determine the target weight parameter of the target scene where the target object is located based on the classification result. For details, please refer to the relevant description of step S104 in the above method embodiment, which will not be repeated here.

The third calculation module 105 is used to calculate the motion state of the target object based on the optical flow model and the target weight parameter. For details, please refer to the relevant description of step S105 in the above method embodiment, which will not be repeated here.

For further description of the above-mentioned moving target recognition device, please refer to the relevant description of the above-mentioned moving target recognition method embodiment, which will not be repeated here.

Through the coordinated cooperation of the above-mentioned components, the moving target recognition device provided by the embodiment of the present invention inputs the pixel data of two adjacent frames of images of the target scene into the optical flow model, calculates the optical flow energy field corresponding to different weight parameters and the classification of the optical flow energy field, and determines the target weight parameters of the target scene where the target object is located based on the classification result, thereby realizing flexible adjustment of the weight parameters for different target scenes, and calculating the motion state of the target object based on the optical flow model and the target weight parameters. On the basis of quickly and accurately determining the target weight parameters, not only the accurate recognition of the motion state of the moving target is realized, but also the efficiency of identifying the moving target is further improved.

An embodiment of the present invention provides an electronic device, as shown in FIG11 , the electronic device includes a processor 901 and a memory 902 , the memory 902 and the processor 901 are communicatively connected with each other, wherein the processor 901 and the memory 902 may be connected via a bus or other means, and FIG11 takes the connection via a bus as an example.

The processor 901 may be a central processing unit (CPU). The processor 901 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

The memory 902 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions/modules corresponding to the method in the embodiment of the present invention. The processor 901 executes various functional applications and data processing of the processor 901 by running the non-transitory software programs, instructions and modules stored in the memory 902, that is, implements the method in the above method embodiment.

The memory 902 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created by the processor 901, etc. In addition, the memory 902 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 902 may optionally include a memory remotely arranged relative to the processor 901, and these remote memories may be connected to the processor 901 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

One or more modules are stored in the memory 902 , and when executed by the processor 901 , the method in the above method embodiment is executed.

The specific details of the above electronic device can be understood by referring to the corresponding descriptions and effects in the above method embodiments, and will not be repeated here.

Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the implemented program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims (5)

1. A moving target recognition method, comprising: Acquire two adjacent frames of images of a target scene, wherein the target scene includes a target object; Respectively extracting pixel data corresponding to the two adjacent frames of image; The pixel data is input into the optical flow model to calculate the optical flow energy field corresponding to different weight parameters; the optical flow model is as follows: Where, E ( u , v ) is the result of optical flow field calculation _; Gt is a filter based on trigonometric polynomial expansion; _Ω represents the range of the target scene; I0 and I1 are the two frames of images before and after the motion _respectively ; x =( px , py ) ^T is the coordinate of a pixel point on the image; _λ is the weight parameter of the smoothing term _; That is , represents the second-order Laplace operator of u ; h ( x ) = [ u ( x ), v ( x )] ^T is the optical flow vector in the x and y directions to be solved; Classifying the optical flow energy fields corresponding to different weight parameters, and determining the target weight parameter of the target object in the target scene based on the classification result; Based on the optical flow model and the target weight parameter, a motion state of the target object is calculated; The step of classifying the optical flow energy fields corresponding to different weight parameters and determining the target weight parameter of the target scene where the target object is located based on the classification result includes: The optical flow energy fields corresponding to different weight parameters are clustered to obtain the rapid decline area, turning area and stable area; Determine the weight parameter corresponding to the turning zone cluster center as the target weight parameter in the target scene where the target object is located; The step of calculating the motion state of the target object based on the optical flow model and the target weight parameter includes: The pixel data is input into an optical flow model and subjected to pyramid filtering to obtain a first filtered image and a second filtered image; Based on the target weight parameter, the first filtered image and the second filtered image, pyramid layered sampling is performed to obtain a first image and a second image respectively; Calculate an optical flow vector result based on the target weight coefficient, the first image, and the second image; Determine the motion state of the target object according to the optical flow vector result; The determining the motion state of the target object according to the optical flow vector result includes: When the optical flow vector result is greater than a preset threshold, determining that the target object is in motion; When the optical flow vector result is not greater than a preset threshold, determining that the target object is in a stationary state; The method further comprises: When the target object is in motion, all the optical flow vector results reaching a threshold are extracted to obtain pixel extraction data of the target object; Extracting data based on the pixels of the target object, marking the moving position of the target object on the original image; The extracting data based on the pixels of the target object and marking the moving position of the target object on the original image comprises: Obtaining position information of the target object in the original image; Matching the position information of the original image with the pixel extraction data of the target object to obtain a matching result; Based on the matching result, the moving position of the target object is marked on the original image. 2. The method according to claim 1, characterized in that the method further comprises: Acquire the scene image to be detected; When the to-be-detected scene image is consistent with the target scene image, the target weight parameter under the target scene is determined as the target weight parameter under the to-be-detected scene. 3. A moving target recognition device, comprising: An acquisition module, used for acquiring two adjacent frames of images of a target scene, wherein the target scene includes a target object; An extraction module, used for respectively extracting pixel data corresponding to the two adjacent frames of images; The first calculation module is used to input the pixel data into the optical flow model to calculate the optical flow energy field corresponding to different weight parameters; the optical flow model is as follows: Where, E ( u , v ) is the result of optical flow field calculation _; Gt is a filter based on trigonometric polynomial expansion; _Ω represents the range of the target scene; I0 and I1 are the two frames of images before and after the motion _respectively ; x =( px , py ) ^T is the coordinate of a pixel point on the image; _λ is the weight parameter of the smoothing term _; That is , represents the second-order Laplace operator of u ; h ( x ) = [ u ( x ), v ( x )] ^T is the optical flow vector in the x and y directions to be solved; The second calculation module is used to classify the optical flow energy fields corresponding to different weight parameters, and determine the target weight parameters of the target scene where the target object is located based on the classification results; the classification of the optical flow energy fields corresponding to different weight parameters, and determining the target weight parameters of the target scene where the target object is located based on the classification results, includes: clustering the optical flow energy fields corresponding to different weight parameters to obtain a rapid decline area, a turning area and a stable area; and determining the weight parameter corresponding to the cluster center of the turning area as the target weight parameter of the target scene where the target object is located; A third calculation module is used to calculate the motion state of the target object based on the optical flow model and the target weight parameter; the calculation of the motion state of the target object based on the optical flow model and the target weight parameter includes: inputting the pixel data into the optical flow model, performing pyramid filtering processing, and obtaining a first filtered image and a second filtered image; performing pyramid layered sampling based on the target weight parameter, the first filtered image, and the second filtered image to obtain a first image and a second image respectively; calculating the optical flow vector result based on the target weight coefficient, the first image, and the second image; determining the motion state of the target object according to the optical flow vector result; determining the motion state of the target object according to the optical flow vector result includes: when the optical flow vector result is greater than When a preset threshold is reached, the target object is determined to be in motion; when the optical flow vector result is not greater than the preset threshold, the target object is determined to be in a stationary state; when the target object is in motion, all the optical flow vector results that reach the threshold are extracted to obtain pixel extraction data of the target object; based on the pixel extraction data of the target object, the motion position of the target object is marked on the original image; marking the motion position of the target object on the original image based on the pixel extraction data of the target object includes: obtaining the position information of the target object in the original image; matching the position information of the original image with the pixel extraction data of the target object to obtain a matching result; based on the matching result, marking the motion position of the target object from the original image. 4. An electronic device, comprising: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 2 by executing the computer instructions. 5. A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 2.