CN114241008A - Long-time region tracking method adaptive to scene and target change - Google Patents

Abstract

The invention discloses a long-term region tracking method adaptive to scene and target changes, which comprises the following steps: the tracking position is quickly and accurately predicted in a mode of combining rough and fine tracking and center position correction; multi-thread research and judgment, wherein the tracking confidence coefficient is corrected by combining the apparent similarity of the target, the multi-frame interval constraint is considered, the tracking result is comprehensively researched and judged, the tracking false alarm is eliminated, and the reliability of the tracking result is improved; and (4) flexible tracking, namely updating a tracking area in a self-adaptive manner when the tracking is successful according to a tracking research and judgment result, and re-capturing the target when the tracking is failed. And when the tracking result is judged to be successful, adapting to the weak change between frames by adopting a frame-by-frame weighting updating mode at short-time small scene intervals, and adapting to the apparent significant change of a large-interval target through an active restarting module at long-time intervals. The method has the advantages of fast tracking response, high precision and reliable confidence coefficient.

Description

Long-time region tracking method adaptive to scene and target change

Technical Field

The invention belongs to the technical field of digital image processing, and mainly solves the problem of long-time tracking of a specific target under the condition of movement of imaging equipment. The algorithm can actively adapt to target scale and apparent change, has small calculation amount, and can be well deployed on various low-performance hardware platforms.

Background

The main purpose of target tracking is to simulate the motion perception function of a physiological visual system, analyze an image sequence captured subsequently by imaging equipment after obtaining the initial state of an interested target to be tracked, calculate the position of the target in each frame of image, obtain the motion trajectory of the target in a specific space-time range, and provide technical support for advanced processing and application such as target identification, target behavior analysis, three-dimensional reconstruction and the like. In recent years, the target tracking technology has been continuously and rapidly developed, and has been widely applied to the fields of national defense and military, such as battlefield reconnaissance and monitoring, border patrol, key target positioning, correction and striking, electronic warfare and damage assessment. The technology is widely applied to civil fields such as security monitoring, traffic operation, aerial photography, disaster monitoring and the like.

A long-time region tracking method adaptive to scene and target changes mainly aims at an unmanned aerial vehicle monitoring system or an overhead observation system, a monitored target is a specific rigid target, the monitoring distance can reach several kilometers or even more than ten kilometers, in order to meet all-weather requirements all day long, infrared monitoring equipment is generally installed, and in order to lock the target, various posture changes of carrying equipment or a holder can occur. In this application mode, the difficulty of target tracking is shown in the following aspects: (1) the apparent change is large from the point of view of the imaging of the target itself. When the shooting distance is long, the infrared imaging method is limited by the limitation of infrared imaging, the target occupies few pixels, insufficient texture information is lacked, and the apparent characteristic is weak. The self apparent imaging change inevitably occurs in the process of continuously tracking the locked target, which comprises the change of the target dimension and the change of the appearance detail caused by the change of the distance between the imaging device and the target, and also comprises the apparent change of the target caused by the change of the posture or the running track. (2) In terms of external environment, similar targets may interfere with each other and shield each other in the operation process when shooting at a long distance, and in addition, due to a nodding view angle, the targets are easily submerged in the surrounding background; finally, the motion of the imaging platform may also cause motion blur and even movement of the target out of view. (3) From the limitation of the tracking algorithm, in the current real-time target tracking algorithm, no matter a generative model or a discriminant model, the basis is that the sizes and the spatial positions of the same object in two frames before and after are not changed greatly in the same video, and the position of the target is determined in the next frame based on a given target template or a trained classifier. The essential difference in tracking over other visual tasks is the ability to accommodate gradual changes in moving objects. Online updates play a crucial role for visual tracking. However, online updating is also a double-edged sword in terms of balanced dynamic information description and accidental noise introduction, accumulates errors for a long time, collects inappropriate samples when the target disappears or over-fits available data, easily degrades the performance of the tracker, resulting in tracking drift.

In the actual tracking process, although many tracking algorithms make obvious progress in establishing an appearance model and robust tracking at present, target tracking is still a very complex problem in the presence of many practical difficulties. Compared with the deep learning algorithm with great heat in recent years, in a real-time processing system, the correlation filtering tracking algorithm is a hot spot of current research by virtue of excellent performances of target tracking accuracy, robustness to target apparent change and speed. The method carries out convolution operation on the image of the next frame and the filtering template by learning the filtering template, predicts the target position according to the output response, converts the convolution operation of the image into dot multiplication in a frequency domain by using FFT (fast Fourier transform) in actual calculation, greatly reduces the calculation complexity, and can achieve the processing speed of hundreds of frames per second when the target is described by using the original pixel value. In order to meet the real-time processing requirement, study and judge the prediction result after completing basic position prediction and retrieve the target again after the target is lost, the invention provides a self-adaptive updating long-term target tracking method under a related filtering framework, and mainly solves the following technical problems.

(1) The predicted position is accurate: aiming at the real problem that a target is weak and small in remote infrared imaging and is easily submerged in a surrounding background, if the target is positioned only on the appearance of the target, the tracking position is easily deviated by the current method for determining the position of the target once based on the maximum response value of confidence coefficient. The method needs to be capable of fully utilizing the target and the surrounding background information, accurately determining the final position of the target and ensuring the accuracy of the tracking result.

(2) The tracking result is credible: the method aims at the problem of target drift caused by various external interferences, and the main reason is that the tracking algorithm cannot reliably judge the tracking result of the tracking algorithm, so that a model updating mechanism has problems. A robust tracker should be able to perform multi-stage verification on the tracking result through various external physical constraints, so as to ensure the reliability of the tracking result.

(3) The updated content is correct: for the realistic problem of the apparent change of the target, the conventional tracker adopts a frame-by-frame updating method to adapt to the dynamic change of the posture or the scale of the target object at the later stage after the target object is determined and tracked in the first frame, but the updating mode of the fixed tracking target object is suitable for the scene with slow apparent change and continuous motion track of the target in the field of view, and the problem of severe scale change cannot be thoroughly solved. When the tracked object exceeds the visual field and only the target is partially left in the visual field, the updating mode causes tracking drift due to edge filling, and meanwhile, the calculation amount is greatly increased. The invention needs the tracker to continuously adapt to the updating of the target template in the whole tracking process, adapts to the severe scale and the apparent change of the target in the tracking process and can ensure the tracking speed and the tracking precision.

Disclosure of Invention

In order to realize wide-area long-time target tracking under continuous visual angles, the invention provides a long-time area tracking method adaptive to scene and target changes.

In order to achieve the purpose, the invention adopts the following technical scheme:

a long-time region tracking method adapting to scene and target changes comprises the following steps:

tracking the target position of an image acquired by an unmanned aerial vehicle monitoring system or a high-altitude observation system based on a three-stage combination mode of rough and fine tracking and center position correction;

calculating a tracking confidence coefficient of the target position, correcting the tracking confidence coefficient of the target position based on the apparent similarity of the target, and comprehensively studying and judging the corrected tracking result by combining multi-frame interval constraint;

and according to the comprehensive judgment result, the tracking model is updated in a self-adaptive manner when the tracking is successful, and the target is recaptured when the tracking is failed.

The long-term area tracking method adaptive to scene and target changes is characterized in that target position tracking of images acquired by an unmanned aerial vehicle monitoring system or a high-altitude observation system based on a mode of three-stage combination of rough and fine tracking and center position correction comprises the following steps:

according to the target position P of the previous frame_t-1Selecting image blocks with the same size as the coarse tracking filter template from the corresponding positions of the frame as coarse tracking search areas to perform coarse tracking search, and obtaining a primary position estimate P of the target_cIf the value of the current tracking response map corresponding to the position is higher than the threshold value thr_ρcDetermining the position of the fine tracking search center point as the point, otherwise, still adopting P_t-1The position of the central point of the fine tracking search of the frame is used;

selecting the image block with the same size as the template of the last frame fine tracking filter at the determined position of the fine tracking search central point as a fine tracking search area to perform fine tracking search to obtain a fine tracking position P_fIf the value of the current tracking response map corresponding to the position is higher than the threshold value thr_ρcReceiving the fine tracking search result, and turning to the next step, otherwise, losing and recapturing the frame if the tracking of the frame fails;

at the fine tracking position P_fTop and surrounding selection and image appearance template T_aImage regions of the same size as the image appearance template T_aPerforming average absolute difference algorithm MAD, and taking the position point with the maximum similarity as the final tracking position P_t。

The long-term region tracking method adapting to scene and target changes is characterized in that calculating a target position tracking confidence coefficient, and correcting the target position tracking confidence coefficient based on the target apparent similarity comprises the following steps:

the peak-to-side lobe ratio PSR is calculated as shown in equation (1) for the tracking response plot and is designated as PSR_curReflecting the intensity of the main peak relative to the side lobe, in the formula F_maxIs the response value of the peak, μ_subAnd σ_subIs the mean and standard deviation of the side lobes;

calculating the peak sidelobe ratio psr of the current frame_curPSR mean PSR of M frame tracking response with latest continuous success_avgThe ratio of (a) to (b) reflects the oscillation degree of the PSR, and determines the target position tracking confidence coefficient rho of the current frame_c；

Calculated to finally track position P_tImage blocks and image appearance templates T in 5-by-5 regions as centers_aComparing the MAD value of the current frame with the MAD average value of the latest continuous successful M frame tracking result to obtain the normalized image apparent similarity rho_a；

Target position tracking confidence ρ_cApparent similarity to image ρ_aAnd obtaining the corrected current tracking confidence coefficient rho by weighted average.

The long-term region tracking method adapting to scene and target changes is characterized in that a multi-frame interval constraint threshold is determined by the interframe displacement change of the latest continuous successful N frames in a historical track and a constant value c, and when a tracking confidence coefficient rho is greater than a threshold thr_aAnd the interval displacement between the current frame and the previous N frames is less than a multi-frame interval constraint threshold thr_mTemporal decision tracking resultCorrect, otherwise the trace fails.

The long-term region tracking method adapting to scene and target changes is characterized in that when tracking is successful, the self-adaptive updating of the tracking region comprises the following steps:

when the continuous successful tracking times are less than N frames, the coarse tracking filter module and the fine tracking filter module are weighted and updated frame by frame to adapt to weak difference between frames;

when the continuous successful tracking times are equal to N frames, the tracking result of the current frame is used for reinitializing the coarse tracking filter module, the fine tracking filter module and the image apparent template T_aTo accommodate significant changes in target appearance;

the method for determining the sizes of the tracking area and the search area when the coarse tracking filter template and the fine tracking filter template are reinitialized simultaneously considers the target size and the calculation speed limit, and specifically comprises the following steps:

according to the initial frame 0 time object distance d₀Focal length f₀Angle of photographing theta₀Object distance d from current frame t_tFocal length f_tAngle of incidence theta_tEstimating an expansion coefficient of a target, and determining the size of the target in the current frame scale, wherein the rough estimation method of the expansion coefficient gamma comprises the following steps:

considering the calculation speed limit, when the imaging short side of the target is equal to or less than 54 pixels, a rectangular frame with the short side extended by 10 pixels is selected as the tracking area with the tracking point as the center. And when the short side of the target imaging is larger than 54 pixels, selecting a 64 x 64 area taking the tracking point as the center as a tracking area, expanding the fine tracking outwards by 1 time as a search area, expanding the coarse tracking outwards by 2 times as a search area, and respectively creating a coarse and fine tracking filter template.

The long-term area tracking method adaptive to scene and target changes is characterized in that loss recapture is required to realize long-term tracking when tracking fails, and mainly comprises the following steps:

sequentially selecting the frame with the highest confidence from the tracking cache according to the size of the tracking area at the current moment, and expanding a double search area to prepare a filter template to search again in the current frame;

tracking the original frame image, the target position and the confidence coefficient which are recently judged to be successful and stored in the cache;

after the search is successful, the coarse tracking filter template, the fine tracking filter template and the image appearance template are initialized again in the current frame;

if the searching fails, the method is continuously repeated for the next frame, the target is declared lost after the target position cannot be obtained again by the continuous N frames, and the target tracking program is terminated.

The long-term region tracking method adaptive to scene and target changes designed by the invention has the following advantages:

(1) the response is quick: and (3) predicting the target position by adopting a related filtering method, wherein the normal tracking processing time of the algorithm on the embedded platform frame by frame is less than 10 ms. The processing time does not exceed 20ms at the maximum when target loss requires reacquisition.

(2) And (3) accurate tracking: compared with the existing related filtering tracking algorithm, the method adopts a coarse and fine tracking combination mode, can utilize the tracking response result and historical tracking information to carry out update opportunity judgment, and has the characteristic of high accuracy of the tracking result compared with the existing CSK, KCF and ECO-HC methods.

(3) And (4) judging: the method combines various clues, comprehensively studies and judges the tracking result by combining the time-space information, provides the confidence judgment of the result on the basis of giving the target prediction position, and is more reliable compared with the output result of the traditional method.

Drawings

The present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.

FIG. 1 is a general framework of the long-term region tracking method of the present invention that adapts to scene and target changes;

FIG. 2 is a flow diagram of a frame-by-frame tracking process;

FIG. 3 is a multi-frame buffer management policy;

fig. 4 is a schematic diagram of tracking target loss recapture.

Detailed Description

The invention proposes a long-term region tracking method adaptive to scene and target changes, and the following description clearly and completely describes the specific embodiments of the invention with reference to the accompanying drawings.

The embodiment discloses a long-term region tracking method adaptive to scene and target changes, and the overall framework is as shown in fig. 1:

after the initial frame obtains the target to be tracked, a coarse and fine tracking filter template and an image apparent template T are carried out_aPreparing, when a new subsequent frame of image arrives, selecting a candidate search area, predicting a target position, then combining various clues to calculate confidence coefficient of a tracking result and comprehensively studying and judging the target tracking confidence coefficient, and eliminating a tracking false alarm; and finally, updating the tracking model according to the research and judgment result.

In this embodiment, a specific processing flow per frame is shown in fig. 2, and includes:

step 1: target location prediction

The device mainly comprises two parts: and carrying out rough and fine tracking search and center position correction.

(a) Firstly, coarse tracking search is carried out to obtain a primary position P of a target_cIf the maximum response value for that location is greater than the acceptable threshold thr_ρcThen fine tracking is performed at this position, otherwise the position P is directly at the previous frame_t-1And performing fine tracking. The rough and fine tracking adopts a tracking principle based on relevant filtering, specifically, according to the position of a target of a previous frame, an image block with the same size as a tracking filter template (including the rough tracking filter template and the fine tracking filter template) is selected as a search area at the corresponding position of the current frame, characteristics are extracted, a cosine window is added to eliminate boundary influence, and the search area characteristics are assumed to be z, and the search area characteristics and a target characteristic model x of the previous frame are operated to obtain a kernel matrix k as shown in formula 1; then, as shown in formula 2, the filter parameter a calculated in the previous frame and the kernel matrix k are used to perform dot multiplication operation in the frequency domain, and the calculation result is subjected to inverse Fourier transform F^-1In the time domain, the phase is obtainedClosing the response graph y after the filtering operation; and finally, the position of the maximum probability of the target in the current frame can be calculated by searching the maximum value Fmax and the corresponding coordinates (px, py) in the response graph y.

y＝F^-1(F(a)·F(k)) (2)

In the process, the coarse tracking and the fine tracking all adopt related filtering methods, the difference between the two methods is that after a tracking area is selected, the size of the searching area is set to be different, the specific size sets an initial value according to the actual situation, and meanwhile, in order to ensure the speed, the coarse tracking carries out downsampling operation based on nearest neighbor interpolation on the searching area.

(b) When the position P is accurately tracked_fThe corresponding maximum response value is greater than the acceptable threshold thr_ρcThen, the center position is corrected at this position by the apparent similarity of the images. The specific method is that the current frame position point P is_fAnd four candidate points P1, P2, P3 and P4 are selected up, down, left and right around. Respectively selecting and imaging the apparent template T by taking the 5 points as the center_aImage regions of the same size, in this embodiment, 5 × 5 regions are selected to form new target apparent images S, S1, S2, S3 and S4, and then the new target apparent images S, S1, S2, S3 and S4 are respectively associated with the image apparent templates T_aPerforming image matching algorithm MAD (mean absolute difference) based on gray scale, and taking the position point with the maximum similarity as the final tracking point to replace P_fAs final tracking result P of the tracker_t. The MAD calculation is shown in equation (3):

wherein: m is equal to N is equal to 5, and d represents the average absolute difference between the pixel values of the image block S and the image apparent template T.

Step 2: confidence correction

The device comprises two parts: and the tracking filter responds to the space-time analysis and the apparent similarity calculation of the tracking area image.

The tracker self-response space-time analysis firstly calculates Peak to side lobe Ratio (PSR) of a tracking response graph solved by a correlation filtering tracking algorithm, and records the PSR as PSR_cur. PSR can be used to represent the correlation peak intensity. The correlation output g is divided into two parts: the peak value is the maximum value, and the side lobe is selected to be in an area 11 x 11 with the peak value as the center. The ratio of the main peak to the side lobe is mainly reflected, and when the mean value of the side lobe is large or the distribution is uneven, the PSR value can be reduced. The specific calculation is shown in formula (4):

F_maxis the response value of the peak, μ_subAnd σ_subIs the mean and standard deviation of the side lobes.

The change of the tracking response graph does not change violently during normal tracking, and only when the target is shielded or lost, the change can change violently, so that the historical tracking information can also provide reference for calculation of the current tracking confidence. The method utilizes the mean value of the tracking result of the historical effective frame to normalize and represent the peak sidelobe ratio as shown in a formula (5), and represents the tracking confidence coefficient rho of the target position according to the normalized expression_cThis method uses the current PSR value PSR_curAnd the PSR mean PSR of the historical effective frame_avgThe ratio of the two-dimensional space-time-domain signal to the confidence coefficient index of the final tracking filter has the advantages that the method can be self-adaptive to each section of test scene, and effectively overcomes the defects of poor generalization of a fixed threshold value and the like.

The tracking confidence degree calculation method based on the image apparent similarity comprises the following steps: the previously calculated MAD is normalized by calculating the MAD average value of the latest continuous successful M-frame tracking results in the history, and in this embodiment, the MAD average value d of the tracking results of the first three frames (M is 3) is specifically taken_avgAs a reference similarity value apparent from the current tracking image, and then currentTracking point MAD value d calculated by frame_curSimilarity to reference value d_avgThe ratio is shown in equation (6).

Target position tracking confidence ρ_cApparent similarity to image ρ_aAnd (4) obtaining the corrected current tracking confidence coefficient rho by weighted average, wherein the calculation is shown as a formula (7).

ρ＝0.5*ρ_c+0.5*ρ_a (7)

And step 3: tracking result study and judgment

And (4) studying and judging the tracking result by combining the tracking confidence coefficient and the target tracking historical track information.

The multi-frame interval constraint specifically refers to motion displacement constraint of a tracking target among multiple frames, and a prediction displacement deviation threshold value of a current frame and a historical frame is calculated according to interframe displacement change of a historical continuous tracking successful frame N. In order to implement the strategy, it is necessary to store the tracking data of the most recent continuous multiframe (such as N frames, where N is 5 in the present invention). Three cache queues are used for storing data, and a specific storage and update strategy is shown in fig. 3 and described as follows:

caching 1: the system is used for calculating the deviation distance of the currently processed N frames, storing the tracking track information of the latest N frames, updating the strategy that the current frame is processed and stored in a cache, and deleting the processing result of the earliest frame;

and (4) caching 2: the method is used for storing N frame data of a previous stage of a current stage and updating a cache 3, wherein the updating strategy is to initialize a cache 2 after the cache 1 is full for the first time, and then press the oldest value in the cache 1 into the cache 2 every time the cache 1 is updated, and simultaneously remove the oldest value in the cache 2.

And (3) caching: the method is used for calculating the latest historical prediction and storing the historical continuous latest successful N frames, the updating strategy is to copy the continuous successful N frames from the 'cache 2', and the average frame interval displacement is calculated once and can be used for the prediction of the next frames for a plurality of times; the average frame interval displacement is calculated and then emptied.

The final tracking result judgment strategy based on the tracking confidence coefficient and the multi-frame interval displacement constraint is as follows: only if the tracking confidence p is greater than an acceptable threshold thr_a(value 0.15) and the interval displacement between the current frame and the previous N frames (if the current frame is t frame, namely the displacement between t frame and t-N frame) is less than the acceptable multi-frame interval constraint threshold thr_mAnd judging that the tracking result is correct, otherwise, failing to track.

And 4, step 4: tracking model management

And when the tracking result judges that the tracking of the current frame is successful, managing the tracking model. The invention adopts a mode of combining frame-by-frame weighted updating and stage restarting preparation to update the filter template to adapt to the target scale and the apparent change. The specific method comprises the following steps:

firstly, counting the continuous tracking success times, and judging whether to restart actively according to the continuous tracking success times.

If the number of times of continuous successful tracking is less than N, in this embodiment, when N is 5, in order to adapt to a weak difference between short-time small scene frames, a policy of weighting and updating the template frame by frame is adopted. However, the result of each frame is used for updating, or frame-by-frame updating is risky, especially when the target is occluded or the tracker is not good enough, the model is updated, so that the tracker can not identify the target more and more, the invention only judges that the tracking is successful in step 3 and the tracking confidence coefficient p is greater than the threshold thr_l(thr_a<thr_l<1，thr_lThe value of 0.3) is taken, the tracking model is updated, the target model is prevented from being polluted, the model drifting and updating times are reduced, and meanwhile, the speed is increased. The calculation method for updating the tracking model frame by frame is shown in equations (8) and (9).

F(x)_t＝(1-r)F(x)_t-1+rF(x)_t (8)

F(a)_t＝(1-r)F(a)_t-1+rF(a)_t (9)

Wherein F (-) represents Fourier transform operation, x represents extracted object and background features, a represents filter parameters, r represents update rate, and the larger the value of rLarge means that the current frame has a larger weight, but a larger value of r easily causes the template to be contaminated by the current frame. Therefore, the update rate needs to be adaptively adjusted according to the tracking confidence level. The current confidence ρ is low (thr)_l<ρ<thr_h,thr_h0.7), the update rate is 0.035, and the confidence p is high (thr)_hRho is not less than 1) and the update rate is 0.085.

If the number of continuous successful tracking times is equal to N, in this embodiment, when N is equal to 5, in order to adapt to a large change in appearance of a long-term large scene, the present invention retrains the filter at the current frame, and initializes the tracking parameters, the tracking filter template, and the target appearance template. When preparing the tracking filter template, the determination of the tracking area and the search area considers the target size and the calculation speed limit at the same time. According to the initial frame 0 time object distance d₀Focal length f₀Angle of photographing theta₀Object distance d from current frame t_tFocal length f_tAngle of incidence theta_tEstimating an expansion coefficient of a target, and determining the size of the target in the current frame scale, wherein the rough estimation method of the expansion coefficient gamma comprises the following steps:

then, considering the calculation speed limit, when the target image is small (the short side is 54 pixels or less), a rectangular frame with the short side extended by 10 pixels is selected as the tracking area with the tracking point as the center, and when the target image is large (the short side is 54 pixels or more), a region with a fixed size of 64 × 64 with the tracking point as the center is selected as the tracking area. And the coarse tracking and the fine tracking are respectively expanded outwards by 1 time and 2 times to be used as a search area to establish a coarse tracking filter template and a fine tracking filter template.

The target appearance template selects an image area with a tracking point as a center 5 x 5.

And 5: loss-weight compensation

When the tracking result is that the tracking of the current frame fails, lost recapture work is needed, whether the tracking is stopped is judged firstly, namely whether continuous multi-frame tracking fails is judged, and the tracking process is ended if continuous N is 5 frames which fail. Otherwise, restarting tracking in a failure state is carried out by utilizing the historical successful tracking information to recapture the lost target. The specific method comprises the following steps:

sequentially selecting the frame with the highest confidence from the tracking cache according to the size of the tracking area at the current moment, and expanding a double search area to prepare a tracking filter template to search again in the current frame;

calculating confidence coefficient rho and multi-frame interval displacement of the search result, studying and judging the tracking result according to the strategy in the step 3, and if the search is successful, re-initializing the coarse and fine tracking filter template and the image appearance template in the current frame according to the tracking area and search area setting method in the step 4;

and if the search fails, the next frame processing is carried out.

The long-term target tracking method for template adaptive update provided by the present invention is described in detail above, but it is obvious that the specific implementation form of the present invention is not limited thereto. It will be apparent to those skilled in the art that various obvious changes may be made therein without departing from the scope of the invention as defined in the appended claims.

Claims (6)

1. A long-time region tracking method adapting to scene and target changes is characterized by comprising the following steps:

tracking the target position of an image acquired by an unmanned aerial vehicle monitoring system or a high-altitude observation system based on a three-stage combination mode of rough and fine tracking and center position correction;

calculating a tracking confidence coefficient of the target position, correcting the tracking confidence coefficient of the target position based on the apparent similarity of the target, and comprehensively studying and judging the corrected tracking result by combining multi-frame interval constraint;

and according to the comprehensive judgment result, the tracking model is updated in a self-adaptive manner when the tracking is successful, and the target is recaptured when the tracking is failed.

2. A long-term region tracking method adapted to scene and object changes as claimed in claim 1, characterized by:

the tracking of the target position of the image acquired by the unmanned aerial vehicle monitoring system or the high-altitude observation system based on the mode of three-stage combination of rough and fine tracking and center position correction comprises the following steps:

according to the target position P of the previous frame_t-1Selecting image blocks with the same size as the coarse tracking filter template from the corresponding positions of the frame as coarse tracking search areas to perform coarse tracking search, and obtaining a primary position estimate P of the target_cIf the value of the current tracking response map corresponding to the position is higher than the threshold value thr_ρcDetermining the position of the fine tracking search center point as the point, otherwise, still adopting P_t-1The position of the central point of the fine tracking search of the frame is used;

selecting the image block with the same size as the template of the last frame fine tracking filter at the determined position of the fine tracking search central point as a fine tracking search area to perform fine tracking search, and obtaining a fine tracking position P_fIf the value of the current tracking response map corresponding to the position is higher than the threshold value thr_ρcReceiving the fine tracking search result, and turning to the next step, otherwise, losing and recapturing the frame if the tracking of the frame fails;

at the fine tracking position P_fTop and surrounding selection and image appearance template T_aImage regions of the same size as the image appearance template T_aPerforming average absolute difference algorithm MAD, and taking the position point with the maximum similarity as the final tracking position P_t。

3. A long-term region tracking method adapted to scene and object changes as claimed in claim 1, characterized by:

calculating a target position tracking confidence coefficient, and correcting the target position tracking confidence coefficient based on the target apparent similarity comprises the following steps:

the peak-to-side lobe ratio PSR is calculated as shown in equation (1) for the tracking response plot and is designated as PSR_curReflecting the intensity of the main peak relative to the side lobe, in the formula F_maxIs the response value of the peak, μ_subAnd σ_subIs the mean and standard deviation of the side lobes;

calculating the peak sidelobe ratio psr of the current frame_curPSR mean PSR of M frame tracking response with latest continuous success_avgThe ratio of (a) to (b) reflects the oscillation degree of the PSR, and determines the target position tracking confidence coefficient rho of the current frame_c；

Calculated to finally track position P_tImage blocks and image appearance templates T in 5-by-5 regions as centers_aComparing the MAD value of the current frame with the MAD average value of the latest continuous successful M frame tracking result to obtain the normalized image apparent similarity rho_a；

Target position tracking confidence ρ_cApparent similarity to image ρ_aAnd obtaining the corrected current tracking confidence coefficient rho by weighted average.

4. A long-term region tracking method adapted to scene and object changes as claimed in claim 1, characterized by:

the multi-frame interval constraint threshold is determined by the interframe displacement change of the latest continuous successful N frames in the historical track and a constant value c, and when the tracking confidence coefficient rho is greater than the threshold thr_aAnd the interval displacement between the current frame and the previous N frames is less than a multi-frame interval constraint threshold thr_mAnd judging that the tracking result is correct, otherwise, failing to track.

5. A long-term region tracking method adapted to scene and object changes as claimed in claim 1, characterized by:

the self-adaptive updating tracking model when the tracking is successful comprises the following steps:

when the continuous successful tracking times are less than N frames, the coarse tracking filter module and the fine tracking filter module are weighted and updated frame by frame to adapt to weak difference between frames;

when the continuous successful tracking times are equal to N frames, the tracking result of the current frame is used for reinitializing the coarse tracking filter module, the fine tracking filter module and the image apparent template T_aTo accommodate significant changes in target appearance;

the method for determining the sizes of the tracking area and the search area when the coarse tracking filter template and the fine tracking filter template are reinitialized simultaneously considers the target size and the calculation speed limit, and specifically comprises the following steps:

according to the initial frame 0 time object distance d₀Focal length f₀Angle of photographing theta₀Object distance d from current frame t_tFocal length f_tAngle of incidence theta_tEstimating an expansion coefficient of a target, and determining the size of the target in the current frame scale, wherein the rough estimation method of the expansion coefficient gamma comprises the following steps:

considering the calculation speed limit, when the short side of the target imaging is smaller than or equal to 54 pixels, a rectangular frame with the length of the short side expanded by 10 pixels is selected as a tracking area by taking the tracking point as the center, when the short side of the target imaging is larger than 54 pixels, a 64 × 64 area with the tracking point as the center is selected as the tracking area, the fine tracking is expanded outwards by 1 time to be used as a search area, the coarse tracking is expanded outwards by 2 times to be used as the search area, and a fine tracking filter template and a coarse tracking filter template are respectively created.

6. A long-term region tracking method that accommodates scene and object changes as recited in claim 5, wherein:

when the tracking fails, in order to realize the long-term tracking, lost recapture is needed, which mainly comprises:

sequentially selecting the frame with the highest confidence from the tracking cache according to the size of the tracking area at the current moment, expanding the frame by two times to be used as a search area for preparing a filter template, and searching again in the current frame;

tracking the original frame image, the target position and the confidence coefficient which are recently judged to be successful and stored in the cache;

after the search is successful, the coarse tracking filter template, the fine tracking filter template and the image appearance template are initialized again in the current frame;

if the searching fails, the method is continuously repeated for the next frame, the target is declared lost after the target position cannot be obtained again by the continuous N frames, and the target tracking program is terminated.

Images