CN110675432A - Multi-dimensional feature fusion-based video multi-target tracking method - Google Patents
Multi-dimensional feature fusion-based video multi-target tracking method Download PDFInfo
- Publication number
- CN110675432A CN110675432A CN201910964726.XA CN201910964726A CN110675432A CN 110675432 A CN110675432 A CN 110675432A CN 201910964726 A CN201910964726 A CN 201910964726A CN 110675432 A CN110675432 A CN 110675432A
- Authority
- CN
- China
- Prior art keywords
- track
- detection target
- target
- historical
- calculating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 230000004927 fusion Effects 0.000 title claims abstract description 18
- 238000001514 detection method Methods 0.000 claims abstract description 127
- 238000004364 calculation method Methods 0.000 claims description 19
- 238000013527 convolutional neural network Methods 0.000 claims description 3
- 239000012634 fragment Substances 0.000 abstract description 4
- 230000009191 jumping Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010205 computational analysis Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a multi-target video tracking method based on multi-dimensional feature fusion, and relates to the technical field of videos and images. The method comprises the following steps: recording time information, position information and content characteristics of the detection target; calculating the content feature similarity of the detection target and all track containers, and if the maximum similarity value is greater than a first threshold value, determining that the detection target and all track containers are effectively matched; for unmatched track containers, further calculating the change proportion of the historical track speed to the temporary track speed, the movement included angle between the detection target and the motion direction of the historical track and the width change proportion, and if the three are within a given threshold interval, determining that the three are effectively matched; and storing the effectively matched detection target into a corresponding track container. The invention has real and reliable matching result and effectively reduces the serial number phenomenon caused by track fragments, shielding and other conditions.
Description
Technical Field
The invention relates to the technical field of video and image processing, in particular to a multi-target video tracking method based on multi-dimensional feature fusion.
Background
The multi-target tracking technology is an important link of a video analysis technology, and the technology obtains a motion direction and a predicted position through target historical track analysis according to the appearance time sequence of input Detection target Detection images and combines the content characteristic similarity between targets for matching connection. How to effectively match and concatenate the target in each video frame with the track target in the historical frame is the key of the technology. The technical field of multi-target tracking at present generally adopts target prediction (a selection scheme of Kalman trajectory prediction in a relatively mature way) position, then matches the actual position of a detected target with the predicted position, and then matches the trajectory and the target which are not matched with each other by feature similarity; or preferentially matching the track and the target by using the feature similarity to ensure that the long-distance moving target can be matched, and then matching the actual position of the detected target with the predicted position.
Aiming at the problems that in a daily monitoring video, targets are overlapped and lost due to mutual shielding of the targets, and the phenomenon of rapid movement of the targets is common; therefore, the problems that simple position matching and feature similarity matching cannot be well solved are caused, the final track fragments are excessive, and the phenomenon of serial numbers (IDSwitch) in the track is serious.
Disclosure of Invention
The invention aims to provide a video multi-target tracking method based on multi-dimensional feature fusion, which is real and reliable in matching result and effectively reduces the serial number phenomenon caused by track fragments, shielding and the like.
In order to achieve the purpose, the invention provides the following technical scheme:
a video multi-target tracking method based on multi-dimensional feature fusion is characterized by comprising the following steps:
s1, recording the time information, the position information and the content characteristics of the detection target;
s2, calculating the content feature similarity of the detection target and all track containers, and if the maximum similarity value is greater than a first threshold value, determining that the detection target and all track containers are effectively matched;
s3, for unmatched track containers, further calculating the change proportion of the historical speed and the temporary speed of the track, the movement included angle between the detection target and the motion direction of the historical track and the width change proportion of the track, and if the three are within a given threshold interval, determining that the three are effectively matched;
and S4, storing the detection target which is effectively matched into the corresponding track container.
Further, the specific content of S2 is as follows:
extracting content characteristics of the detection target, and performing content characteristic similarity calculation with the content characteristics stored in all track containers to obtain a corresponding similarity value sequence; finding out the maximum similarity value from the similarity value sequence to be compared with a first threshold value, calculating the distance between the detection target and the last frame in the track container, and comparing the distance with a distance threshold value; if the maximum similarity value is greater than the first threshold and the distance is within the distance threshold, a valid match is identified.
Further, the specific content of S2 is as follows:
if the track container has historical record information of the detection target, predicting the position information of the detection target which should appear in the current frame in real time to obtain a predicted rectangular area;
detecting a rectangular area of an actual detection target, calculating an intersection ratio of a prediction rectangular area and the rectangular area, and taking the difference between 1 and the intersection ratio as the distance between the actual detection target and the prediction detection target; and calculating the similarity of the content characteristics of the track container corresponding to the minimum distance and the detection target, and comparing the similarity with a first threshold value, wherein if the maximum similarity value is greater than the first threshold value, the track container is determined to be effectively matched.
Further, the specific content of S3 is as follows:
for unmatched track containers, the track container PTrack with the maximum similarity to the content features of the detection target is selectedContSimultaneously recording the corresponding similarity values SimiCont(ii) a Calculating the distances between the detection target and all track containers, and taking the track container Ptrack corresponding to the minimum distanceEucCalculating the track container PtrackEucSimilarity value Simi with content feature of detection targetEuc；
S31 if PTrackContAnd PtrackEucSame, and SimiContIf the three are within the corresponding given threshold interval, the effective matching is determined;
s32 if PTrackContAnd PtrackEucSame as when SimiContAnd SimiEucWhen the larger value is larger than or equal to the given parameter threshold, calculating the width change proportion, and if the width change proportion is within the given threshold interval, determining that the width change proportion is effectively matched; when SimiContAnd SimiEucWhen the larger value of the two values is smaller than a given parameter threshold value, taking the track container corresponding to the larger value of the two values, calculating the change proportion of the historical track speed and the temporary speed, and the movement included angle and the width change proportion of the detection target and the historical track movement direction, and if the three values are in the corresponding given threshold value interval, determining that the three values are effectively matched;
s33, if the detection target still does not find a track container with effective matching, taking SimiContAnd SimiEucAnd (4) calculating the change proportion of the historical track speed and the temporary speed, and the movement included angle and the width change proportion of the detection target and the historical track movement direction, and if the three values are in the corresponding given threshold interval, determining that the three values are effectively matched.
Further, the calculation method of the historical track speed and the temporary track speed is as follows:
Vhist＝(Rt-Rt-Δt) [ Delta ] t, wherein RtIs position information of the last to last frame of the history information, Rt-ΔtIs the position information of the penultimate frame of the history information, and Δ t is the time interval of two frames of history information;
Vcur＝(Rcur-Rt)/Δtcurwherein，RtIs position information of the last to last frame of the history information, RcurIs position information of the detection target, Δ tcurIs the time interval between the detection target time and the last but one frame of the history information.
Further, the method for calculating the motion included angle between the detection target and the motion direction of the historical track is as follows:
Further, in the S31, if PTrackContIf the history information of (2) has only one frame of information, the width change ratio is directly calculated, and if the width change ratio is within a given threshold interval, the matching is determined to be valid.
Further, the distance is a euclidean distance.
Further, the content features are extracted by adopting a deep convolutional neural network.
Further, the position information includes a center position, an aspect ratio, and a height of the detection target.
Compared with the prior art, the invention has the beneficial effects that: according to the invention, from the perspective of multi-feature fusion, feature matching is preferred, and in the first two links, content feature similarity constraint is added to the matching result of each link, so that the matching result is output truly and reliably; and selecting the optimal matching logic for the Euclidean distance space and the content characteristic space for the last remaining detection targets which are not successfully matched in a discrete mode, and then ensuring the truth and reliability of matching by using the multidimensional constraints of time, position, speed and angle. The invention has high operation efficiency and far-exceeding real-time requirements, effectively reduces track fragments and the serial number phenomenon caused by shielding and the like, and obtains good application effect.
Drawings
Fig. 1 is an overall schematic block diagram of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention are clearly and completely described below, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, the present invention provides a video multi-target tracking method based on multi-dimensional feature fusion, which is characterized by comprising the following steps:
and S1, recording the time information, the position information and the content characteristics of the detection target.
Specifically, for a newly added Detection target Detection, a new track container PathTrack is opened, the life limit MaxAge of the track is initialized, and the preferential MaxAge is 56; initializing a state matrix of a Kalman filter of the new track container PathTrack to be an initialization state; and recording time information, position information, corresponding speed information in image coordinates and image content characteristics of the Detection target Detection, and storing the time information, the position information, the corresponding speed information and the image content characteristics of the Detection target Detection in the MsgBox unit stack. The position information includes a center position, an aspect ratio, and a height of the detection target. Preferably, the image content features adopt abstract features extracted by a deep convolutional neural network.
And S2, calculating the content feature similarity of the detection target and all track containers, and if the maximum similarity value is greater than a first threshold value, determining that the detection target and all track containers are effectively matched. And the content feature matching is adopted, so that the matched result is output really and reliably.
There are three specific embodiments of step S2, which are as follows:
in the first embodiment, the specific content of S2 is as follows:
and extracting image content characteristics of the Detection target Detection, and performing content characteristic similarity calculation with the image content characteristics of the MsgBox information stored in all track containers PathTrack. Wherein, it is noted that two N-dimensional image content features X, Y are described as: x (X)1,x2,...,xN)，Y(y1,y2,...,yN) The corresponding image content similarity calculation formula between X and Y is as follows:
and according to a similarity calculation formula of the content features, obtaining a similarity result sequence of the image content features corresponding to the Detection and MsgBox information stored in the PathTrack. Finding out the maximum similarity value and the corresponding relation between the Detection and PathTrack from the similarity value sequence, wherein the central position of the Detection is (x)det,ydet) The position information of the last MsgBox center in PathTrack is (x)last,ylast) The corresponding distance calculation formula:
an invalid match is considered if the ratio of the corresponding distance D to the width dimension information of Detection is greater than the corresponding distance threshold condition distThr. Otherwise, comparing the maximum similarity value with a first threshold value, if the maximum similarity value is greater than the first threshold value SimiThr, determining that the maximum similarity value is a valid match, and jumping to S4; otherwise, an invalid match is considered. Preferably, the distance threshold value distThr is 1.1 and the first threshold value simitr is 0.5.
In the second embodiment, the specific content of S2 is as follows:
if the track container PathTrack has history information of the Detection target Detection, a Kalman filter is used to predict the history information in real time for the position information (including the central position, the aspect ratio and the height of the Detection target Detection and the corresponding speed information predictDetection in the image coordinate) of the Detection target Detection to appear in the current frame, so as to obtain a predicted rectangular area PreRect. Wherein, the Kalman filter adopts a uniform velocity model and a linear observation model.
Detecting a rectangular area Rect where an actual Detection target Detection is located, and respectively carrying out position distance calculation with a prediction rectangular area PreRect of all track containers where the Detection target Detection exists to obtain corresponding distance relation sequences. Here, the method for calculating the position distance according to the present invention includes: calculating the intersection ratio of the prediction rectangular region PreRect and the rectangular region Rect, and then taking the difference between 1 and the intersection ratio as the distance dist between the actual detection target and the prediction detection target; the specific formula is as follows:
from the distance relation sequence, calculating the content feature similarity of the track container PathTrack corresponding to the minimum distance and the Detection target Detection, comparing the content feature similarity with the first threshold value SimiThr in the first embodiment, if the maximum similarity is greater than the first threshold value SimiThr, determining that the content feature similarity is effective matching, and jumping to S4; otherwise, an invalid match is considered.
And in the second embodiment, the Kalman prediction is utilized to analyze and predict the position of the historical track data, the predicted position is used for carrying out position matching with the target, and the matched pair is subjected to image content similarity constraint, so that the matching reliability is further improved.
In the third embodiment, the specific content of S2 may also be a superposition of the first embodiment and the second embodiment, specifically as follows:
if the track container PathTrack has history information of the Detection target Detection, a Kalman filter is used to predict the history information in real time for the position information (including the central position, the aspect ratio and the height of the Detection target Detection and the corresponding speed information predictDetection in the image coordinate) of the Detection target Detection to appear in the current frame, so as to obtain a predicted rectangular area PreRect. Wherein, the Kalman filter adopts a uniform velocity model and a linear observation model.
Extracting image content characteristics of Detection target Detection and all railsAnd performing content feature similarity calculation on the image content features of the MsgBox information stored in the track container PathTrack. Wherein, it is noted that two N-dimensional image content features X, Y are described as: x (X)1,x2,...,xN)，Y(y1,y2,...,yN) The corresponding image content similarity calculation formula between X and Y is as follows:
and according to a similarity calculation formula of the content features, obtaining a similarity result sequence of the image content features corresponding to the Detection and MsgBox information stored in the PathTrack. Finding out the maximum similarity value and the corresponding relation between the Detection and PathTrack from the similarity value sequence, wherein the central position of the Detection is (x)det,ydet) The position information of the last MsgBox center in PathTrack is (x)last,ylast) The corresponding distance calculation formula:an invalid match is considered if the ratio of the corresponding distance D to the width dimension information of Detection is greater than the corresponding distance threshold condition distThr. Otherwise, comparing the maximum similarity value with a first threshold value, if the maximum similarity value is greater than the first threshold value SimiThr, determining that the maximum similarity value is a valid match, and jumping to S4; otherwise, an invalid match is considered. Preferably, the distance threshold value distThr is 1.1 and the first threshold value simitr is 0.5.
If the Detection target Detection is invalid and matched, further detecting the rectangular area Rect where the actual Detection target Detection is located, and respectively calculating the position distance with the predicted rectangular area Rect of all the track containers where the Detection target Detection exists to obtain the corresponding distance relation sequence. Here, the method for calculating the position distance according to the present invention includes: calculating the intersection ratio of the prediction rectangular region PreRect and the rectangular region Rect, and then taking the difference between 1 and the intersection ratio as the distance dist between the actual detection target and the prediction detection target; the specific formula is as follows:
calculating the content feature similarity of the track container PathTrack corresponding to the minimum distance from the distance relation sequence and the Detection target Detection, comparing the content feature similarity with the first threshold value SimiThr, if the maximum similarity is greater than the first threshold value SimiThr, determining that the content feature similarity is effective matching, and jumping to S4; otherwise, an invalid match is considered.
Compared with the first embodiment and the second embodiment, the third embodiment adopts two links of feature matching, and content feature similarity constraint is added to the matching result of each link, so that the matching result is output truly and reliably.
And S3, for the unmatched track container PathTrack, further calculating the change proportion of the historical speed and the temporary speed of the track, the movement included angle between the detection target and the motion direction of the historical track and the width change proportion, and if the three are within a given threshold interval, determining that the three are effectively matched.
The specific contents are as follows:
for the unmatched track container PathTrack, on one hand, the track container PTTrack with the maximum content feature similarity with the Detection target Detection is takenContSimultaneously recording the corresponding similarity values SimiCont(ii) a It is worth mentioning that if the similarity value SimiContIf the threshold value is less than 0.3, the PTrack is judgedContAbsent, jump to S1.
On the other hand, the Euclidean distances between the detection target and all the track containers are calculated, and the track container Ptrack corresponding to the minimum distance is selectedEucCalculating the track container PtrackEucSimilarity value Simi with content feature of detection targetEuc(ii) a If the ratio of the minimum distance to the width of Detection is greater than a given threshold value MoveDist, preferably 3.3, then Ptrack is determinedEucAbsent, jump to S1. It is worth mentioning that the euclidean distance dist herein is calculated as:
wherein dist is the Euclidean distance; (x)new,ynew) The coordinate of the central point of the rectangular region Rect; (x)hist,yhist) Coordinates of the center point of the rectangular area preret are predicted.
S31 if PTrackContAnd PtrackEucSame, and SimiContGreater than or equal to a similarity threshold ValidSimiThr for effective matching, preferably, ValidSimiThr ═ 0.65. Then, if the PTrackContWhen the history information MsgBox exists for two frames or more, calculating the history speed V of the trackhistWith a temporary speed VcurIf the three are within the corresponding given threshold interval, the change proportion of the motion Angle and the width change proportion of the motion direction of the detection target and the historical track is determined as effective matching, and the step is carried out to S4. In particular, computational analysis VcurAnd VhistVariation amplitude, variation multiple less than a given parameter threshold [1/VThr, VThr]Preferably, VThr ═ 2; meanwhile, the Angle is smaller than a given parameter threshold value anglerthr, and the preferred Angle thr is 45 °. If not, an invalid match is considered; otherwise, calculating the width ratio change, namely the ratio of the width size information in the specific first-last MsgBox to the width of the rectangular region Rect, and if the ratio is in a given change threshold interval [1/changeRate, changeRate]Then a valid match is considered, the preferred ChangeRate is 1.5; then the match is deemed a valid match and a jump is made to S4.
If the PTrackContIf the history information of (1) has only one frame, the width change ratio is directly calculated, and if the width change ratio is in a given threshold interval [1/ChangeRate, ChangeRate ]]If yes, the process goes to S4. The following is the historical speed VhistTemporary velocity VcurAnd a specific calculation method of the Angle of motion.
The calculation method of the track historical speed and the temporary speed is as follows:
Vhist＝(Rt-Rt-Δt) [ Delta ] t, wherein RtIs the position information of the last frame of the MsgBox information, Rt-ΔtIs MsgBox penultimatePosition information of two frames, note RtCenter (x) of (c)t,yt)，Rt-ΔtCenter (x) of (c)t-Δt,yt-Δt) Then (R)t-Rt-Δt) The calculation formula of (2):
at is the time interval of two frames of history information.
Vcur＝(Rcur-Rt)/ΔtcurWherein R istIs the position information of the last frame of the MsgBox information, RcurIs position information of Detection, note RcurCenter (x) of (c)cur,ycur) Then (R)cur-Rt) The calculation formula of (2):Δtcuris the time interval between the detection target time and the last but one frame of the history information.
The calculation method of the motion included angle between the detection target and the motion direction of the historical track is as follows:
S32 if PTrackContAnd PtrackEucSame as when SimiContAnd SimiEucWhen the larger value is equal to or greater than the given parameter threshold value directmatcthr, the width change ratio is preferably calculated such that directmatcthr is 0.85. If the width variation ratio is within a given threshold range [1/ChangeRate, ChangeRate]Interior, then it is determined to haveMatching, jumping to S4; when SimiContAnd SimiEucWhen the larger value of the two values is smaller than a given parameter threshold value, the track container corresponding to the larger value of the two values is taken to be recorded as a PTrackBetter. Like S31, when PTrackBetterExtracting V when the medium history track MsgBox exists for more than two framescurAnd VhistAngle information. Calculating the historical velocity V of the trackhistWith a temporary speed VcurIf the three are within the corresponding given threshold interval, the change proportion of the motion Angle and the width change proportion of the motion direction of the detection target and the historical track is determined as effective matching, and the step is carried out to S4.
S33, if the Detection target Detection still does not find a track container which is effectively matched, taking SimiContAnd SimiEucThe track container corresponding to the smaller value (i.e. the track container remaining for matching in S32) is marked as PTrackLast. Like S31, when PTrackLastExtracting V when the medium history track MsgBox exists for more than two framescurAnd VhistAngle information. Calculating the historical velocity V of the trackhistWith a temporary speed VcurIf the change proportion, the motion included Angle between the detection target and the motion direction of the historical track and the width change proportion are all in the corresponding given threshold value interval, effective matching is determined, and S4 is skipped. Otherwise, the process jumps to S1 to open a new track container.
And S4, storing the detection target which is effectively matched into the corresponding track container. Specifically, if the track container is matched with a new Detection target Detection, the MsgBox correspondingly recording the Detection information is stored in a stack of the track container PathTrack and is stored in sequence; meanwhile, the existence life counter of the track is 0, and simultaneously, the state of the Kalman state matrix is updated; if no new Detection target Detection is matched, the life counter is increased by 1 until the track life counter reaches MaxAge, and the track is ended.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.
Claims (10)
1. A video multi-target tracking method based on multi-dimensional feature fusion is characterized by comprising the following steps:
s1, recording the time information, the position information and the content characteristics of the detection target;
s2, calculating the content feature similarity of the detection target and all track containers, and if the maximum similarity value is greater than a first threshold value, determining that the detection target and all track containers are effectively matched;
s3, for unmatched track containers, further calculating the change proportion of the historical speed and the temporary speed of the track, the movement included angle between the detection target and the motion direction of the historical track and the width change proportion of the track, and if the three are within a given threshold interval, determining that the three are effectively matched;
and S4, storing the detection target which is effectively matched into the corresponding track container.
2. The multi-target video tracking method based on multi-dimensional feature fusion of claim 1, wherein the specific contents of the step S2 are as follows:
extracting content characteristics of the detection target, and performing content characteristic similarity calculation with the content characteristics stored in all track containers to obtain a corresponding similarity value sequence; finding out the maximum similarity value from the similarity value sequence to be compared with a first threshold value, calculating the distance between the detection target and the last frame in the track container, and comparing the distance with a distance threshold value; if the maximum similarity value is greater than the first threshold and the distance is within the distance threshold, a valid match is identified.
3. The multi-dimensional feature fusion based video multi-target tracking method according to claim 1 or 2, wherein the specific contents of the step S2 are as follows:
if the track container has historical record information of the detection target, predicting the position information of the detection target which should appear in the current frame in real time to obtain a predicted rectangular area;
detecting a rectangular area of an actual detection target, calculating an intersection ratio of a prediction rectangular area and the rectangular area, and taking the difference between 1 and the intersection ratio as the distance between the actual detection target and the prediction detection target; and calculating the similarity of the content characteristics of the track container corresponding to the minimum distance and the detection target, and comparing the similarity with a first threshold value, wherein if the maximum similarity value is greater than the first threshold value, the track container is determined to be effectively matched.
4. The multi-target video tracking method based on multi-dimensional feature fusion of claim 1, wherein the specific contents of the step S3 are as follows:
for unmatched track containers, the track container PTrack with the maximum similarity to the content features of the detection target is selectedContSimultaneously recording the corresponding similarity values SimiCont(ii) a Calculating the distances between the detection target and all track containers, and taking the track container Ptrack corresponding to the minimum distanceEucCalculating the track container PtrackEucSimilarity value Simi with content feature of detection targetEuc；
S31 if PTrackContAnd PtrackEucSame, and SimiContIf the three are within the corresponding given threshold interval, the effective matching is determined;
s32 if PTrackContAnd PtrackEucSame as when SimiContAnd SimiEucWhen the larger value is larger than or equal to the given parameter threshold, calculating the width change proportion, and if the width change proportion is within the given threshold interval, determining that the width change proportion is effectively matched; when SimiContAnd SimiEucWhen the larger value is smaller than the given parameter threshold value, the two are takenCalculating the change proportion of the historical speed and the temporary speed of the track, the movement included angle and the width change proportion of the movement direction of the detection target and the historical track, and if the change proportion of the historical speed and the temporary speed of the track, the movement included angle and the width change proportion of the detection target and the historical track are within a corresponding given threshold interval, determining that the change proportion is effective matching;
s33, if the detection target still does not find a track container with effective matching, taking SimiContAnd SimiEucAnd (4) calculating the change proportion of the historical track speed and the temporary speed, and the movement included angle and the width change proportion of the detection target and the historical track movement direction, and if the three values are in the corresponding given threshold interval, determining that the three values are effectively matched.
5. The multi-dimensional feature fusion-based video multi-target tracking method according to claim 1 or 4, wherein the calculation method of the track historical speed and the temporary speed is as follows:
Vhist＝(Rt-Rt-Δt) [ Delta ] t, wherein RtIs position information of the last to last frame of the history information, Rt-ΔtIs the position information of the penultimate frame of the history information, and Δ t is the time interval of two frames of history information;
Vcur＝(Rcur-Rt)/Δtcurwherein R istIs position information of the last to last frame of the history information, RcurIs position information of the detection target, Δ tcurIs the time interval between the detection target time and the last but one frame of the history information.
6. The multi-dimensional feature fusion-based video multi-target tracking method according to claim 1 or 4, wherein the calculation method of the motion included angle between the detection target and the motion direction of the historical track is as follows:
7. The multi-target video tracking method based on multi-dimensional feature fusion of claim 4, wherein in the step S31, if PTrackContIf the history information of (2) has only one frame of information, the width change ratio is directly calculated, and if the width change ratio is within a given threshold interval, the matching is determined to be valid.
8. The multi-target video tracking method based on multi-dimensional feature fusion of claim 4, wherein the distance is Euclidean distance.
9. The multi-target video tracking method based on multi-dimensional feature fusion of claim 1, wherein the content features are extracted by a deep convolutional neural network.
10. The multi-dimensional feature fusion based video multi-target tracking method according to claim 1, wherein the position information comprises a center position, an aspect ratio and a height of a detection target.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211130193.3A CN115311329B (en) | 2019-10-11 | 2019-10-11 | Video multi-target tracking method based on double-link constraint |
| CN202211130202.9A CN115311330B (en) | 2019-10-11 | 2019-10-11 | Video multi-target tracking method based on position prediction |
| CN201910964726.XA CN110675432B (en) | 2019-10-11 | 2019-10-11 | Video multi-target tracking method based on multi-dimensional feature fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910964726.XA CN110675432B (en) | 2019-10-11 | 2019-10-11 | Video multi-target tracking method based on multi-dimensional feature fusion |
Related Child Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211130193.3A Division CN115311329B (en) | 2019-10-11 | 2019-10-11 | Video multi-target tracking method based on double-link constraint |
| CN202211130202.9A Division CN115311330B (en) | 2019-10-11 | 2019-10-11 | Video multi-target tracking method based on position prediction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110675432A true CN110675432A (en) | 2020-01-10 |
| CN110675432B CN110675432B (en) | 2022-11-08 |
Family
ID=69081635
Family Applications (3)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211130202.9A Active CN115311330B (en) | 2019-10-11 | 2019-10-11 | Video multi-target tracking method based on position prediction |
| CN201910964726.XA Active CN110675432B (en) | 2019-10-11 | 2019-10-11 | Video multi-target tracking method based on multi-dimensional feature fusion |
| CN202211130193.3A Active CN115311329B (en) | 2019-10-11 | 2019-10-11 | Video multi-target tracking method based on double-link constraint |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211130202.9A Active CN115311330B (en) | 2019-10-11 | 2019-10-11 | Video multi-target tracking method based on position prediction |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211130193.3A Active CN115311329B (en) | 2019-10-11 | 2019-10-11 | Video multi-target tracking method based on double-link constraint |
Country Status (1)
| Country | Link |
|---|---|
| CN (3) | CN115311330B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111914754A (en) * | 2020-08-03 | 2020-11-10 | 杭州云栖智慧视通科技有限公司 | Image content similarity measurement method and device and computer equipment |
| CN112184769A (en) * | 2020-09-27 | 2021-01-05 | 上海高德威智能交通系统有限公司 | Tracking abnormity identification method, device and equipment |
| CN113112526A (en) * | 2021-04-27 | 2021-07-13 | 北京百度网讯科技有限公司 | Target tracking method, device, equipment and medium |
| CN113971216A (en) * | 2021-10-22 | 2022-01-25 | 北京百度网讯科技有限公司 | Data processing method and device, electronic equipment and memory |
| CN114155273A (en) * | 2021-10-20 | 2022-03-08 | 浙江大立科技股份有限公司 | Video image single-target tracking method combined with historical track information |
| CN114329063A (en) * | 2021-10-29 | 2022-04-12 | 腾讯科技（深圳）有限公司 | Video clip detection method, device and equipment |
| CN114913198A (en) * | 2021-01-29 | 2022-08-16 | 清华大学 | Multi-target tracking method and device, storage medium and terminal |
| CN117315421A (en) * | 2023-09-26 | 2023-12-29 | 中国人民解放军91977 部队 | Method and device for predicting flight path of offshore target |
| CN117495917A (en) * | 2024-01-03 | 2024-02-02 | 山东科技大学 | Multi-target tracking method based on JDE multi-task network model |
| CN118279568A (en) * | 2024-05-31 | 2024-07-02 | 西北工业大学 | Multi-target identity judging method for distributed double-infrared sensor time sequence twin network |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115965657B (en) * | 2023-02-28 | 2023-06-02 | 安徽蔚来智驾科技有限公司 | Target tracking method, electronic device, storage medium and vehicle |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104094279A (en) * | 2014-04-30 | 2014-10-08 | 中国科学院自动化研究所 | Large-range-first cross-camera visual target re-identification method |
| CN104915970A (en) * | 2015-06-12 | 2015-09-16 | 南京邮电大学 | Multi-target tracking method based on track association |
| CN105261035A (en) * | 2015-09-15 | 2016-01-20 | 杭州中威电子股份有限公司 | Method and device for tracking moving objects on highway |
| CN107291216A (en) * | 2016-04-05 | 2017-10-24 | 中兴通讯股份有限公司 | A kind of mobile terminal method for tracking target, device and mobile terminal |
| CN109191497A (en) * | 2018-08-15 | 2019-01-11 | 南京理工大学 | A kind of real-time online multi-object tracking method based on much information fusion |
| CN109344712A (en) * | 2018-08-31 | 2019-02-15 | 电子科技大学 | A kind of road vehicle tracking |
| CN110135314A (en) * | 2019-05-07 | 2019-08-16 | 电子科技大学 | A kind of multi-object tracking method based on depth Trajectory prediction |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8116527B2 (en) * | 2009-10-07 | 2012-02-14 | The United States Of America As Represented By The Secretary Of The Army | Using video-based imagery for automated detection, tracking, and counting of moving objects, in particular those objects having image characteristics similar to background |
| CN104424648B (en) * | 2013-08-20 | 2018-07-24 | 株式会社理光 | Method for tracing object and equipment |
| CN104517275A (en) * | 2013-09-27 | 2015-04-15 | 株式会社理光 | Object detection method and system |
| CN106845385A (en) * | 2017-01-17 | 2017-06-13 | 腾讯科技（上海）有限公司 | The method and apparatus of video frequency object tracking |
| WO2019006632A1 (en) * | 2017-07-04 | 2019-01-10 | 深圳大学 | Video multi-target tracking method and device |
| CN108460787B (en) * | 2018-03-06 | 2020-11-27 | 北京市商汤科技开发有限公司 | Target tracking method and apparatus, electronic device, program, and storage medium |
| CN110276783B (en) * | 2019-04-23 | 2021-01-08 | 上海高重信息科技有限公司 | Multi-target tracking method and device and computer system |
| CN110084836B (en) * | 2019-04-26 | 2022-03-04 | 西安电子科技大学 | Target tracking method based on deep convolution characteristic hierarchical response fusion |
-
2019
- 2019-10-11 CN CN202211130202.9A patent/CN115311330B/en active Active
- 2019-10-11 CN CN201910964726.XA patent/CN110675432B/en active Active
- 2019-10-11 CN CN202211130193.3A patent/CN115311329B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104094279A (en) * | 2014-04-30 | 2014-10-08 | 中国科学院自动化研究所 | Large-range-first cross-camera visual target re-identification method |
| CN104915970A (en) * | 2015-06-12 | 2015-09-16 | 南京邮电大学 | Multi-target tracking method based on track association |
| CN105261035A (en) * | 2015-09-15 | 2016-01-20 | 杭州中威电子股份有限公司 | Method and device for tracking moving objects on highway |
| CN107291216A (en) * | 2016-04-05 | 2017-10-24 | 中兴通讯股份有限公司 | A kind of mobile terminal method for tracking target, device and mobile terminal |
| CN109191497A (en) * | 2018-08-15 | 2019-01-11 | 南京理工大学 | A kind of real-time online multi-object tracking method based on much information fusion |
| CN109344712A (en) * | 2018-08-31 | 2019-02-15 | 电子科技大学 | A kind of road vehicle tracking |
| CN110135314A (en) * | 2019-05-07 | 2019-08-16 | 电子科技大学 | A kind of multi-object tracking method based on depth Trajectory prediction |
Non-Patent Citations (4)
| Title |
|---|
| NICOLAI WOJKE ET AL: "SIMPLE ONLINE AND REALTIME TRACKING WITH A DEEP ASSOCIATION METRIC", 《ARXIV:1703.07402V1》 * |
| NICOLAI WOJKE ET AL: "SIMPLE ONLINE AND REALTIME TRACKING WITH A DEEP ASSOCIATION METRIC", 《ARXIV:1703.07402V1》, 21 March 2017 (2017-03-21), pages 1 - 5, XP080758706, DOI: 10.1109/ICIP.2017.8296962 * |
| 杜永胜: "基于运动特征的视频网络多目标匹配和跟踪方法研究", 《中国优秀博硕士学位论文全文数据库（硕士） 信息科技辑》 * |
| 杜永胜: "基于运动特征的视频网络多目标匹配和跟踪方法研究", 《中国优秀博硕士学位论文全文数据库（硕士） 信息科技辑》, 15 February 2017 (2017-02-15), pages 19 - 25 * |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111914754A (en) * | 2020-08-03 | 2020-11-10 | 杭州云栖智慧视通科技有限公司 | Image content similarity measurement method and device and computer equipment |
| CN112184769B (en) * | 2020-09-27 | 2023-05-02 | 上海高德威智能交通系统有限公司 | Method, device and equipment for identifying tracking abnormality |
| CN112184769A (en) * | 2020-09-27 | 2021-01-05 | 上海高德威智能交通系统有限公司 | Tracking abnormity identification method, device and equipment |
| CN114913198A (en) * | 2021-01-29 | 2022-08-16 | 清华大学 | Multi-target tracking method and device, storage medium and terminal |
| CN113112526A (en) * | 2021-04-27 | 2021-07-13 | 北京百度网讯科技有限公司 | Target tracking method, device, equipment and medium |
| CN113112526B (en) * | 2021-04-27 | 2023-09-22 | 北京百度网讯科技有限公司 | Target tracking method, device, equipment and medium |
| WO2022227771A1 (en) * | 2021-04-27 | 2022-11-03 | 北京百度网讯科技有限公司 | Target tracking method and apparatus, device and medium |
| CN114155273B (en) * | 2021-10-20 | 2024-06-04 | 浙江大立科技股份有限公司 | Video image single-target tracking method combining historical track information |
| CN114155273A (en) * | 2021-10-20 | 2022-03-08 | 浙江大立科技股份有限公司 | Video image single-target tracking method combined with historical track information |
| CN113971216B (en) * | 2021-10-22 | 2023-02-03 | 北京百度网讯科技有限公司 | Data processing method and device, electronic equipment and memory |
| CN113971216A (en) * | 2021-10-22 | 2022-01-25 | 北京百度网讯科技有限公司 | Data processing method and device, electronic equipment and memory |
| CN114329063A (en) * | 2021-10-29 | 2022-04-12 | 腾讯科技（深圳）有限公司 | Video clip detection method, device and equipment |
| CN114329063B (en) * | 2021-10-29 | 2024-06-11 | 腾讯科技（深圳）有限公司 | Video clip detection method, device and equipment |
| CN117315421A (en) * | 2023-09-26 | 2023-12-29 | 中国人民解放军91977 部队 | Method and device for predicting flight path of offshore target |
| CN117495917A (en) * | 2024-01-03 | 2024-02-02 | 山东科技大学 | Multi-target tracking method based on JDE multi-task network model |
| CN117495917B (en) * | 2024-01-03 | 2024-03-26 | 山东科技大学 | Multi-target tracking method based on JDE multi-task network model |
| CN118279568A (en) * | 2024-05-31 | 2024-07-02 | 西北工业大学 | Multi-target identity judging method for distributed double-infrared sensor time sequence twin network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115311329B (en) | 2023-05-23 |
| CN115311330B (en) | 2023-04-07 |
| CN115311330A (en) | 2022-11-08 |
| CN115311329A (en) | 2022-11-08 |
| CN110675432B (en) | 2022-11-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110675432B (en) | Video multi-target tracking method based on multi-dimensional feature fusion | |
| Luo et al. | Fast and furious: Real time end-to-end 3d detection, tracking and motion forecasting with a single convolutional net | |
| Ren et al. | Deep reinforcement learning with iterative shift for visual tracking | |
| CN110084831B (en) | Multi-target detection tracking method based on YOLOv3 multi-Bernoulli video | |
| CN110400332B (en) | Target detection tracking method and device and computer equipment | |
| CN106780557B (en) | Moving object tracking method based on optical flow method and key point features | |
| McLaughlin et al. | Enhancing linear programming with motion modeling for multi-target tracking | |
| CN110853078B (en) | On-line multi-target tracking method based on shielding pair | |
| CN103927764B (en) | A kind of wireless vehicle tracking of combining target information and estimation | |
| CN102147851A (en) | Device and method for judging specific object in multi-angles | |
| CN112037257B (en) | Target tracking method, terminal and computer readable storage medium thereof | |
| US20120224744A1 (en) | Image analysis method | |
| Tsintotas et al. | DOSeqSLAM: Dynamic on-line sequence based loop closure detection algorithm for SLAM | |
| CN110531618B (en) | Closed loop detection robot self-positioning error elimination method based on effective key frame | |
| CN115423846A (en) | Multi-target track tracking method and device | |
| CN113052873A (en) | Single-target tracking method for on-line self-supervision learning scene adaptation | |
| CN116645396A (en) | Track determination method, track determination device, computer-readable storage medium and electronic device | |
| CN112990119A (en) | Video multi-target facial expression recognition method and system | |
| CN116343080A (en) | Dynamic sparse key frame video target detection method, device and storage medium | |
| CN107424163A (en) | A kind of lens boundary detection method based on TextTiling | |
| Wang et al. | Deep learning and multi-modal fusion for real-time multi-object tracking: Algorithms, challenges, datasets, and comparative study | |
| Song et al. | A cross frame post-processing strategy for video object detection | |
| Gao et al. | Beyond group: Multiple person tracking via minimal topology-energy-variation | |
| Tang et al. | Place recognition using line-junction-lines in urban environments | |
| Zhang et al. | AIPT: Adaptive information perception for online multi-object tracking |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220913 Address after: 310000 Room 401, building 2, No.16, Zhuantang science and technology economic block, Xihu District, Hangzhou City, Zhejiang Province Applicant after: Hangzhou yunqi smart Vision Technology Co.,Ltd. Address before: 310000 room 279, building 6, No. 16, Zhuantang science and technology economic block, Zhuantang street, Xihu District, Hangzhou City, Zhejiang Province Applicant before: Smart vision (Hangzhou) Technology Development Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |