CN113128290A - Moving object tracking method, system, device and computer readable storage medium - Google Patents
Moving object tracking method, system, device and computer readable storage medium Download PDFInfo
- Publication number
- CN113128290A CN113128290A CN201911420284.9A CN201911420284A CN113128290A CN 113128290 A CN113128290 A CN 113128290A CN 201911420284 A CN201911420284 A CN 201911420284A CN 113128290 A CN113128290 A CN 113128290A
- Authority
- CN
- China
- Prior art keywords
- tracked
- moving object
- image
- current frame
- edge information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000001514 detection method Methods 0.000 claims abstract description 28
- 238000003062 neural network model Methods 0.000 claims abstract description 24
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 21
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 16
- 230000000306 recurrent effect Effects 0.000 claims abstract description 14
- 238000012937 correction Methods 0.000 claims description 18
- 238000012549 training Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 5
- 238000006243 chemical reaction Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/48—Matching video sequences
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biophysics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Image Analysis (AREA)
Abstract
A moving object tracking method, system, device and computer readable storage medium, wherein the method comprises: performing object detection on the current frame image based on the convolutional neural network to obtain edge information of an object in the image and corresponding object classification; identifying a moving object in the image according to object classification, and calculating according to edge information of the moving object by adopting a multi-path regression algorithm to obtain position information of the moving object; judging whether an object to be tracked matched with the moving object exists in the tracked list or not; if so, updating the edge information and the position information of the corresponding object to be tracked in the tracked list; if not, adding the moving object as a new object to be tracked into the tracked list; and predicting the edge information and the position information of the object to be tracked in the next frame of image by adopting a recurrent neural network model according to the edge information and the position information of the current object to be tracked. The embodiment of the invention improves the generalization of image feature extraction and the precision of moving object track prediction.
Description
Technical Field
The embodiment of the invention relates to the technical field of object tracking, in particular to a moving object tracking method, a system, a device and a computer readable storage medium.
Background
With the development of video analysis technology, moving object tracking technology is widely applied in video monitoring, intelligent analysis, navigation and other fields. At present, a moving object tracking method in a video analysis technology generally adopts a machine learning-based mode to perform image extraction feature and motion trajectory prediction, and needs to apply prior knowledge (manual intervention is needed in the image feature extraction and motion trajectory prediction process, and the prior experience of people is applied), so that the defects that the extracted features do not have good generalization and the predicted trajectory has great deviation from the actual trajectory are caused.
Disclosure of Invention
In view of the above, embodiments of the present invention provide a moving object tracking method, system, device and computer readable storage medium, so as to solve the problems that the extracted features do not have good generalization and the predicted trajectory and the actual trajectory have great deviation in the existing moving object tracking method.
The technical scheme adopted by the embodiment of the invention for solving the technical problems is as follows:
according to a first aspect of embodiments of the present invention, there is provided a moving object tracking method, including:
performing object detection on the current frame image based on a convolutional neural network to obtain edge information of an object in the current frame image and corresponding object classification;
identifying a moving object in the current frame image according to the object classification, and calculating according to the edge information of the moving object by adopting a multi-path regression algorithm to obtain the position information of the moving object;
judging whether an object to be tracked matched with the moving object exists in the tracked list or not;
if the object to be tracked matched with the moving object is inquired in the tracked list, updating the edge information and the position information of the corresponding object to be tracked in the tracked list;
if the object to be tracked matched with the moving object is not inquired in the tracked object list, the moving object is taken as a new object to be tracked and added into the tracked list;
and predicting the edge information and the position information of the object to be tracked in the next frame of image by adopting a recurrent neural network model according to the edge information and the position information of the object to be tracked in the tracked list.
The object detection of the current frame image based on the convolutional neural network, and the obtaining of the edge information of the object in the current frame image and the corresponding object classification comprises:
performing forward operation on the current frame image by adopting a convolutional neural network to obtain high-dimensional characteristics of different dimensions of an object in the current frame image;
and regressing the high-dimensional characteristics of the object with different dimensions to obtain the edge information of the object and the corresponding object classification.
The method for detecting the object of the current frame image based on the convolutional neural network further comprises the following steps before the object detection is performed on the current frame image and the edge information of the object in the current frame image and the corresponding object classification are obtained:
and determining a distortion correction model according to the parameter information of the image acquisition device, and carrying out distortion removal processing on the current frame image by using the distortion correction model.
The method for calculating and obtaining the position information of the object according to the edge information of the moving object by adopting a multi-path regression algorithm comprises the following steps:
calculating the angle of the object relative to the image acquisition device according to the edge information of the object and the parameter information of the image acquisition device;
and calculating the distance from the center of the object to the image acquisition device according to the adjacent point of the object and the ground, the height of the image acquisition device from the ground and the focal length and the posture of the image acquisition device.
Wherein the judging whether the object to be tracked matched with the moving object exists in the tracked list comprises:
splitting the current frame image into a plurality of moving object region images according to edge information of a moving object in the current frame image;
coding the moving object region image corresponding to the moving object region image into a 128-dimensional first vector by adopting a neural network model of the image characteristics of the moving object generated in advance through ternary loss training;
encoding the image of the object to be tracked in the tracked list into a 128-dimensional second vector by adopting a neural network model of the image characteristics of the moving object generated in advance through ternary loss training;
calculating a euclidean distance between the first vector and the second vector;
if the Euclidean distance is smaller than or equal to a preset threshold value, the moving object and the object to be tracked are the same object;
if the Euclidean distance is larger than the preset threshold value, the moving object and the object to be tracked are not the same object.
According to a second aspect of embodiments of the present invention, there is provided a moving object tracking system including:
the object detection unit is used for carrying out object detection on the current frame image based on the convolutional neural network to obtain the edge information of the object in the current frame image and the corresponding object classification;
the 3D information regression unit is used for identifying a moving object in the current frame image according to the object classification and calculating the position information of the moving object according to the edge information of the moving object by adopting a multi-path regression algorithm;
the object tracking unit is used for judging whether an object to be tracked matched with the moving object exists in the tracked list or not; if the object to be tracked matched with the moving object is inquired in the tracked list, updating the edge information and the position information of the corresponding object to be tracked in the tracked list; if the object to be tracked matched with the moving object is not inquired in the tracked object list, the moving object is taken as a new object to be tracked and added into the tracked list;
and the motion trail prediction unit is used for predicting the edge information and the position information of the object to be tracked in the next frame of image by adopting a recurrent neural network model according to the edge information and the position information of the object to be tracked in the tracked list.
Wherein the object detection unit is specifically configured to:
performing forward operation on the current frame image by adopting a convolutional neural network to obtain high-dimensional characteristics of different dimensions of an object in the current frame image;
and regressing the high-dimensional characteristics of the object with different dimensions to obtain the edge information of the object and the corresponding object classification.
Wherein the moving object tracking system further comprises:
and the distortion correction unit is used for determining a distortion correction model according to the parameter information of the image acquisition device and carrying out distortion removal processing on the current frame image by using the distortion correction model.
Wherein the 3D information regression unit is specifically configured to:
calculating the angle of the object relative to the image acquisition device according to the edge information of the object and the parameter information of the image acquisition device;
and calculating the distance from the center of the object to the image acquisition device according to the adjacent point of the object and the ground, the height of the image acquisition device from the ground and the focal length and the posture of the image acquisition device.
Wherein the object tracking unit is specifically configured to:
splitting the current frame image into a plurality of moving object region images according to edge information of a moving object in the current frame image;
coding the moving object region image corresponding to the moving object region image into a 128-dimensional first vector by adopting a neural network model of the image characteristics of the moving object generated in advance through ternary loss training;
encoding the image of the object to be tracked in the tracked list into a 128-dimensional second vector by adopting a neural network model of the image characteristics of the moving object generated in advance through ternary loss training;
calculating a euclidean distance between the first vector and the second vector;
if the Euclidean distance is smaller than or equal to a preset threshold value, the moving object and the object to be tracked are the same object;
if the Euclidean distance is larger than the preset threshold value, the moving object and the object to be tracked are not the same object.
According to a third aspect of embodiments of the present invention, there is provided a moving object tracking device including: memory, a processor and a computer program stored on the memory and executable on the processor, which computer program, when executed by the processor, performs the steps of the moving object tracking method according to any of the above first aspects.
According to a fourth aspect of embodiments of the present invention, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the moving object tracking method according to any one of the above-mentioned first aspects.
According to the moving object tracking method, the moving object tracking system, the moving object tracking equipment and the computer readable storage medium, due to the fact that the current frame image is subjected to object detection based on the convolutional neural network, the edge information of the object in the current frame image and the corresponding object classification are obtained, prior knowledge does not need to be applied in the object detection process, and the object edge information extracted by object detection and the generalization of the corresponding object classification are improved; the position information of the moving object is obtained by adopting the multi-path algorithm according to the edge information of the moving object, and the edge information and the position information of the object to be tracked in the next frame of image are predicted by adopting a recurrent neural network model according to the edge information of the moving object and the position information of the moving object obtained by calculation based on the multi-path algorithm, so that the accuracy of track prediction is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a schematic flow chart of a specific implementation of a moving object tracking method according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of a specific implementation of a moving object tracking method according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of a moving object tracking system according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a moving object tracking apparatus according to a fourth embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantageous effects to be solved by the present invention clearer and clearer, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Example one
Fig. 1 is a schematic flowchart of a specific implementation of a moving object tracking method according to an embodiment of the present invention. Referring to fig. 1, the moving object tracking method provided in this embodiment may include the following steps:
step S101, performing object detection on the current frame image based on a convolutional neural network, and acquiring edge information of an object in the current frame image and corresponding object classification.
In this embodiment, step S101 specifically includes:
performing forward operation on the current frame image by adopting a convolutional neural network to obtain high-dimensional characteristics of different dimensions of an object in the current frame image;
and regressing the high-dimensional characteristics of the object with different dimensions to obtain the edge information of the object and the corresponding object classification.
In this embodiment, forward operation is performed on the current frame image by using a convolutional neural network to obtain a higher-dimensional feature, so that the extracted image feature has higher generalization; after the high-dimensional features of different dimensions of the image are obtained, the edge information (2D frame) of the object in the image and the corresponding object classification are regressed at the same time on different dimensions, and the 2D frame of the object and the corresponding object classification are obtained based on the high-dimensional feature regression of different dimensions, so that the method has stronger robustness on the objects with different sizes.
And S102, identifying a moving object in the current frame image according to the object classification, and calculating to obtain the position information of the moving object according to the edge information of the moving object by adopting a multi-path regression algorithm.
In this embodiment, after detecting the 2D frame of the object and the corresponding object classification in the current frame image, it may be identified whether the object is a static object or a moving object according to the corresponding object classification of the object, and if the object is identified as a static object, no processing is performed; and if the object is identified, calculating to obtain the position information of the object according to the edge information of the object by adopting a multi-path regression algorithm. The edge information of the object comprises the coordinates and the size of the object in the current frame image, and is also called a 2D frame of the object; the position information of the object includes an angle of the object with respect to the image capturing device and a distance of the object corresponding to the image capturing device, also referred to as 3D information of the object.
Specifically, the obtaining of the position information of the object by calculating according to the edge information of the moving object by using a multipath regression algorithm includes:
calculating the angle of the object relative to the image acquisition device according to the edge information of the object and the parameter information of the image acquisition device;
and calculating the distance from the center of the object to the image acquisition device according to the adjacent point of the object and the ground, the height of the image acquisition device from the ground and the focal length and the posture of the image acquisition device.
In this embodiment, the 2D frame of the object projects 8 vertices of the object in the 3-dimensional space onto the minimum bounding rectangle in the 2D space, so that the angle of the current object in the space relative to the image capturing device can be calculated by combining the internal reference and the external reference of the image capturing device based on the detected edge information of the object in the current frame image, where the edge information includes the coordinates and the size information of the object in the current frame image. Specifically, the method comprises the following steps: and combining the internal reference and the external reference of the camera to obtain a conversion relation between an image coordinate system and a camera coordinate system, obtaining the coordinates of the center of the object in the image under the camera coordinate system according to the conversion relation between the image coordinate system and the camera coordinate system, and then obtaining the angle of the object relative to the camera according to the coordinates of the center of the object under the camera coordinate system.
In this embodiment, based on the principle of pinhole imaging, under the condition that the height of the image capturing device from the ground, the focal length of the image capturing device, and the posture information of the image capturing device are known, the distance between the object and the image capturing device can be calculated based on the proximity point between the object and the ground. Specifically, the method comprises the following steps: converting the height of the detected image plane of the object into the height h mapped to the pixel screen, and then calculating the distance D from the object to the camera according to the following formula by the height D of the camera and the focal length f of the camera: d ═ f × D/h.
Step S103, judging whether an object to be tracked matched with the moving object exists in a tracked list, and entering step S104-1 if the object to be tracked matched with the moving object is inquired in the tracked list; and if the object to be tracked matched with the moving object is not inquired in the tracked object list, the step S104-2 is carried out.
In this embodiment, the determining whether the object to be tracked matching the moving object exists in the tracked list specifically includes:
splitting the current frame image into a plurality of moving object region images according to edge information of a moving object in the current frame image;
coding the moving object region image corresponding to the moving object region image into a 128-dimensional first vector by adopting a neural network model of the image characteristics of the moving object generated in advance through ternary loss training;
encoding the image of the object to be tracked in the tracked list into a 128-dimensional second vector by adopting a neural network model of the image characteristics of the moving object generated in advance through ternary loss training;
calculating a euclidean distance between the first vector and the second vector;
if the Euclidean distance is smaller than or equal to a preset threshold value, the moving object and the object to be tracked are the same object;
if the Euclidean distance is larger than the preset threshold value, the moving object and the object to be tracked are not the same object.
And step S104-1, updating the edge information and the position information of the corresponding object to be tracked in the tracked list.
And step S104-2, adding the moving object as a new object to be tracked into a tracked list.
In this embodiment, if the object to be tracked which is matched with the moving object in the tracked list is inquired, it is indicated that the moving object exists in the tracked list, and at this time, the edge information and the position information of the object to be tracked which are matched in the tracked list are updated to the current edge information and the current position information of the moving object; on the contrary, if the object to be tracked matched with the moving object is not inquired in the tracked list, it indicates that the moving object is not tracked before the current frame image, so that the moving object is added into the tracked list as a new object to be tracked, and the current edge information and position information of the newly added object to be tracked are recorded.
And step S105, adopting a recurrent neural network model to predict the edge information and the position information of the object to be tracked in the next frame of image according to the edge information and the position information of the object to be tracked in the tracked list.
The recurrent neural network model is a time recurrent network, and is suitable for processing the problem that a predicted time sequence contains long-distance dependence. In this embodiment, the prediction of the motion trajectory by the recurrent neural network model is to predict the motion trajectory of the object at a future time based on the time sequence information, take the 3D information of the tracked object (including the angle of the moving object relative to the image acquisition device and the distance of the moving object relative to the image acquisition device) and the 2D frame of the object as the input of the predicted motion trajectory, and finally predict the 3D information and the 2D frame of the moving object at the next time through the recurrent neural network.
According to the moving object tracking method, the moving object tracking system, the moving object tracking equipment and the computer readable storage medium, due to the fact that the current frame image is subjected to object detection based on the convolutional neural network, the edge information of the object in the current frame image and the corresponding object classification are obtained, prior knowledge does not need to be applied in the object detection process, and the object edge information extracted by object detection and the generalization of the corresponding object classification are improved; the position information of the moving object is obtained by adopting the multi-path algorithm according to the edge information of the moving object, and the edge information and the position information of the object to be tracked in the next frame of image are predicted by adopting a recurrent neural network model according to the edge information of the moving object and the position information of the moving object obtained by calculation based on the multi-path algorithm, so that the accuracy of the trajectory prediction of the moving object is improved.
Example two
Fig. 2 is a schematic flowchart of a specific implementation of the moving object tracking method according to the second embodiment of the present invention. Referring to fig. 2, the moving object tracking method provided in this embodiment further includes:
step S201, determining a distortion correction model according to the parameter information of the image acquisition device, and carrying out distortion removal processing on the current frame image by using the distortion correction model.
In this embodiment, because the precision and the process of the lens of the image acquisition device may cause distortion of the acquired image, thereby causing image distortion, before performing object detection on the image acquired by the image acquisition device, a distortion correction model of the image acquisition device needs to be used to perform distortion correction on the image, so that the reliability of a subsequent object detection result can be ensured, and the precision of motion trajectory prediction is further improved. Further, step S201 specifically includes: and determining a conversion relation between the distorted image and the real image according to the internal reference and the external reference of the image acquisition device, and then obtaining the pixel coordinates of the corrected object image according to the coordinates of the detected object in the image pixel coordinate system and the conversion relation between the distorted image and the real image.
Step S202, performing object detection on the current frame image based on the convolutional neural network, and acquiring edge information of an object in the current frame image and corresponding object classification.
And step S203, identifying a moving object in the current frame image according to the object classification, and calculating to obtain the position information of the moving object according to the edge information of the moving object by adopting a multi-path regression algorithm.
Step S204, judging whether an object to be tracked matched with the moving object exists in the tracked list; if the object to be tracked matched with the moving object is inquired in the tracked list, the step S205-1 is carried out; if the object to be tracked matched with the moving object is not found in the tracked object list, the step S205-2 is performed.
And step S205-1, updating the edge information and the position information of the corresponding object to be tracked in the tracked list.
And step S205-2, adding the moving object as a new object to be tracked into the tracked list.
Step S206, adopting a recurrent neural network model to predict the edge information and the position information of the object to be tracked in the next frame of image according to the edge information and the position information of the object to be tracked in the tracked list.
It should be noted that, since the implementation manners of step S202 to step S206 in the present embodiment are completely the same as the implementation manners of step S101 to step S105 in the previous embodiment, detailed descriptions thereof are omitted here.
Compared with the previous embodiment, the moving object tracking method provided by the embodiment performs distortion correction on the image before performing object detection on the image, so that the reliability of object detection can be improved, and the accuracy of a moving object track prediction result is further improved.
EXAMPLE III
Fig. 3 is a schematic structural diagram of a moving object tracking system according to a third embodiment of the present invention. Only the portions related to the present embodiment are shown for convenience of explanation.
Referring to fig. 3, the moving object tracking system 3 according to the present embodiment includes:
the object detection unit 31 is configured to perform object detection on the current frame image based on a convolutional neural network, and acquire edge information of an object in the current frame image and a corresponding object classification;
the 3D information regression unit 32 is configured to identify a moving object in the current frame image according to the object classification, and calculate position information of the moving object according to edge information of the moving object by using a multipath regression algorithm;
an object tracking unit 33, configured to determine whether an object to be tracked that matches the moving object exists in the tracked list; if the object to be tracked matched with the moving object is inquired in the tracked list, updating the edge information and the position information of the corresponding object to be tracked in the tracked list; if the object to be tracked matched with the moving object is not inquired in the tracked object list, the moving object is taken as a new object to be tracked and added into the tracked list;
and the motion trail prediction unit 34 is configured to predict the edge information and the position information of the object to be tracked in the next frame of image by using a recurrent neural network model according to the edge information and the position information of the object to be tracked in the tracked list.
Optionally, the object detection unit 31 is specifically configured to:
performing forward operation on the current frame image by adopting a convolutional neural network to obtain high-dimensional characteristics of different dimensions of an object in the current frame image;
and regressing the high-dimensional characteristics of the object with different dimensions to obtain the edge information of the object and the corresponding object classification.
Optionally, the moving object tracking system 3 further includes:
and the distortion correction unit 35 is configured to determine a distortion correction model according to the parameter information of the image acquisition device, and perform distortion removal processing on the current frame image by using the distortion correction model.
Optionally, the 3D information regression unit 32 is specifically configured to:
calculating the angle of the object relative to the image acquisition device according to the edge information of the object and the parameter information of the image acquisition device;
and calculating the distance from the center of the object to the image acquisition device according to the adjacent point of the object and the ground, the height of the image acquisition device from the ground and the focal length and the posture of the image acquisition device.
Optionally, the object tracking unit 33 is specifically configured to:
splitting the current frame image into a plurality of moving object region images according to edge information of a moving object in the current frame image;
coding the moving object region image corresponding to the moving object region image into a 128-dimensional first vector by adopting a neural network model of the image characteristics of the moving object generated in advance through ternary loss training;
encoding the image of the object to be tracked in the tracked list into a 128-dimensional second vector by adopting a neural network model of the image characteristics of the moving object generated in advance through ternary loss training;
calculating a euclidean distance between the first vector and the second vector;
if the Euclidean distance is smaller than or equal to a preset threshold value, the moving object and the object to be tracked are the same object;
if the Euclidean distance is larger than the preset threshold value, the moving object and the object to be tracked are not the same object.
It should be noted that the system of this embodiment and the moving object tracking method provided in the first embodiment or the second embodiment belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments, and technical features in the method embodiments are correspondingly applicable in this embodiment, and are not described herein again.
Example four
An embodiment of the present invention provides a moving object tracking device 4, including: a memory 41, a processor 42 and a computer program 43 stored on the memory 41 and executable on the processor 42, wherein the computer program 43 when executed by the processor 42 implements the steps of the moving object tracking method as described in the first embodiment or the second embodiment.
It should be noted that the device of this embodiment and the moving object tracking method provided in the first embodiment or the second embodiment belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments, and technical features in the method embodiments are correspondingly applicable in this embodiment, and are not described herein again.
EXAMPLE five
A fifth embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method for tracking a moving object according to the first embodiment or the second embodiment is implemented.
It should be noted that the computer-readable storage medium of this embodiment and the moving object tracking method provided in the first embodiment or the second embodiment belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments, and technical features in the method embodiments are correspondingly applicable in this embodiment, and are not described herein again.
It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof.
In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, and are not to be construed as limiting the scope of the invention. Any modifications, equivalents and improvements which may occur to those skilled in the art without departing from the scope and spirit of the present invention are intended to be within the scope of the claims.
Claims (12)
1. A moving object tracking method, comprising:
performing object detection on the current frame image based on a convolutional neural network to obtain edge information of an object in the current frame image and corresponding object classification;
identifying a moving object in the current frame image according to the object classification, and calculating according to the edge information of the moving object by adopting a multi-path regression algorithm to obtain the position information of the moving object;
judging whether an object to be tracked matched with the moving object exists in the tracked list or not;
if the object to be tracked matched with the moving object is inquired in the tracked list, updating the edge information and the position information of the corresponding object to be tracked in the tracked list;
if the object to be tracked matched with the moving object is not inquired in the tracked object list, the moving object is taken as a new object to be tracked and added into the tracked list;
and predicting the edge information and the position information of the object to be tracked in the next frame of image by adopting a recurrent neural network model according to the edge information and the position information of the object to be tracked in the tracked list.
2. The moving object tracking method of claim 1, wherein the performing object detection on the current frame image based on the convolutional neural network, and obtaining the edge information of the object in the current frame image and the corresponding object classification comprises:
performing forward operation on the current frame image by adopting a convolutional neural network to obtain high-dimensional characteristics of different dimensions of an object in the current frame image;
and regressing the high-dimensional characteristics of the object with different dimensions to obtain the edge information of the object and the corresponding object classification.
3. The moving object tracking method according to claim 1, wherein before the performing object detection on the current frame image based on the convolutional neural network and obtaining the edge information of the object in the current frame image and the corresponding object classification, the method further comprises:
and determining a distortion correction model according to the parameter information of the image acquisition device, and carrying out distortion removal processing on the current frame image by using the distortion correction model.
4. The moving object tracking method of claim 1, wherein calculating the position information of the object from the edge information of the moving object using a multipath regression algorithm comprises:
calculating the angle of the object relative to the image acquisition device according to the edge information of the object and the parameter information of the image acquisition device;
and calculating the distance from the center of the object to the image acquisition device according to the adjacent point of the object and the ground, the height of the image acquisition device from the ground and the focal length and the posture of the image acquisition device.
5. The moving object tracking method of claim 1, wherein determining whether an object to be tracked that matches the moving object exists in the tracked list comprises:
splitting the current frame image into a plurality of moving object region images according to edge information of a moving object in the current frame image;
coding the moving object region image corresponding to the moving object region image into a 128-dimensional first vector by adopting a neural network model of the image characteristics of the moving object generated in advance through ternary loss training;
encoding the image of the object to be tracked in the tracked list into a 128-dimensional second vector by adopting a neural network model of the image characteristics of the moving object generated in advance through ternary loss training;
calculating a euclidean distance between the first vector and the second vector;
if the Euclidean distance is smaller than or equal to a preset threshold value, the moving object and the object to be tracked are the same object;
if the Euclidean distance is larger than the preset threshold value, the moving object and the object to be tracked are not the same object.
6. A moving object tracking system, comprising:
the object detection unit is used for carrying out object detection on the current frame image based on the convolutional neural network to obtain the edge information of the object in the current frame image and the corresponding object classification;
the 3D information regression unit is used for identifying a moving object in the current frame image according to the object classification and calculating the position information of the moving object according to the edge information of the moving object by adopting a multi-path regression algorithm;
the object tracking unit is used for judging whether an object to be tracked matched with the moving object exists in the tracked list or not; if the object to be tracked matched with the moving object is inquired in the tracked list, updating the edge information and the position information of the corresponding object to be tracked in the tracked list; if the object to be tracked matched with the moving object is not inquired in the tracked object list, the moving object is taken as a new object to be tracked and added into the tracked list;
and the motion trail prediction unit is used for predicting the edge information and the position information of the object to be tracked in the next frame of image by adopting a recurrent neural network model according to the edge information and the position information of the object to be tracked in the tracked list.
7. The moving object tracking system of claim 6, wherein the object detection unit is specifically configured to:
performing forward operation on the current frame image by adopting a convolutional neural network to obtain high-dimensional characteristics of different dimensions of an object in the current frame image;
and regressing the high-dimensional characteristics of the object with different dimensions to obtain the edge information of the object and the corresponding object classification.
8. The moving object tracking system of claim 6, further comprising:
and the distortion correction unit is used for determining a distortion correction model according to the parameter information of the image acquisition device and carrying out distortion removal processing on the current frame image by using the distortion correction model.
9. The moving object tracking system of claim 6, wherein the 3D information regression unit is specifically configured to:
calculating the angle of the object relative to the image acquisition device according to the edge information of the object and the parameter information of the image acquisition device;
and calculating the distance from the center of the object to the image acquisition device according to the adjacent point of the object and the ground, the height of the image acquisition device from the ground and the focal length and the posture of the image acquisition device.
10. The moving object tracking system of claim 6, wherein the object tracking unit is specifically configured to:
splitting the current frame image into a plurality of moving object region images according to edge information of a moving object in the current frame image;
coding the moving object region image corresponding to the moving object region image into a 128-dimensional first vector by adopting a neural network model of the image characteristics of the moving object generated in advance through ternary loss training;
encoding the image of the object to be tracked in the tracked list into a 128-dimensional second vector by adopting a neural network model of the image characteristics of the moving object generated in advance through ternary loss training;
calculating a euclidean distance between the first vector and the second vector;
if the Euclidean distance is smaller than or equal to a preset threshold value, the moving object and the object to be tracked are the same object;
if the Euclidean distance is larger than the preset threshold value, the moving object and the object to be tracked are not the same object.
11. A moving object tracking device, comprising: memory, processor and computer program stored on the memory and executable on the processor, which computer program, when executed by the processor, carries out the steps of the moving object tracking method according to any one of claims 1 to 5.
12. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the moving object tracking method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911420284.9A CN113128290A (en) | 2019-12-31 | 2019-12-31 | Moving object tracking method, system, device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911420284.9A CN113128290A (en) | 2019-12-31 | 2019-12-31 | Moving object tracking method, system, device and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113128290A true CN113128290A (en) | 2021-07-16 |
Family
ID=76769420
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911420284.9A Withdrawn CN113128290A (en) | 2019-12-31 | 2019-12-31 | Moving object tracking method, system, device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113128290A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106371599A (en) * | 2016-09-08 | 2017-02-01 | 清华大学 | Method and device for high-precision fingertip positioning in depth image |
| CN107481270A (en) * | 2017-08-10 | 2017-12-15 | 上海体育学院 | Table tennis target following and trajectory predictions method, apparatus, storage medium and computer equipment |
| CN109035304A (en) * | 2018-08-07 | 2018-12-18 | 北京清瑞维航技术发展有限公司 | Method for tracking target, calculates equipment and device at medium |
| CN109448023A (en) * | 2018-10-23 | 2019-03-08 | 武汉大学 | A kind of satellite video Small object method for real time tracking of combination space confidence map and track estimation |
-
2019
- 2019-12-31 CN CN201911420284.9A patent/CN113128290A/en not_active Withdrawn
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106371599A (en) * | 2016-09-08 | 2017-02-01 | 清华大学 | Method and device for high-precision fingertip positioning in depth image |
| CN107481270A (en) * | 2017-08-10 | 2017-12-15 | 上海体育学院 | Table tennis target following and trajectory predictions method, apparatus, storage medium and computer equipment |
| CN109035304A (en) * | 2018-08-07 | 2018-12-18 | 北京清瑞维航技术发展有限公司 | Method for tracking target, calculates equipment and device at medium |
| CN109448023A (en) * | 2018-10-23 | 2019-03-08 | 武汉大学 | A kind of satellite video Small object method for real time tracking of combination space confidence map and track estimation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230077355A1 (en) | Tracker assisted image capture | |
| CN111145238B (en) | Three-dimensional reconstruction method and device for monocular endoscopic image and terminal equipment | |
| CN109035304B (en) | Target tracking method, medium, computing device and apparatus | |
| CN107633526B (en) | Image tracking point acquisition method and device and storage medium | |
| CN108229475B (en) | Vehicle tracking method, system, computer device and readable storage medium | |
| CN111144213B (en) | Object detection method and related equipment | |
| EP3709266A1 (en) | Human-tracking methods, apparatuses, systems, and storage media | |
| CN111512317A (en) | Multi-target real-time tracking method and device and electronic equipment | |
| CN110335313B (en) | Audio acquisition equipment positioning method and device and speaker identification method and system | |
| US20230214981A1 (en) | Method for detecting appearance defects of a product and electronic device | |
| CN111340749B (en) | Image quality detection method, device, equipment and storage medium | |
| CN109859104B (en) | Method for generating picture by video, computer readable medium and conversion system | |
| CN113763466B (en) | Loop detection method and device, electronic equipment and storage medium | |
| CN115797440A (en) | Battery cell positioning method, controller and computer readable storage medium | |
| CN111951328A (en) | Object position detection method, device, equipment and storage medium | |
| CN112802112B (en) | Visual positioning method, device, server and storage medium | |
| CN111488762A (en) | Lane-level positioning method and device and positioning equipment | |
| CN110706257B (en) | Identification method of effective characteristic point pair, and camera state determination method and device | |
| CN109741370B (en) | Target tracking method and device | |
| CN116330658B (en) | Target tracking method, device and system based on depth image and image pickup equipment | |
| CN113128290A (en) | Moving object tracking method, system, device and computer readable storage medium | |
| CN115294358A (en) | Feature point extraction method and device, computer equipment and readable storage medium | |
| CN113538337B (en) | Detection method, detection device and computer readable storage medium | |
| KR20150137698A (en) | Method and apparatus for movement trajectory tracking of moving object on animal farm | |
| CN112262411B (en) | Image association method, system and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 518000 Room 401, block D, building 7, Shenzhen International Innovation Valley, Dashi 1st Road, Xili community, Xili street, Nanshan District, Shenzhen, Guangdong Applicant after: JUXING TECHNOLOGY (SHENZHEN) Co.,Ltd. Address before: 518000 building 101, building R3b, Gaoxin industrial village, No.018, Gaoxin South 7th Road, community, high tech Zone, Yuehai street, Nanshan District, Shenzhen City, Guangdong Province Applicant before: JUXING TECHNOLOGY (SHENZHEN) Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20210716 |