CN109270523B - Multi-sensor data fusion method and device and vehicle - Google Patents
Multi-sensor data fusion method and device and vehicle Download PDFInfo
- Publication number
- CN109270523B CN109270523B CN201811109761.5A CN201811109761A CN109270523B CN 109270523 B CN109270523 B CN 109270523B CN 201811109761 A CN201811109761 A CN 201811109761A CN 109270523 B CN109270523 B CN 109270523B
- Authority
- CN
- China
- Prior art keywords
- dimension
- data
- sensor
- interval
- data interval
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000007500 overflow downdraw method Methods 0.000 title abstract description 5
- 238000000034 method Methods 0.000 claims abstract description 31
- 230000001133 acceleration Effects 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 4
- 230000004927 fusion Effects 0.000 description 4
- 238000001514 detection method Methods 0.000 description 3
- 206010039203 Road traffic accident Diseases 0.000 description 2
- 238000007499 fusion processing Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
- G01S13/867—Combination of radar systems with cameras
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
- G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Physics & Mathematics (AREA)
- Electromagnetism (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
The disclosure provides a multi-sensor data fusion method and device and a vehicle, which aim to solve the problem that the correlation accuracy of multi-sensor data in the related art is not high enough. A plurality of sensors for detecting data of multiple dimensions of an object in the same region, the method comprising: setting a corresponding data interval for each dimension of a first object according to data of multiple dimensions of the first object detected by a first sensor, wherein the first object is any one object detected by the first sensor; judging whether a unique second object exists in all the objects detected by the second sensor, wherein the data of each dimension of the second object are in each data interval set for the first object; and if the second object is unique in all the objects detected by the second sensor, associating the first object and the second object as corresponding to the same object in the area.
Description
Technical Field
The disclosure relates to the field of data processing, in particular to a multi-sensor data fusion method and device and a vehicle.
Background
More and more vehicles adopt sensors to acquire data information of a target object on a road, and the driving is assisted through the data information, so that the automatic driving or semi-automatic driving function is realized. In the related art, a technical scheme that data information of a target object is acquired through a plurality of sensors, and then the data information acquired by the plurality of sensors is subjected to data fusion to obtain information of the target object is provided.
Before data fusion processing is carried out on data of a plurality of sensors, data correlation needs to be carried out on target objects acquired by different sensors. For example, data association is performed on a plurality of obstacles detected by the millimeter wave radar and the camera, and data information of each obstacle after data fusion is obtained. Furthermore, the obstacle detection system judges whether the obstacle can influence the normal running of the vehicle according to the fused data.
However, the accuracy of the plurality of sensors is different, and when data association is performed, data results corresponding to different actual targets may be associated to correspond to the same target, which may cause misjudgment of the vehicle during target detection, reduce the performance of the target detection system, and even cause traffic accidents when the vehicle runs in a semi-automatic driving mode or an automatic driving mode.
Disclosure of Invention
The disclosure provides a multi-sensor data fusion method and device and a vehicle, which aim to solve the problem that the correlation accuracy of multi-sensor data in the related art is not high enough.
To achieve the above object, in a first aspect, the embodiments of the present disclosure provide a multi-sensor data association method, where multiple sensors are used to detect data of multiple dimensions of an object in the same area, the method includes:
setting a corresponding data interval for each dimension of a first object according to data of multiple dimensions of the first object detected by a first sensor, wherein the first object is any one object detected by the first sensor;
judging whether a unique second object exists in all the objects detected by the second sensor, wherein the data of each dimension of the second object are in each data interval set for the first object;
and if the second object is unique in all the objects detected by the second sensor, associating the first object and the second object as corresponding to the same object in the area.
Optionally, the method includes:
and when the second sensor detects that the unique second object does not exist in all the objects detected by the second sensor, setting a corresponding data interval for each dimension of the first object again according to a preset interval gradient to obtain a new data interval corresponding to each dimension of the first object, and re-executing the step of judging whether the unique second object exists in all the objects detected by the second sensor.
Optionally, the method includes:
each time it is determined that the second object does not exist in all the objects detected by the second sensor, the length of the original data interval corresponding to each dimension of the first object is synchronously enlarged according to the first gradient increment corresponding to each dimension to obtain a new data interval corresponding to each dimension of the first object, and the step of judging whether the unique second object exists in all the objects detected by the second sensor is executed again;
optionally, the method includes:
if the length of the original data interval corresponding to each dimension of the first object is synchronously enlarged according to the first gradient increment corresponding to each dimension, when a plurality of second objects with data of each dimension in the corresponding new data interval exist in all the objects detected by the second sensor, the length of the original data interval corresponding to each dimension of the first object is synchronously enlarged according to the second gradient increment corresponding to each dimension to obtain a new data interval corresponding to each dimension of the first object, wherein the second gradient increment corresponding to each dimension is smaller than the first gradient increment corresponding to each dimension, and the step of judging whether the unique second object exists in all the objects detected by the second sensor is executed again.
Optionally, the method includes:
and synchronously reducing the length of the original data interval corresponding to each dimension of the first object according to the first gradient reduction amount corresponding to each dimension when a plurality of second objects exist in all the objects detected by the second sensor every time, so as to obtain a new data interval corresponding to each dimension of the first object, and re-executing the step of judging whether the unique second objects exist in all the objects detected by the second sensor.
Optionally, the method includes:
when the length of the data interval corresponding to each dimension of the first object is set as the threshold of the length of the data interval corresponding to the dimension, if the second object of which the data of each dimension is in the corresponding new data interval does not exist in all the objects detected by the second sensor, it is determined that the object associated with the first object as the same object in the corresponding area does not exist in all the objects detected by the second sensor.
Optionally, the setting, according to the data of the multiple dimensions of the first object detected by the first sensor, a corresponding data interval for each dimension of the first object includes:
and taking the data of each dimension of the first object as a middle value of a data interval of the corresponding dimension, taking the product of the data of each dimension of the first object and a preset gradient percentage corresponding to the dimension as an interval length of the data interval of the corresponding dimension, and setting the data interval.
Optionally, the data of multiple dimensions includes data of any of the following dimensions of the detected object: coordinate position data, velocity data, acceleration data, motion trend data.
In a second aspect, the disclosed embodiments provide a multi-sensor data association apparatus, where multiple sensors are used to detect data of multiple dimensions of an object in the same area, the apparatus comprising:
the device comprises a setting module, a data processing module and a display module, wherein the setting module is used for setting a corresponding data interval for each dimension of a first object according to data of multiple dimensions of the first object detected by a first sensor, and the first object is any object detected by the first sensor;
the judging module is used for judging whether a unique second object exists in all the objects detected by the second sensor, and data of each dimension of the second object are all in each data interval correspondingly set for the first object;
and the association module is used for associating the first object and the second object as the same object in the corresponding area when the unique second object exists in all the objects detected by the second sensor.
Optionally, the setting module is configured to, when it is determined that there is no unique second object in all the objects detected by the second sensor, set a corresponding data interval for each dimension of the first object again according to a predetermined interval gradient, so as to obtain a new data interval corresponding to each dimension of the first object;
the judging module is configured to re-execute the step of judging whether the only second object exists in all the objects detected by the second sensor.
Optionally, the setting module is configured to, when it is determined that the second object does not exist in all the objects detected by the second sensor each time, synchronously increase the length of the original data interval corresponding to each dimension of the first object according to a first gradient increase amount corresponding to each dimension, so as to obtain a new data interval corresponding to each dimension of the first object;
the judging module is configured to re-execute the step of judging whether the only second object exists in all the objects detected by the second sensor.
Optionally, the setting module is configured to, after the length of the original data interval corresponding to each dimension of the first object is synchronously increased according to a first gradient increase amount corresponding to each dimension, and when a plurality of pieces of data of each dimension are in the second object in the corresponding new data interval in all the objects detected by the second sensor, synchronously increase the length of the original data interval corresponding to each dimension of the first object according to a second gradient increase amount corresponding to each dimension, to obtain a new data interval corresponding to each dimension of the first object, where the second gradient increase amount corresponding to each dimension is smaller than the first gradient increase amount corresponding to each dimension;
the judging module is configured to re-execute the step of judging whether the only second object exists in all the objects detected by the second sensor.
Optionally, the setting module is configured to, when it is determined that a plurality of second objects exist in all the objects detected by the second sensor each time, synchronously reduce, according to a first gradient reduction amount corresponding to each dimension, a length of an original data interval corresponding to each dimension of the first object, to obtain a new data interval corresponding to each dimension of the first object;
the judging module is configured to re-execute the step of judging whether the only second object exists in all the objects detected by the second sensor.
Optionally, the apparatus includes:
the association module is configured to determine that, when the length of the data interval corresponding to each dimension of the first object is set to be the threshold of the length of the data interval corresponding to the dimension, and the second object whose data of each dimension is in the corresponding new data interval does not exist in all the objects detected by the second sensor, an object associated with the first object as the same object in the corresponding area does not exist in all the objects detected by the second sensor.
Optionally, the setting module is configured to use the data of each dimension of the first object as a middle value of a data interval of a corresponding dimension, use a product of the data of each dimension of the first object and a preset gradient percentage corresponding to the dimension as an interval length of the data interval of the corresponding dimension, and set the data interval.
Optionally, the data of multiple dimensions includes data of any of the following dimensions of the detected object: coordinate position data, velocity data, acceleration data, motion trend data.
In a third aspect, embodiments of the present disclosure provide a vehicle including any one of the multi-sensor data correlation devices described above.
According to the technical scheme, the data intervals are set according to the data of the multiple dimensions of the first object detected by the first sensor, and whether the second sensor has the only object of which each dimension data falls into the corresponding dimension data interval is judged. In this way, the accuracy of association of objects between different sensors can be improved through data association of multiple dimensions.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure without limiting the disclosure. In the drawings:
FIG. 1 is a schematic diagram of a scenario shown in accordance with an exemplary embodiment.
FIG. 2 is a flow chart illustrating a method of multi-sensor data correlation in an exemplary embodiment of the present disclosure.
FIG. 3 is a flow chart illustrating a method of multi-sensor data correlation in an exemplary embodiment of the present disclosure.
FIG. 4 is a block diagram of a multi-sensor data correlation device, shown in an exemplary embodiment of the present disclosure.
Detailed Description
The following detailed description of specific embodiments of the present disclosure is provided in connection with the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present disclosure, are given by way of illustration and explanation only, not limitation.
FIG. 1 is a schematic diagram of a scenario shown in accordance with an exemplary embodiment. As shown in fig. 1, objects in a fan-shaped area detectable by a sensor in front of the vehicle are detected by a first sensor and a second sensor, respectively. The two objects detected by the first sensor are T11 and T12, and the two objects detected by the second sensor are T21 and T22.
In the related art, object association is performed by calculating three-dimensional euclidean distances between objects detected by a plurality of sensors, that is, objects having the smallest three-dimensional euclidean distances detected by different sensors are associated with the same object in a corresponding detectable sector.
However, in an actual application scenario, the objects with the minimum euclidean distance in the three-dimensional space detected by different sensors do not necessarily correspond to the same object in the area. As can be seen from fig. 1, the euclidean distance D between T12 and T21 (T12-T21), smaller than the euclidean distance D between T11 and T21 (T11-T21), smaller than the euclidean distance D between T11 and T22 (T11-T22), smaller than the euclidean distance D between T12 and T22 (T12-T22), according to the object association method in the related art, objects T12 and T21 would be associated to correspond to the same object in the region, whereas in practice, objects T11 and T21 correspond to the same object in the region (both are dotted fills), and objects T12 and T22 correspond to the same object in the region (both are lattice fills). Associating the objects T12 and T21 as corresponding to the same object in the region, then in the subsequent data fusion process, the data corresponding to T12 and T21 detected by the two sensors respectively are fused, resulting in system misjudgment. If the object is an obstacle, such misjudgment may even cause a vehicle traffic accident.
In order to improve the accuracy of multi-sensor data association, an exemplary embodiment of the present disclosure provides a multi-sensor data association method, wherein the plurality of sensors are used for detecting data of multiple dimensions of an object in the same area. As shown in fig. 2, the method includes:
s21, according to the data of multiple dimensions of the first object detected by the first sensor, setting a corresponding data interval for each dimension of the first object, where the first object is any object detected by the first sensor.
Wherein the data of the plurality of dimensions comprises data of any of the following dimensions of the detected object: coordinate position data, velocity data, acceleration data, motion trend data. Differences between different objects can be more accurately represented by data information in multiple dimensions, for example, objects that are close in spatial distance at a certain time may have a large difference in acceleration values. Data of multiple dimensions are considered, different objects can be distinguished conveniently, and therefore matching accuracy of multiple sensor objects is improved.
Specifically, data of each dimension of the first object is set as a middle value of a data interval of a corresponding dimension; and setting the product of the data of each dimension of the first object and the preset gradient percentage corresponding to the dimension as the interval length of the data interval corresponding to the dimension.
Taking the speed dimension as an example, the data of the speed dimension of the first object is 80Km/h, 2% is taken as a gradient percentage, correspondingly, the middle value of the obtained data interval corresponding to the speed dimension is 80, the interval length is 1.6, and then the obtained data interval corresponding to the speed dimension is 79.2 Km/h-80.8 Km/h.
And S22, judging whether a unique second object exists in all the objects detected by the second sensor, wherein the data of each dimension of the second object are all in each data interval set corresponding to the first object.
Each object is detected by the second sensor, and data of multiple dimensions corresponding to the object is acquired. In a specific implementation, the detected data of the multiple dimensions of each object may be respectively compared with the data interval of each dimension corresponding to the first object, so as to find out whether there is a unique second object whose data of each dimension are all in the data intervals set for the first object.
S23, if there is a unique second object in all the objects detected by the second sensor, associating the first object and the second object as corresponding to the same object in the area.
Further, the method further comprises: fusing the data of the plurality of dimensions of the first object with the data of the plurality of dimensions of the second object, wherein the fused data result is the data of the plurality of dimensions of the same object.
Specifically, according to data of multiple dimensions of a first object, data of multiple dimensions of a second object, and weight values respectively corresponding to the dimensions of the first sensor and the second sensor, the data of multiple dimensions of the first object and the data of multiple dimensions of the second object are fused by a weighted average method.
For example, the X-direction dimension and the Y-direction dimension corresponding to the millimeter radar wave sensor are set, the weight value of the velocity V dimension is set to 1, and the weight value of the Y-direction dimension of the image sensor (camera) is set to 1. Then, in fact, in the multi-dimensional data corresponding to the entity object after fusion, the values of the X, Y, V dimensions are taken from the millimeter radar wave sensor, and the values of the Y-direction dimension are taken from the image sensor.
According to the technical scheme, the data intervals are set according to the data of the multiple dimensions of the first object detected by the first sensor, and whether the second sensor has the only object of which each dimension data falls into the corresponding dimension data interval is judged. In this way, the accuracy of association of objects between different sensors can be improved through data association of multiple dimensions.
In addition, when the data interval length corresponding to each dimension of the first object is set as the data interval length threshold of the dimension, if there is no second object whose data of each dimension is in the corresponding new data interval in all the objects detected by the second sensor, it is determined that there is no object associated with the first object as the same object in the corresponding area in all the objects detected by the second sensor.
The length threshold of the data interval corresponding to each dimension can be set according to the precision of different sensors. For example, if the error of the first sensor in the X-direction dimension is 1% and the error of the second sensor in the X-direction dimension is 2%, the error corresponding to the X-direction dimension may be set to 3%, that is, a difference of 3% between values of the objects corresponding to the same physical object detected by the two sensors in the X-direction dimension is tolerated. Accordingly, the threshold value of the interval length may be set to be 3% of the value of the first object in the X-direction dimension acquired by the first sensor, for example, if the value of the first object in the X-direction dimension is 100m, the threshold value of the data interval length corresponding to the X-direction dimension is 3 m.
Fig. 3 is a flowchart illustrating a multi-sensor data correlation method according to an exemplary embodiment of the present disclosure, as shown in fig. 3, the method including:
s31, according to the data of multiple dimensions of the first object detected by the first sensor, setting a corresponding data interval for each dimension of the first object, where the first object is any object detected by the first sensor.
Wherein the data of the plurality of dimensions comprises data of any of the following dimensions of the detected object: coordinate position data, velocity data, acceleration data, motion trend data.
Specifically, data of each dimension of the first object is set as a middle value of a data interval of a corresponding dimension; and setting the product of the data of each dimension of the first object and the preset gradient percentage corresponding to the dimension as the interval length of the data interval corresponding to the dimension.
Taking the speed dimension as an example, the data of the speed dimension of the first object is 80Km/h, 2% is taken as a gradient percentage, correspondingly, the middle value of the obtained data interval corresponding to the speed dimension is 80, the interval length is 1.6, and then the obtained data interval corresponding to the speed dimension is 79.2 Km/h-80.8 Km/h.
And S32, judging whether a unique second object exists in all the objects detected by the second sensor, wherein the data of each dimension of the second object are all in each data interval set corresponding to the first object.
It should be noted that the second sensor detects each object and obtains data corresponding to a plurality of dimensions of the object. In a specific implementation, the detected data of the multiple dimensions of each object may be respectively compared with the data interval of each dimension corresponding to the first object, so as to find out whether there is a unique second object whose data of each dimension are all in the data intervals set for the first object.
If it is determined that the second object does not exist in all the objects detected by the second sensor each time, step S33 is performed.
And S33, synchronously enlarging the length of the original data interval corresponding to each dimension of the first object according to the first gradient increment corresponding to each dimension to obtain a new data interval corresponding to each dimension of the first object.
And S34, judging whether a unique second object exists in all the objects detected by the second sensor, wherein the data of each dimension of the second object are all in each data interval set corresponding to the first object.
Further, if there are a plurality of second objects in the new data interval among all the objects detected by the second sensor, step S34 is executed.
S35, according to the second gradient increment corresponding to each dimension, synchronously expanding the length of the original data interval corresponding to each dimension of the first object to obtain a new data interval corresponding to each dimension of the first object, wherein the second gradient increment corresponding to each dimension is smaller than the first gradient increment corresponding to each dimension.
Illustratively, the data in the X-direction dimension is 100m, the initial interval length is 10m, and the initial data interval corresponding to the X-direction dimension is 95m to 105 m. The data in the velocity dimension is 100Km/h, the initial interval length is 10Km/h, and the data interval in the initial velocity dimension is 95 Km/h-105 Km/h.
And synchronously expanding the length of the original data interval corresponding to the X-direction dimension of the first object by a first gradient increment amount of 5m (the upper and lower boundaries of the data interval are both expanded by 5m), so as to obtain the expanded data interval corresponding to the X-direction dimension of 90-110 m. And synchronously expanding the length of the original data interval corresponding to the speed dimension of the first object by a first gradient increment of 5Km/h (the upper and lower boundaries of the data interval are both expanded by 5Km/h) corresponding to the speed dimension, and obtaining the expanded data interval corresponding to the speed dimension of 90 Km/h-110 Km/h.
If the length of the data interval is once enlarged, the second object, in which the data of each dimension is not in the corresponding new data interval, among all the objects detected by the second sensor, changes to the presence of a plurality of second objects, for example, the object a detected by the second sensor is present, the X-direction dimension data is 92m, and the speed-direction dimension data is 92 Km/h; for the object B, the X-direction dimension data is 94m, and the speed direction dimension data is 105 Km/h. The increment of the adjustment interval is reduced, and the interval length is re-expanded on the basis of the original data interval of each dimension.
For example, based on the initial interval length, the length of the original data interval corresponding to the X-direction dimension of the first object is synchronously enlarged by a second gradient increment 1m (the upper and lower boundaries of the data interval are enlarged by 1m), and the enlarged data interval corresponding to the X-direction dimension is 94-106 m. And synchronously expanding the length of the original data interval corresponding to the speed dimension of the first object by a second gradient increment of 1Km/h (the upper and lower boundaries of the data interval are expanded by 1Km/h) corresponding to the speed dimension to obtain an expanded data interval corresponding to the speed dimension of 94 Km/h-106 Km/h.
According to the above example, after the interval length is extended again by the smaller interval increase amount, the object matched with the first object obtained by screening is the object b.
That is, if the result that a plurality of second objects exist is obtained after the original data interval length of each dimension is expanded by a larger gradient increment, the original data interval length of each dimension is expanded by a smaller gradient increment, so that the gradient increment of each time is refined, and the unique facility second object is screened out. In addition, the number of objects to be compared can be gradually reduced, and the real-time performance of multi-sensor object matching is improved.
And S36, judging whether a unique second object exists in all the objects detected by the second sensor, wherein the data of each dimension of the second object are all in each data interval set corresponding to the first object.
Further, if there is a unique second object in all the objects detected by the second sensor, step S37 is executed.
S37, associating the first object and the second object to correspond to the same object in the area.
Further, the method further comprises: fusing the data of the plurality of dimensions of the first object with the data of the plurality of dimensions of the second object, wherein the fused data result is the data of the plurality of dimensions of the same object.
Specifically, according to data of multiple dimensions of a first object, data of multiple dimensions of a second object, and weight values respectively corresponding to the dimensions of the first sensor and the second sensor, the data of multiple dimensions of the first object and the data of multiple dimensions of the second object are fused by a weighted average method.
For example, the X-direction dimension and the Y-direction dimension corresponding to the millimeter radar wave sensor are set, the weight value of the velocity V dimension is set to 1, and the weight value of the Y-direction dimension of the image sensor (camera) is set to 1. Then, in fact, in the multi-dimensional data corresponding to the entity object after fusion, the values of the X, Y, V dimensions are taken from the millimeter radar wave sensor, and the values of the Y-direction dimension are taken from the image sensor.
According to the technical scheme, data intervals are set according to data of multiple dimensions of the first object detected by the first sensor, when no unique object with all dimension data falling into the corresponding dimension data interval exists, a new data interval corresponding to each dimension of the first object is readjusted, and the step of judging whether the unique second object exists in all the objects detected by the second sensor is executed again. In this way, when the dimension data of the plurality of objects detected by the second sensor are relatively close to each other, the unique object matched with the dimension data of the first object detected by the first sensor can be screened out, and therefore the association precision of the objects among different sensors can be improved.
In another optional embodiment, the method comprises: and when the second sensor detects that the unique second object does not exist in all the objects detected by the second sensor, setting a corresponding data interval for each dimension of the first object again according to a preset interval gradient to obtain a new data interval corresponding to each dimension of the first object, and re-executing the step of judging whether the unique second object exists in all the objects detected by the second sensor.
Wherein the case where there is no unique second object includes the presence of a plurality of the second objects or the absence of the second objects.
In an example, the extent of the interval length increase may be different each time the interval is reset. For example, the data interval corresponding to each dimension of the first object is gradually enlarged by an increment of 2%, 4%, or 6%, respectively. That is, when the data interval is short, a small increment is selected, and the data interval corresponding to each dimension can be adjusted more flexibly according to an actual numerical value, so that the association range is finer.
In another example, each time it is determined that a plurality of second objects exist in all the objects detected by the second sensor, the length of the original data interval corresponding to each dimension of the first object is synchronously reduced according to the first gradient reduction amount corresponding to each dimension, so as to obtain a new data interval corresponding to each dimension of the first object, and the step of determining whether a unique second object exists in all the objects detected by the second sensor is performed again.
In a specific implementation, a data interval with a larger interval length corresponding to each dimension may be generated according to data of each dimension of the first object, and during the first comparison, a plurality of objects may exist in the object detected by the second sensor, and each dimension data of the plurality of objects is located in the data interval corresponding to the dimension. Further, the data interval corresponding to each dimension is gradually reduced at the same time until there is a unique second object whose data of each dimension are all in the data intervals set for the first object.
In addition, when the data interval length corresponding to each dimension of the first object is set as the data interval length threshold of the dimension, if there is no second object whose data of each dimension is in the corresponding new data interval in all the objects detected by the second sensor, it is determined that there is no object associated with the first object as the same object in the corresponding area in all the objects detected by the second sensor.
The length threshold of the data interval corresponding to each dimension can be set according to the precision of different sensors. For example, if the error of the first sensor in the X-direction dimension is 1% and the error of the second sensor in the X-direction dimension is 2%, the error corresponding to the X-direction dimension may be set to 3%, that is, a difference of 3% between values of the objects corresponding to the same physical object detected by the two sensors in the X-direction dimension is tolerated. Accordingly, the threshold value of the interval length may be set to be 3% of the value of the first object in the X-direction dimension acquired by the first sensor, for example, if the value of the first object in the X-direction dimension is 100m, the threshold value of the data interval length corresponding to the X-direction dimension is 3 m.
In addition, different search ranges can be set corresponding to different dimensions according to the performance of the sensor. In the specific implementation, different data interval lengths can be set according to different dimensions, and different data interval increasing amounts are set, so that objects collected by a plurality of sensors can be matched more flexibly.
FIG. 4 is a block diagram of a multi-sensor data correlation device, shown in an exemplary embodiment of the present disclosure. The plurality of sensors are for detecting data in multiple dimensions of an object in the same region, the apparatus comprising:
a setting module 410, configured to set a corresponding data interval for each dimension of a first object according to data of multiple dimensions of the first object detected by a first sensor, where the first object is any object detected by the first sensor;
a determining module 420, configured to determine whether a unique second object exists in all objects detected by the second sensor, where data of each dimension of the second object is in each data interval set for the first object;
an associating module 430, configured to associate the first object and the second object as corresponding to a same object in the area when a unique second object exists among all objects detected by the second sensor.
Optionally, the setting module 410 is configured to, each time it is determined that there is no unique second object in all the objects detected by the second sensor, re-set a corresponding data interval for each dimension of the first object according to a predetermined interval gradient, so as to obtain a new data interval corresponding to each dimension of the first object;
the determining module 420 is configured to re-execute the step of determining whether the second object is unique among all the objects detected by the second sensor.
Wherein the case where there is no unique second object includes the presence of a plurality of the second objects or the absence of the second objects.
Optionally, the setting module 410 is configured to, when it is determined that the second object does not exist in all the objects detected by the second sensor each time, synchronously increase the length of the original data interval corresponding to each dimension of the first object according to the first gradient increment corresponding to each dimension, so as to obtain a new data interval corresponding to each dimension of the first object;
the determining module 420 is configured to re-execute the step of determining whether the second object is unique among all the objects detected by the second sensor.
The setting module 410 is configured to, after the length of the original data interval corresponding to each dimension of the first object is synchronously extended according to a first gradient increment corresponding to each dimension, synchronously extend the length of the original data interval corresponding to each dimension of the first object according to a second gradient increment corresponding to each dimension when a plurality of pieces of data of each dimension are in the second object corresponding to the new data interval in all the objects detected by the second sensor, and obtain a new data interval corresponding to each dimension of the first object, where the second gradient increment corresponding to each dimension is smaller than the first gradient increment corresponding to each dimension;
the determining module 420 is configured to re-execute the step of determining whether the second object is unique among all the objects detected by the second sensor.
Optionally, the setting module 410 is configured to, when it is determined that a plurality of second objects exist in all the objects detected by the second sensor each time, synchronously reduce, according to a first gradient reduction amount corresponding to each dimension, a length of an original data interval corresponding to each dimension of the first object, to obtain a new data interval corresponding to each dimension of the first object;
the determining module 420 is configured to re-execute the step of determining whether the second object is unique among all the objects detected by the second sensor.
Optionally, the apparatus includes:
the associating module 430 is configured to determine that there is no object associated with the first object as the same object in the area when the length of the data interval corresponding to each dimension of the first object is set to the threshold of the length of the data interval corresponding to the dimension, and there is no second object whose data of each dimension is in the corresponding new data interval in all the objects detected by the second sensor.
Optionally, the setting module 410 is configured to use the data of each dimension of the first object as a middle value of a data interval of a corresponding dimension, use a product of the data of each dimension of the first object and a preset gradient percentage of the corresponding dimension as an interval length of the data interval of the corresponding dimension, and set the data interval.
Optionally, the data of multiple dimensions includes data of any of the following dimensions of the detected object: coordinate position data, velocity data, acceleration data, motion trend data.
According to the technical scheme, the data intervals are set according to the data of the multiple dimensions of the first object detected by the first sensor, and whether the second sensor has the only object of which each dimension data falls into the corresponding dimension data interval is judged. In this way, the accuracy of association of objects between different sensors can be improved through data association of multiple dimensions.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
The disclosed embodiment provides a vehicle including any one of the above-described multi-sensor data correlation devices.
The preferred embodiments of the present disclosure are described in detail with reference to the accompanying drawings, however, the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solution of the present disclosure within the technical idea of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.
It should be noted that, in the foregoing embodiments, various features described in the above embodiments may be combined in any suitable manner, and in order to avoid unnecessary repetition, various combinations that are possible in the present disclosure are not described again.
In addition, any combination of various embodiments of the present disclosure may be made, and the same should be considered as the disclosure of the present disclosure, as long as it does not depart from the spirit of the present disclosure.
Claims (9)
1. A multi-sensor data association method, wherein a plurality of sensors are used to detect data of a plurality of dimensions of an object in the same area, the method comprising:
setting a corresponding data interval for each dimension of a first object according to data of multiple dimensions of the first object detected by a first sensor, wherein the first object is any one object detected by the first sensor;
judging whether a unique second object exists in all the objects detected by the second sensor, wherein the data of each dimension of the second object are in each data interval set for the first object;
if the second object is unique in all the objects detected by the second sensor, associating the first object and the second object as corresponding to the same object in the area;
and when the second sensor detects that the unique second object does not exist in all the objects detected by the second sensor, setting a corresponding data interval for each dimension of the first object again according to a preset interval gradient to obtain a new data interval corresponding to each dimension of the first object, and re-executing the step of judging whether the unique second object exists in all the objects detected by the second sensor.
2. The method according to claim 1, characterized in that it comprises:
and synchronously expanding the length of the original data interval corresponding to each dimension of the first object according to the first gradient increment corresponding to each dimension when the second object does not exist in all the objects detected by the second sensor every time, obtaining a new data interval corresponding to each dimension of the first object, and re-executing the step of judging whether the unique second object exists in all the objects detected by the second sensor.
3. The method of claim 2, wherein the method comprises:
if the length of the original data interval corresponding to each dimension of the first object is synchronously enlarged according to the first gradient increment corresponding to each dimension, when a plurality of second objects with data of each dimension in the corresponding new data interval exist in all the objects detected by the second sensor, the length of the original data interval corresponding to each dimension of the first object is synchronously enlarged according to the second gradient increment corresponding to each dimension to obtain a new data interval corresponding to each dimension of the first object, wherein the second gradient increment corresponding to each dimension is smaller than the first gradient increment corresponding to each dimension, and the step of judging whether the unique second object exists in all the objects detected by the second sensor is executed again.
4. The method according to claim 1, characterized in that it comprises:
and synchronously reducing the length of the original data interval corresponding to each dimension of the first object according to the first gradient reduction amount corresponding to each dimension when a plurality of second objects exist in all the objects detected by the second sensor every time, so as to obtain a new data interval corresponding to each dimension of the first object, and re-executing the step of judging whether the unique second objects exist in all the objects detected by the second sensor.
5. The method according to any one of claims 1-4, characterized in that the method comprises:
when the length of the data interval corresponding to each dimension of the first object is set as the threshold value of the length of the data interval corresponding to the dimension, if the second object of which the data of each dimension is in the corresponding new data interval does not exist in all the objects detected by the second sensor, it is determined that the object associated with the first object as the same object in the corresponding area does not exist in all the objects detected by the second sensor.
6. The method according to any one of claims 1-4, wherein setting a corresponding data interval for each dimension of the first object based on the data of the plurality of dimensions of the first object detected by the first sensor comprises:
and taking the data of each dimension of the first object as a middle value of a data interval of the corresponding dimension, taking the product of the data of each dimension of the first object and a preset gradient percentage corresponding to the dimension as an interval length of the data interval of the corresponding dimension, and setting the data interval.
7. The method of any of claims 1-4, wherein the data in the plurality of dimensions comprises data in any of the following dimensions of the detected object: coordinate position data, velocity data, acceleration data, motion trend data.
8. A multi-sensor data association apparatus, wherein a plurality of sensors are used to detect data of a plurality of dimensions of an object in the same area, the apparatus comprising:
the device comprises a setting module, a data processing module and a display module, wherein the setting module is used for setting a corresponding data interval for each dimension of a first object according to data of multiple dimensions of the first object detected by a first sensor, and the first object is any object detected by the first sensor;
the judging module is used for judging whether a unique second object exists in all the objects detected by the second sensor, and data of each dimension of the second object are all in each data interval correspondingly set for the first object;
a correlation module, configured to correlate the first object and the second object to correspond to a same object in the area when a unique second object exists among all objects detected by the second sensor;
the setting module is further configured to, when it is determined that there is no unique second object in all the objects detected by the second sensor, set a corresponding data interval for each dimension of the first object again according to a predetermined interval gradient, so as to obtain a new data interval corresponding to each dimension of the first object;
the judging module is further configured to re-execute the step of judging whether a unique second object exists in all objects detected by the second sensor.
9. A vehicle characterized in that it comprises a multi-sensor data correlation device according to claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811109761.5A CN109270523B (en) | 2018-09-21 | 2018-09-21 | Multi-sensor data fusion method and device and vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811109761.5A CN109270523B (en) | 2018-09-21 | 2018-09-21 | Multi-sensor data fusion method and device and vehicle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109270523A CN109270523A (en) | 2019-01-25 |
| CN109270523B true CN109270523B (en) | 2020-09-18 |
Family
ID=65198241
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811109761.5A Expired - Fee Related CN109270523B (en) | 2018-09-21 | 2018-09-21 | Multi-sensor data fusion method and device and vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109270523B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110108282B (en) * | 2019-05-09 | 2023-11-07 | 仲恺农业工程学院 | Multi-source information obstacle avoidance device and obstacle avoidance system |
| CN112799079B (en) * | 2019-10-24 | 2024-03-26 | 华为技术有限公司 | Data association method and device |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7796081B2 (en) * | 1997-10-22 | 2010-09-14 | Intelligent Technologies International, Inc. | Combined imaging and distance monitoring for vehicular applications |
| US7460951B2 (en) * | 2005-09-26 | 2008-12-02 | Gm Global Technology Operations, Inc. | System and method of target tracking using sensor fusion |
| JP6036724B2 (en) * | 2014-02-17 | 2016-11-30 | トヨタ自動車株式会社 | Vehicle surrounding situation recognition device and vehicle control device |
| CN105160356B (en) * | 2015-08-31 | 2019-07-19 | 上汽通用汽车有限公司 | A kind of active safety systems of vehicles Data Fusion of Sensor method and system |
| US10565468B2 (en) * | 2016-01-19 | 2020-02-18 | Aptiv Technologies Limited | Object tracking system with radar/vision fusion for automated vehicles |
| CN106842188B (en) * | 2016-12-27 | 2018-01-09 | 上海思致汽车工程技术有限公司 | A kind of object detection fusing device and method based on multisensor |
-
2018
- 2018-09-21 CN CN201811109761.5A patent/CN109270523B/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| CN109270523A (en) | 2019-01-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109212521B (en) | Target tracking method based on fusion of forward-looking camera and millimeter wave radar | |
| CN104965202B (en) | Obstacle detection method and device | |
| KR101992548B1 (en) | Method for generating a surroundings map, and driver assistance system | |
| JP2017220157A (en) | Object recognition processor, object recognition processing method and automatic driving system | |
| JP2004508627A (en) | Route prediction system and method | |
| CN112285714A (en) | Obstacle speed fusion method and device based on multiple sensors | |
| KR101735557B1 (en) | System and Method for Collecting Traffic Information Using Real time Object Detection | |
| KR20170080480A (en) | The vehicle detecting system by converging radar and image | |
| CN109270523B (en) | Multi-sensor data fusion method and device and vehicle | |
| JP6311211B2 (en) | Obstacle detection device | |
| JP2017207348A (en) | Radar device and sensor fusion device using the same | |
| CN112116031A (en) | Target fusion method and system based on road side equipment, vehicle and storage medium | |
| KR20200108464A (en) | Method and apparatus for detecting critical transverse movement | |
| CN111007492A (en) | Detection system and method | |
| US20210070311A1 (en) | Method and apparatus for multi vehicle sensor suite diagnosis | |
| CN115151836A (en) | Method for detecting a moving object in the surroundings of a vehicle and motor vehicle | |
| JP5509615B2 (en) | Vehicle detection device, vehicle detection method, and vehicle detection program | |
| US11474535B2 (en) | Method of processing data, apparatus for processing data, and system for controlling vehicle | |
| EP3467545A1 (en) | Object classification | |
| JP6555132B2 (en) | Moving object detection device | |
| JP6834020B2 (en) | Object recognition device and object recognition method | |
| JP4644590B2 (en) | Peripheral vehicle position detection device and peripheral vehicle position detection method | |
| KR102363924B1 (en) | Distance measuring device and method for nearing vehicle using image | |
| KR102402088B1 (en) | Device for detecting a moving object using a plurality of radar and method thereof | |
| CN111596288B (en) | Method and device for measuring speed, vehicle-mounted terminal and vehicle-mounted speed measuring system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Multi sensor data fusion method and device, vehicle Effective date of registration: 20211213 Granted publication date: 20200918 Pledgee: BEIJING AUTOMOTIVE GROUP Co.,Ltd. Pledgor: Borgward Automotive (China) Co., Ltd. Registration number: Y2021990001167 |
|
| PC01 | Cancellation of the registration of the contract for pledge of patent right | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right |
Date of cancellation: 20231201 Granted publication date: 20200918 Pledgee: BEIJING AUTOMOTIVE GROUP Co.,Ltd. Pledgor: Beijing baowo Automobile Co.,Ltd.|Borgward Automotive (China) Co., Ltd. Registration number: Y2021990001167 |
|
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200918 |