Nothing Special   »   [go: up one dir, main page]

WO2023036032A1 - 一种车道线检测方法及装置 - Google Patents

一种车道线检测方法及装置 Download PDF

Info

Publication number
WO2023036032A1
WO2023036032A1 PCT/CN2022/116161 CN2022116161W WO2023036032A1 WO 2023036032 A1 WO2023036032 A1 WO 2023036032A1 CN 2022116161 W CN2022116161 W CN 2022116161W WO 2023036032 A1 WO2023036032 A1 WO 2023036032A1
Authority
WO
WIPO (PCT)
Prior art keywords
lane line
lane
feature map
image
feature point
Prior art date
Application number
PCT/CN2022/116161
Other languages
English (en)
French (fr)
Inventor
鲁恒宇
苏鹏
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2023036032A1 publication Critical patent/WO2023036032A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Definitions

  • the embodiments of the present application relate to the technical field of automatic driving, and in particular to a lane line detection method and device.
  • Lane line detection is an important task of the Advanced Driver Assistance System (ADAS), and is the key to realize adaptive cruise control (Adaptive Cruise Control, ACC), lane departure warning system (Lane Departure Warning System, LDWS), etc. technology.
  • ADAS Advanced Driver Assistance System
  • ACC Adaptive Cruise Control
  • LDWS Lane Departure Warning System
  • Lane lines as a major part of the road, play a role in providing reference for unmanned vehicles and guiding safe driving.
  • lane line detection can also be used to implement road positioning, determine the relative position between the vehicle and the road, and assist the vehicle's decision-making planning.
  • Embodiments of the present application provide a lane marking detection method and device, which help to improve lane marking detection efficiency.
  • the embodiment of the present application provides a lane line detection method, which can be used in a lane line detection device, and the lane line detection device can be deployed on the vehicle side or the server side, and can be an independent device or in a device
  • the chips or components may also be software modules.
  • the embodiment of the present application does not limit the deployment method or product form of the lane marking detection device.
  • the method includes: acquiring a feature map of the first image; determining a target feature point in the feature map; determining a first topological relationship according to the target feature point, and the target feature point is associated with a change in the first topological relationship position, the first topological relationship is used to indicate the relationship between lane lines in the first image.
  • the complex lane line detection scene can be converted into a simple scene to determine the relationship between the lane lines in the first image.
  • the determining the target feature point in the feature map includes: calculating the confidence that each feature point in the feature map is the target feature point; The confidence level is to determine the target feature point in the feature map.
  • target feature points can be determined from multiple feature points in the feature map according to the target detection algorithm and confidence.
  • the determining the first topological relationship according to the target feature point includes: according to the position of the target feature point in the feature map, Slicing is performed to obtain at least two feature map slices; and the first topological relationship is determined according to encoding of lane lines in the at least two feature slices.
  • the feature map is divided into at least two feature map slices according to the target feature points, so as to respectively detect lane lines in the at least two feature map slices.
  • the method further includes: adjusting the coding of the lane line where the target feature point is located and/or the adjacent lane line according to the position associated with the target feature point.
  • the lane lines in the image sequence or at least two feature map slices belonging to the same image are encoded and matched, reducing the parameters introduced by the algorithm and helping to increase the robustness of the lane line detection algorithm. Stickiness.
  • the target feature point is associated with any of the following positions: a lane line stop position, a lane line bifurcation position, or a lane line merge position.
  • the position points that affect the transformation of the lane topological relationship can be predefined according to the transformation relationship of the lane topological relationship. It should be understood that this is only an illustration of several possible positions without any limitation. In other embodiments, there may be other positions, which will not be repeated here.
  • the first image belongs to a group of image sequences
  • the method further includes: determining the first image according to the coding of the lane lines in the multiple images in the image sequence Two topological relationships, the second topological relationship is used to indicate the association relationship between lane lines in the image sequence.
  • the lane line detection device can determine the topological relationship between the lane lines in different images according to a set of image sequences, and improve the detection efficiency of the lane line topological relationship.
  • the method further includes: determining a similarity matrix according to the feature map, where the similarity matrix is used to indicate a global association relationship of each feature point in the feature map.
  • the lane line detection device can learn the global topological relationship between each feature point in the feature map of a frame image, so as to enhance the association relationship between each feature point.
  • an embodiment of the present application provides a lane line detection device, including: an acquiring unit, configured to acquire a feature map of a first image; a first determining unit, configured to determine a target feature point in the feature map; A second determining unit, configured to determine a first topological relationship according to the target feature point, the target feature point is associated with a position where the first topological relationship changes, and the first topological relationship is used to indicate the first image The association relationship between the lane lines in .
  • the first determining unit is configured to: calculate a confidence degree that each feature point in the feature map is the target feature point; according to the confidence degree, in the The target feature point is determined in the feature map.
  • the second determination unit is configured to: segment the feature map according to the position of the target feature point in the feature map to obtain at least two feature map slices; according to the encoding of lane lines in the at least two feature slices, determine the first topological relationship.
  • the device further includes: an adjustment unit, configured to adjust the lane line where the target feature point is located and/or the adjacent lane according to the position associated with the target feature point Encoding of the line.
  • the target feature point is associated with any of the following positions: a lane line stop position, a lane line bifurcation position, or a lane line merge position.
  • the first image belongs to a group of image sequences
  • the device further includes: a third determination unit, configured to The encoding of the lane lines determines the second topological relationship, and the second topological relationship is used to indicate the association relationship between the lane lines in the image sequence.
  • the apparatus further includes: a fourth determining unit, configured to determine a similarity matrix according to the feature map, and the similarity matrix is used to indicate that each feature in the feature map Global associations of points.
  • a fourth determining unit configured to determine a similarity matrix according to the feature map, and the similarity matrix is used to indicate that each feature in the feature map Global associations of points.
  • an embodiment of the present application provides a lane line detection device, including: a processor and a memory; the memory is used to store programs; the processor is used to execute the programs stored in the memory, so that the The device implements the method described in the first aspect and any possible design of the first aspect.
  • an embodiment of the present application provides a computer-readable storage medium, wherein program code is stored in the computer-readable storage medium, and when the program code is run on a computer, the computer executes the above-mentioned first aspect and The method described in the first aspect is a possible design.
  • the embodiment of the present application provides a computer program product, which, when the computer program product is run on a computer, enables the computer to execute the method described in the first aspect and possible design of the first aspect.
  • an embodiment of the present application provides a chip system, the chip system includes a processor, configured to call a computer program or a computer instruction stored in a memory, so that the processor performs the above-mentioned first aspect and the possibility of the first aspect The method described for the design.
  • the processor may be coupled to the memory through an interface.
  • the chip system may further include a memory, where computer programs or computer instructions are stored.
  • the embodiment of the present application provides a processor, the processor is used to call the computer program or computer instruction stored in the memory, so that the processor executes the above-mentioned first aspect and the possible design of the first aspect. method.
  • Fig. 1 is an example of a lane line detection method
  • FIG. 2 shows a schematic diagram of an application scenario applicable to an embodiment of the present application
  • FIG. 3 shows a schematic diagram of a vehicle perception system according to an embodiment of the present application
  • FIG. 4 shows a schematic diagram of the principle of a lane line detection device according to an embodiment of the present application
  • Fig. 5a-Fig. 5c show the schematic diagram of the position associated with the target feature point according to the embodiment of the present application
  • FIG. 6 shows a schematic diagram of the principle of the target detection module of the embodiment of the present application.
  • Fig. 7 shows the schematic diagram of the principle of the feature segmentation module of the embodiment of the present application.
  • FIG. 8 shows a schematic diagram of the principle of the lane line detection module of the embodiment of the present application.
  • FIG. 9 shows a schematic flowchart of a lane line detection method according to an embodiment of the present application.
  • Fig. 10a-Fig. 10c show the schematic diagram of the lane coding of the embodiment of the present application
  • FIG. 11 shows a schematic diagram of global relationship detection in an embodiment of the present application.
  • Figure 12a- Figure 12b shows a schematic diagram of the display mode of the embodiment of the present application
  • FIG. 13 shows a schematic diagram of a lane line detection method according to an embodiment of the present application.
  • FIG. 14 shows a schematic diagram of a lane line detection method according to an embodiment of the present application.
  • Embodiments of the present application provide a lane line detection method and device, which determine a first topological relationship by identifying target feature points in a feature map of a first image, which helps to improve lane line detection efficiency.
  • the method and the device are based on the same technical conception. Since the principle of solving the problem of the method and the device is similar, the implementation of the device and the method can be referred to each other, and the repetition will not be repeated.
  • the lane line detection solution in the embodiment of the present application can be applied to the Internet of Vehicles, such as vehicle-to-everything (V2X), long-term evolution-vehicle (LTE-V), Vehicle-vehicle (vehicle to vehicle, V2V), etc.
  • V2X vehicle-to-everything
  • LTE-V long-term evolution-vehicle
  • V2V Vehicle-vehicle to vehicle
  • the other devices include but are not limited to: other sensors such as vehicle-mounted terminals, vehicle-mounted controllers, vehicle-mounted modules, vehicle-mounted modules, vehicle-mounted components, vehicle-mounted chips, vehicle-mounted units, vehicle-mounted radars, or vehicle-mounted cameras.
  • Vehicles can implement the implementation of this application through these other devices.
  • the lane line detection method provided by the example.
  • the lane line detection solution in the embodiment of the present application can also be used in smart terminals with mobile control functions other than vehicles, or set in smart terminals with mobile control functions other than vehicles, or set in the components of the smart terminal.
  • the smart terminal may be a smart transportation device, a smart home device, a robot, and the like.
  • it includes but is not limited to smart terminals or controllers, chips, radars or cameras and other sensors in the smart terminals, and other components.
  • At least one refers to one or more, and “multiple” refers to two or more.
  • And/or describes the association relationship of associated objects, indicating that there may be three types of relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A, B can be singular or plural.
  • the character “/” generally indicates that the contextual objects are an “or” relationship.
  • At least one of the following” or similar expressions refer to any combination of these items, including any combination of single or plural items.
  • At least one item (piece) of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.
  • ordinal numerals such as “first” and “second” mentioned in the embodiments of the present application are used to distinguish multiple objects, and are not used to limit the priority or importance of multiple objects.
  • first topological relationship and the second topological relationship are only for distinguishing different topological relationships, rather than indicating the difference in priority or importance of the two topological relationships.
  • Fig. 2 shows a schematic diagram of an application scenario to which the embodiment of the present application is applicable.
  • the application scenario may include a vehicle and a server
  • the server may be a cloud
  • the cloud may include a cloud server and/or a cloud virtual machine.
  • the server can communicate with the vehicle to provide various services for the vehicle, such as over the air (OTA) service, high-precision map service, automatic driving or assisted driving service, etc.
  • OTA over the air
  • high-precision map service high-precision map service
  • automatic driving or assisted driving service etc.
  • Vehicles can download high-precision map data from the cloud to obtain high-precision maps, providing users with more accurate navigation services.
  • Road information updates are very frequent.
  • This service can not only update road information to the map in a more timely manner, but also reduce the local storage space requirements of the vehicle. For example, for a large city or region, the entire set of high-precision maps has a large amount of data.
  • the high-precision map service provided by the cloud allows the vehicle to obtain a high-precision map of a small area of the current location in real time while driving, and the high-precision map of the area The fine map can be released from the vehicle when not needed.
  • Vehicles can interact with the cloud to improve automatic driving or assisted driving functions, thereby improving vehicle safety and travel efficiency.
  • the vehicle can collect road surface information and surrounding vehicle information through the sensor device installed on the vehicle body, and upload the collected information to the cloud.
  • the update continuously optimizes the driving algorithm and updates it to the vehicle, so that the vehicle's automatic driving ability to cope with various scenarios is continuously improved.
  • the training of the image processing algorithm can be completed in the cloud and updated as the training data is updated; correspondingly, the vehicle can obtain updated images from the cloud Processing algorithm, so as to improve the image processing capability of the sensing device.
  • vehicles can obtain weather information and road traffic accident information through the cloud, so as to assist vehicles in planning, improve travel efficiency, and reduce the risk of vehicle accidents.
  • the cloud can send real-time road information to the vehicle, such as traffic light information.
  • the vehicle can receive the traffic light change interval time at the intersection ahead, and calculate the time it takes for the vehicle to pass according to the current speed, so as to judge the appropriate and safe road.
  • timing and planning the driving speed of the vehicle it can not only reduce the energy consumption of the vehicle, but also increase the safety of driving.
  • the vehicle can obtain third-party services through the cloud.
  • the courier can open the trunk of the vehicle through a one-time digital authorization and place items in the vehicle, so as to realize the situation where the driver is not present. Next to receive courier.
  • the vehicle can exchange information with the cloud through wireless communication.
  • the wireless communication can follow the wireless protocol of the network connected to the vehicle, such as the V2X (C-V2X) communication of the cellular network.
  • the cellular network is, for example, a long term evolution (long term evolution, LTE) wireless network or fifth generation (5th generation, 5G) wireless network, etc.
  • This application scenario can also include a roadside unit (RSU), which can be installed on the roadside and can communicate with the cloud and the vehicle.
  • the roadside unit communicating with the cloud can be regarded as a terminal device similar to the vehicle.
  • the roadside unit that communicates with the vehicle can be regarded as a terminal device similar to the vehicle, and can also be regarded as the service end device of the vehicle.
  • the roadside unit can use wireless communication to interact with the vehicle or the cloud, and the communication with the vehicle can use dedicated short range communication (DSRC) technology, or V2X (C-V2X) communication based on cellular network. For example, based on LTE communication protocol or based on 5G communication protocol.
  • DSRC dedicated short range communication
  • C-V2X V2X
  • the communication with the cloud may use cellular network-based V2X (C-V2X) communication, for example, based on an LTE communication protocol or a 5G communication protocol.
  • Roadside units can provide services for vehicles, such as vehicle identification, electronic toll collection, and electronic point deduction.
  • Roadside units can be equipped with sensing devices to collect road information and provide vehicle-road coordination services.
  • the roadside unit can be connected to roadside traffic signs (for example, electronic traffic lights, or electronic speed limit signs, etc.) to realize real-time control of traffic lights or speed limit signs, or can provide road information to vehicles through the cloud or directly to Improve automatic driving or assisted driving functions.
  • lane line detection is an important task of the Advanced Driver Assistance System (ADAS), which is to realize adaptive cruise control (Adaptive Cruise Control, ACC), lane departure warning system (Lane Departure Warning System, LDWS) and other key technologies.
  • ADAS Advanced Driver Assistance System
  • ACC Adaptive Cruise Control
  • LDWS Lane Departure Warning System
  • lane line detection is a complex and challenging topic.
  • Lane lines as a major part of the road, play a role in providing reference for unmanned vehicles and guiding safe driving.
  • lane line detection can also be used to implement road positioning, determine the relative position between the vehicle and the road, and assist the vehicle's decision-making planning.
  • a variety of sensors can be installed on the vehicle, such as one or more of the camera, laser radar, millimeter wave radar, ultrasonic sensor, etc., to obtain the environment around the vehicle through the sensor Information, and analyze and process the acquired information to realize functions such as obstacle perception, target recognition, vehicle positioning, path planning, driver monitoring/reminder, etc., thereby improving the safety, automation and comfort of vehicle driving.
  • the vehicle conducts a comprehensive analysis based on the perception information obtained by various sensors, and can also determine which lane the vehicle is in on the current road, the topological relationship between the lane lines on the road, etc., so as to improve the vehicle's automatic Driving or assisted driving functions.
  • LiDAR is the abbreviation of LightLaser Detection and Ranging (LiDAR) system, which mainly includes a transmitter, a receiver and a signal processing unit.
  • the transmitter is the laser emitting mechanism in the LiDAR; After arriving at the target object, reflected by the target object, the reflected light will converge to the receiver through the lens group.
  • the signal processing unit is responsible for controlling the emission of the transmitter, processing the signal received by the receiver, and calculating information such as the position, speed, distance, and/or size of the target object.
  • Millimeter-wave radar uses millimeter-wave as the detection medium, which can measure the distance, angle and relative speed between the millimeter-wave radar and the measured object.
  • Millimeter wave radar can be divided into long-range radar (LRR), mid-range radar (MRR) and short-range radar (Short Range Radar, SRR) according to its detection distance.
  • LRR long-range radar
  • MRR mid-range radar
  • SRR Short Range Radar
  • the main application scenarios for LRR include active cruise and brake assist, etc.
  • LRR does not have high requirements for the angular width of the detection, and the reflection on the antenna is that the 3dB beamwidth of the antenna is relatively low.
  • the main application scenarios for MRR/SRR include automatic parking, lane merging assistance, and blind spot detection, etc.
  • MRR/SRR has high requirements for the angular width of the detection, and the antenna has a high requirement for the 3dB beam width of the antenna, and Antennas with low sidelobe levels are required.
  • the beam width is used to ensure the detectable angular range, and the low sidelobe is used to reduce the clutter energy reflected by the ground, reduce the probability of false alarms, and ensure driving safety.
  • LRR can be installed in front of the vehicle body, and MRR/SRR can be installed in the four corners of the vehicle. Together, they can achieve 360-degree coverage around the vehicle body.
  • the millimeter-wave radar can include a housing, and at least one printed circuit board (PCB) is built in the housing, for example, it can include a power supply PCB and a radar PCB, wherein the power supply PCB can provide the internal voltage of the radar, and can also provide a The interface and safety function of device communication; the radar PCB can provide the transmission and reception and processing of millimeter wave signals, on which are integrated components for millimeter wave signal processing and antennas for millimeter wave signal transmission and reception (transmitting antenna Tx and receiving antenna Rx) .
  • the antenna can be formed on the back of the radar PCB in the form of a microstrip array for transmitting and receiving millimeter waves.
  • Ultrasonic sensor also known as ultrasonic radar, is a sensing device that uses ultrasonic detection. Its working principle is to emit ultrasonic waves through the ultrasonic transmitting device, and receive the ultrasonic waves reflected by obstacles through the receiving device. According to the time difference of ultrasonic reflection and reception to measure the distance. At present, the distance measured by the ultrasonic sensor can be used to prompt the distance from the car body to obstacles, assist parking or reduce unnecessary collisions. It should be understood that the above-mentioned sensors are only examples of sensors that may be configured on the vehicle in the embodiment of the present application without any limitation. In other embodiments, the sensors may include but are not limited to the above-mentioned examples.
  • the lane line detection device may be an application program, which may be installed or run on a chip or component of the vehicle, or on smart devices such as mobile phones and tablet computers on the vehicle.
  • the lane line detection device can be a software module, which can be deployed in any electronic control unit (electronic control unit, ECU) of the vehicle.
  • the lane line detection device can be a newly added hardware module in the vehicle, which can be configured with relevant judgment logic or algorithms, and can be used as an ECU in the vehicle to exchange information with other ECUs or various sensors through the vehicle bus.
  • the embodiment of the present application does not limit the product form or deployment method of the lane line detection device.
  • the lane line detection device may, for example, acquire a single frame image or a group of image sequences to be processed from the camera device.
  • the lane line detection device can obtain the feature map of the first image, and determine the target feature points in the feature map, and determine the first topology according to the target feature points relationship, the target feature point is associated with a position where the first topological relationship changes, and the first topological relationship is used to indicate the relationship between lane lines in the first image.
  • the lane line detection device can divide the feature map into at least two feature map slices according to the target feature points, so as to determine the first topological relationship according to the at least two feature map slices, the complex The lane line detection scene is converted into a simple scene to improve the efficiency of lane line detection.
  • the lane line detection device may be based on the lanes in the plurality of images in the image sequence.
  • the encoding of the lines determines the second topological relationship, and the second topological relationship is used to indicate the association relationship between the lane lines in the image sequence. Therefore, the topological relationship of the lane can be obtained only by relying on the image sequence. Due to the reduction of parameters that can be introduced in the detection process and the reduction of errors caused by intermediate processes such as projection, it helps to improve the robustness of the lane line detection method.
  • a lane detection network and an encoding matching module may be configured in the lane detection device, and the lane detection network may include at least one of the following: a neural network (Backbone) module, target detection module (Point Proposal Head), feature segmentation module, feature fusion module, lane line detection module (Lane Head), and global relationship detection module.
  • a neural network Backbone
  • target detection module Point Proposal Head
  • feature segmentation module feature segmentation module
  • feature fusion module feature fusion module
  • lane line detection module LiD Head
  • global relationship detection module global relationship detection module
  • the neural network module can learn local features and global topological features in a frame of images according to the input single-frame image or image sequence, and generate a feature map of the frame of images.
  • the target detection module can be used to determine the target feature points in the feature map, so as to determine the position where the topological relationship of lane lines in the frame image changes.
  • the feature segmentation module can slice and divide the feature map according to the position of the target feature point in the feature map to obtain at least two feature map slices, and determine the at least two feature maps after parsing The association relationship between the feature points of the slice, that is, the local relationship (location relation).
  • the global relationship detection module can be used to output a global similarity matrix (Similarity Matrix) for the complete feature map, to indicate the global association relationship (ie global relation) of each feature point in the feature map, and enhance a frame of image
  • the association relationship between the lane lines in can be performed.
  • the fusion module can perform feature fusion (fusion) on the feature map or at least two feature map slices of the feature map according to the above local relationship and global relationship, and input the feature fusion result into the lane line detection module.
  • the lane line detection module can be used to detect lane lines in the feature map or at least two feature slices.
  • the encoding matching module can be used to perform encoding matching on the lane lines in at least two feature map slices belonging to the same frame image, or perform encoding matching on the lane lines in multiple images in a group of image sequences.
  • the lane line detection device can output the following results corresponding to a frame of image (represented as the first image): the first topological relationship, the lane line position and lane line code in each feature map slice, and the similarity matrix (the similarity matrix is used to indicate the global association relationship of each feature point in the feature map), and the second topological relationship of a group of image sequences to which the first image belongs, the second topological relationship is used to indicate the The association relationship between the lane lines in the image sequence.
  • the above results can be provided to the aforementioned ACC, LDWS and other systems, so that the ACC, LDWS and other systems can improve the automatic driving or assisted driving function of the vehicle according to the first topological relationship and/or the second topological relationship.
  • the function introduction of the lane line detection device in FIG. 4 is not limited.
  • the lane line detection device may also include other functional modules, or the functional modules of the lane line detection device may have Other names are not limited in this embodiment of the present application.
  • the target feature point is associated with a position when the topological relationship of the lane line changes, and this position may also be called a key position.
  • the first topological relationship is used to represent the association relationship between the lane lines in the first image, and the target feature point is the position where the first topological relationship changes.
  • the target feature point may be associated with any of the following positions: a lane line stop position, a lane line split position, or a lane line merge position.
  • Lane A and Lane B there are two parallel lanes on the same road, Lane A and Lane B. Due to the change of lane topology, Lane A and Lane B converge into Lane C in front.
  • the lane line ab between is terminated at the position point c, and the topological relationship of the lane line changes, and the position point c is the stop position of the lane line ab.
  • FIG. 5b there is a lane D on the same road. Due to the change of the lane topology, the lane D diverges into two lanes in the front and right front, namely lane E and lane F, resulting in the right lane of lane D
  • the line d0 bifurcates into the right lane line ed of lane E and the left lane line df of lane F at the position point d.
  • the topological relationship of the lane lines changes, and the position point d is the stop position of the lane line d0.
  • lane G and lane H located on two roads, due to changes in the road topology, the lane G and lane H converge into lane I, resulting in the original left lane line g0 and lane H of lane G
  • the original right lane line h0 converges at the position point g and merges into the left lane line i0 of lane I.
  • the topological relationship of the lane lines changes, and the position point g is the merged position of the lane line g0 and the lane line h0.
  • the target feature points and training model can be defined according to the three positions shown in Fig. 5a-Fig. And using the target detection model obtained through training to determine the target feature points in the corresponding feature map to identify the position where the topological relationship of the lane lines represented by the target feature points changes, so as to obtain the distance between the lane lines in the first image
  • the association relationship is the first topological relationship.
  • Fig. 5a-Fig. 5c are illustrations of the change position of the lane line topological relationship predefined in the embodiment of the present application rather than any limitation. In other embodiments, it can be based on business needs Either the scene requirements or the real road topological relationship, etc., define the target feature points, which is not limited in this embodiment of the present application.
  • the neural network (Backbone) module may include models such as convolutional neural networks (Convolutional Neural Networks, CNN) or transformation (Transformer) neural networks.
  • the input of the neural network module can be a single frame image or an image sequence.
  • the image sequence may contain multiple images collected continuously, the sequence direction of the image sequence (ie, the transformation direction of the multiple images) is the same as the vehicle's forward direction, and the neural network model can process the image sequence each time A frame of image in .
  • a single frame of image input to the neural network model or a frame of image currently to be processed in the image sequence is referred to as the first image.
  • the neural network module may perform feature extraction on the first image to obtain a feature map (feature map) of the first image.
  • the lane line detection device can use the feature map as an intermediate result, and further perform subsequent steps of lane line detection based on the feature map, so as to output the following results corresponding to the first image: the first topological relationship, each feature map slice Lane line position and lane line code, similarity matrix (the similarity matrix is used to indicate the global association relationship of each feature point in the feature map), and the second topological relationship of a group of image sequences to which the first image belongs.
  • Target detection module (Point Proposal Head)
  • the target detection module may be used to calculate the confidence that each feature point in the feature map of the first image is a target feature point, and determine the target feature point in the feature map according to the confidence.
  • the parameter meanings are shown in Table 1 below:
  • the target detection model can use the N ⁇ 1-dimensional confidence map (Confidence map) (N is the total number of cells in the feature map, N is an integer greater than or equal to) to obtain feature points in the feature map is the confidence of the target feature point, and filter out the feature points with higher confidence (for example, the confidence is greater than or equal to the first threshold) through masking (ie, the feature points with confidence lower than the first threshold are regarded as the background) as target feature points.
  • N 1-dimensional confidence map
  • N the total number of cells in the feature map
  • N is an integer greater than or equal to
  • the confidence loss function of the feature point can be shown in the following expressions (1) and (2), for example:
  • L exist indicates that there is a loss function corresponding to the loss, which can be applied to the cell containing the target feature point in the feature map; L noneexist indicates that there is no loss function corresponding to the loss, and this function can be used to reduce each background in the feature map confidence value of . If there is a target feature point at a certain feature point position in the feature map, the confidence value of this feature point can be approximately 1, and if there is no target feature point at a certain feature point position, the confidence value of this feature point is 0. Gn is a cell with lane lines.
  • the target detection module can also obtain the position of the feature point of each output cell in the UV coordinate system through the adjustment (fine-tune) of the position loss function of the feature point.
  • the UV coordinate system can be based on the upper left corner of the picture (including the first image, feature map, any feature map slice, etc.) as the origin, the U coordinate in the horizontal direction, and the V coordinate in the vertical direction. Coordinates, (u, v) are the coordinates of feature points in the picture.
  • the position loss function can use the second norm to calculate the deviation from the true value position, as shown in the following expression (3):
  • the feature segmentation module can divide the feature map into at least two feature maps in the lateral direction (perpendicular to the direction of travel of the vehicle) through identity transformation according to the lateral position of the target feature point in the feature map Fragmentation, as shown in Figure 7.
  • the feature segmentation module can also process the at least two feature map slices through mapping (such as ROI Align), and unify the output size of each feature map slice.
  • the identity transformation can be, for example, an introduced residual network (equivalent network) to transfer the information of the predicted target feature point to a proper position of the feature map, so as to ensure that the feature map can be correctly divided.
  • the global relationship detection module can learn the relationship between the position points of the lane line through multi-point to multi-point, so as to enhance the global relationship feature of the lane line.
  • the global relationship detection module may use a similarity matrix to describe the global relationship of lane lines, and position points on the same lane line may uniformly use the same element value. For example, when the position points on the lane belong to the same lane line, the corresponding elements in the similarity matrix can be set to 1; when the position points on the lane do not belong to the same lane line, the corresponding elements in the similarity matrix can be set to 2; The corresponding elements of the position points on the lane in the similarity matrix can be set to 3.
  • the loss function of the global relationship detection module can use the following expression (4), and the similarity matrix can be expressed as (5):
  • L Globel represents the global correlation between each feature point in the feature map
  • l(i, j) represents the element in row i and column j in the similarity matrix
  • C ij represents the element value
  • Np represents the dimension of the similarity matrix
  • the feature fusion module can perform feature fusion based on the output results of the global relationship detection module and the output results of the feature segmentation module, and then input them to the lane line detection module.
  • Lane line detection module (Lane Head)
  • the lane line detection module can be used to detect the confidence that the feature point in any feature map slice of the feature map or feature map is the center point of the lane line, and determine in the feature map or feature map slice according to the confidence
  • the lane line, and the center point of the lane line with a higher confidence is screened out by masking.
  • the lane line detection model can use the Np ⁇ 1-dimensional confidence map (Confidence map) to obtain the confidence of the lane line, and filter out the high confidence (for example, the confidence is greater than equal to the second threshold) (that is, the feature points whose confidence is lower than the second threshold are regarded as the background).
  • the high confidence for example, the confidence is greater than equal to the second threshold
  • the feature points whose confidence is lower than the second threshold are regarded as the background.
  • the confidence loss function of the center point of the lane line can be shown in the following expressions (6) and (7), for example:
  • L exist indicates that there is a loss function corresponding to the loss, which can be applied to the cell containing the center point of the lane line in the feature map or feature map slice; L none_exist indicates that there is no loss function corresponding to the loss, this function can be used to reduce Confidence values for each background in a feature map or feature map slice. If there is a center point of the lane line at a certain feature point position in the feature map or feature map slice, the confidence value of the feature point is approximately 1; if there is no center point of the lane line at a certain feature point position, the confidence value of the feature point Confidence value is 0.
  • the lane line detection module can also use Np ⁇ 1-dimensional semantic prediction (semantic prediction) to determine the code of the lane line, and determine the lane lines with the same code through the group class (group class).
  • semantic prediction semantic prediction
  • group class group class
  • L encode represents the encoding of the lane line.
  • the lane line detection module can also fine-tune the position of the lane line center point of each output cell in the UV coordinate system through the lane line center point position loss function.
  • the UV coordinate system can be based on the upper left corner of the picture (including the first image, feature map, and any feature map slice) as the origin, the U coordinate in the horizontal direction, and the V coordinate in the vertical direction .
  • the position loss function of the center point of the lane line can use the second norm to calculate the deviation from the true value position, as shown in the following expression (9):
  • FIG. 9 shows a schematic flowchart of a lane line detection method according to an embodiment of the present application.
  • the method can be implemented by the aforementioned lane line detection device, and the lane line detection device can be deployed on a vehicle or in a cloud server.
  • the method may include the following steps:
  • the lane line detection device acquires a feature map of the first image.
  • the first image may be a frame of image currently to be processed in a group of image sequences, and the image sequence includes a plurality of images collected continuously.
  • the lane line detection device may sequentially use the images in the plurality of images as the first image, and obtain the feature map of the first image through the aforementioned neural network module.
  • the lane line detection device determines target feature points in the feature map.
  • the lane line detection device can calculate the confidence that each feature point in the feature map is the target feature point through the aforementioned target detection module, and according to the confidence, in the feature map Determine the target feature points in .
  • the target feature point may be associated with, but not limited to, any of the following positions: a lane line stop position, a lane line bifurcation position, or a lane line merge position.
  • S930 The lane line detection device determines a first topological relationship according to the target feature points.
  • the target feature point may be predefined according to business requirements or scene requirements or real road topological relationship, and the target feature point may be associated with the position where the first topological relationship changes, and the first topology Relationships may be used to indicate associations between lane lines in the first image.
  • the above-mentioned service requirements may include but not limited to the above-mentioned high-precision map service requirements, or the requirements of the automatic driving service, or the requirements of the assisted driving service.
  • the above-mentioned scene requirements may include the need to apply the high-precision map service, or automatic Scenarios of driving business or assisted driving business, including but not limited to high-precision map building service scenarios, navigation service scenarios, automatic driving service scenarios or assisted driving service scenarios, etc.
  • the lane line detection device may segment the feature map through the aforementioned feature segmentation module according to the position of the target feature point in the feature map to obtain at least two The feature map is segmented, and the first topological relationship is determined according to the encoding of the lane lines in the at least two feature segments through the lane line detection module and the code matching module.
  • the lane line detection device can also adjust the encoding of the lane line where the target feature point is located and/or the adjacent lane line at the position associated with the target feature point.
  • the lane where the vehicle is currently located may be the current driving lane, and when the lane line detection device encodes the detected lane line, for example, the first lane from the left of the current driving lane may be The code is -1, the second lane from the left is coded as -2, and so on; the first lane from the right of the current driving lane is coded as 1, the second lane from the right is coded as 2, and so on, as shown in Figure 10a shown.
  • the feature map of the first image can be divided into at least two feature map slices by the lane line detection device according to the target feature points, and the lane line detection device can be the at least two feature map slices
  • the lane markings identified in the slices are encoded separately.
  • the segmented position of the feature map along the horizontal direction is represented by a dotted line.
  • the feature map of a single frame image can be divided into several feature map slices, for example, Feature map slice 1 and feature map slice 2.
  • the lane line detection device can identify the lane line in the feature map slice, and encode the recognized lane line according to the vehicle position, for example, in feature map slice 1, the vehicle is currently on the left side of the lane
  • the lane lines on the side can be coded as -1 and -2 respectively
  • the lane lines on the right side of the vehicle’s current lane can be coded as 1 and 2 respectively.
  • the lane lines on the left side of the vehicle’s current lane can be coded respectively is -1, -2
  • the lane line on the right side of the vehicle's current lane can be coded as 1, 2, and 3, respectively.
  • FIG. 10a shows the corresponding slice area of each feature map slice in its corresponding image.
  • the lane line that does not contain the position associated with the target feature point it can be matched according to the encoding of the lane line in different feature map slices, according to the lane line encoding of the front and rear feature map slices
  • the rules (that is, the same code is the same lane line) are uniformly classified to determine the relationship between lane lines in different feature map slices.
  • the lane line-2 in the feature map slice 1 has an association with the lane line-2 in the feature map slice 2
  • the lane line-1 in the feature map slice 1 is related to the lane line-1 in the feature map slice 2.
  • the lane line-1 has an association relationship.
  • the encoding of the lane line containing the position associated with the target feature point or the adjacent lane line of the lane line can be adjusted according to the type of position associated with the target feature point.
  • the encoding of the lane lines in the image slice determines the relationship between the lane lines in different feature map slices. For example, the encoding of the right adjacent lane line of the right lane line of the lane where the merging position and the stop position are located is decremented by 1, and the encoding of the right lane line of the right lane line of the lane where the fork position is located is decremented by 1.
  • lane line 1 and lane line 2 contain the merged position of the lane line associated with the target feature point, the encoding of the lane line 1 can remain unchanged, and the encoding of the lane line 2 is reduced by 1 Afterwards, it can be adjusted to code 1, so that it can be determined that lane line 1 and lane line 2 in feature map slice 1 have an association relationship with lane line 1 in feature map slice 2.
  • lane line 3 is the adjacent lane line located on the right side of lane line 2, and the code of lane line 3 can be adjusted to lane line 2 after subtracting 1, so that the lane in feature map slice 1 can be determined
  • Line 3 is associated with lane line 2 in feature map slice 2.
  • the encoding adjustment method shown in FIG. 10b is only an example. In actual implementation, it is necessary to adjust the lane line containing the position associated with the target feature point or the relative position of the lane line according to the actual position of the vehicle in the lane and the change of the lane topology. The coding of adjacent lane lines will not be repeated here.
  • Scenario 2 If there is a lane-changing behavior during the driving process of the vehicle, there will be a situation where the vehicle presses the lane line during the lane-changing process. Due to the change of the vehicle position, it will cause the lane line in the collected single-frame image or a group of image sequences In order to accurately know the topological relationship of lane lines in different feature map slices or different images, in a possible design, the lane line detection device can encode the lane line pressed by the vehicle as 0.
  • the lane line detection device needs to adjust the left side of lane line 0 and / Or after encoding other lane lines on the right side, the lane lines with the same number in different feature map slices or different images are classified as the same one.
  • the vehicle is driving in lane A and changes lanes to lane B.
  • the vehicle will pass the lane line between lane A and lane B.
  • Several feature map slices corresponding to the frame image such as feature map slice 1, feature map slice 2, and feature map slice 3 in Figure 10c (or multiple images in a group of image sequences, such as image 1, For the lane lines in image 2 and image 3)
  • the lane line detection device can adjust the code of the relevant lane line through the lane line detection module and the code matching module, for example, in the feature map slice of Figure 10c 2 (or Image 2), according to the vehicle pressing lane line 0 and changing to lane B on the right side of lane A, the coding of the lane line on the left side of lane A can be kept unchanged, and the other lanes on the right side of lane line 0 can be sequentially changed to Add 1 to the code of the line.
  • the relationship between the lane lines in feature map slice 1 and feature map slice 2 can be determined.
  • the coding of the lane line on the left side of lane A can be kept unchanged, and the lane line 0 The coding of other lane lines on the right is reduced by 1.
  • the relationship between the lane lines in feature map slice 1 and feature map slice 2 can be determined.
  • feature map slice 2 (or image 2) and feature map slice 3 (or image 3) in Figure 10c
  • the coding of the lane line on the right side of lane B can be kept unchanged, and the coding of the lane line on the left side of lane B can be adjusted sequentially plus 1.
  • the coding of the lane line on the right side of lane B is kept unchanged, and the coding of the lane line on the left side of lane B is sequentially adjusted minus 1.
  • the lane line detection device may also use the lane line detection module and the code matching module to determine the second A topological relationship, the second topological relationship is used to indicate an association relationship between lane lines in the image sequence.
  • Case 3 For multiple images in a group of image sequences, the position points on the lane can be classified according to the code along the vehicle’s traveling direction. For example, the position points coded as 1 in the front and rear images belong to lane line 1.
  • the lane line detection device can also determine the similarity matrix according to the feature map through the aforementioned global relationship detection module, and the similarity matrix is used to indicate the global association relationship of each feature point in the feature map .
  • the feature map can be input to the global relationship detection module obtained in advance learning and training, the global relationship
  • the detection module may output a similarity matrix corresponding to the feature map according to the position points on each lane line associated with the feature points in the feature map.
  • the global relationship detection module can use the aforementioned expression (5) to determine the similarity matrix, and the loss function can use the aforementioned expression (4) and the truth matrix.
  • the lane line detection device can analyze the obtained environment image around the vehicle, determine the correlation relationship between the lane lines in the image according to the target feature points, and detect complex lane lines.
  • the scene is converted to a simple scene to improve the efficiency of lane line detection.
  • the lane line detection device can also output relevant information on the human-machine interaction interface (Human-Machine Interaction, HMI) of the vehicle, such as the lane line topology Information, including but not limited to the current lane where the vehicle is located, each lane line contained in the road to which the current lane belongs, and the topological relationship of each lane line; high-precision maps or navigation information obtained according to the topological relationship of lane lines; The automatic driving strategy or assisted driving strategy, etc., so that the driver on the vehicle side can conveniently realize the driving control of the vehicle according to the relevant information output by the HMI, or understand the automatic driving control process of the vehicle.
  • HMI Human-Machine Interaction
  • Fig. 12a shows a schematic structural diagram of the interior of a vehicle.
  • the HMI can be the screen of the car (or called the central control display screen or central control screen) 102, 104, 105, and the HMI can output the first picture in real time, and the first picture can include the above-mentioned lane line topology Information, or high-precision maps or navigation information obtained according to the topological relationship of lane lines, or automatic driving strategies or assisted driving strategies obtained according to the topological relationship of lane lines.
  • Fig. 12b shows a schematic diagram of a head up display (head up display, HUD) scene applicable to the embodiment of the present application.
  • HUD head up display
  • the image projection device in the HUD device can project the aforementioned topological information of lane lines, or high-precision map or navigation information obtained according to the topological relationship of lane lines, or the automatic driving strategy or assisted driving strategy obtained according to the topological relationship of lane lines, etc.
  • On the windshield through the reflection of the windshield, a virtual image is formed directly in front of the driver's line of sight, so that the driver can see the information without looking down.
  • the HUD reduces the driver's inability to take into account the road conditions when looking down, and the possible driving risks caused by changes in the pupils of the eyes caused by changes in the driver's line of sight. , which is a safer vehicle-mounted display method applicable to the embodiment of the present application.
  • the embodiment of the present application is also applicable to augmented reality (augmented reality, AR) HUD (AR-HUD), so that the digital image is superimposed on the real environment outside the car, so that the driver can obtain the visual effect of augmented reality , can be used for AR navigation, adaptive cruise, lane departure warning, etc., which is not limited in this embodiment of the present application.
  • AR augmented reality
  • AR-HUD augmented reality HUD
  • the embodiment of the present application also provides a lane line detection device, which is used to implement the method performed by the lane line detection device in the above embodiment.
  • a lane line detection device which is used to implement the method performed by the lane line detection device in the above embodiment.
  • the apparatus 1300 may include: an acquiring unit 1301, configured to acquire a feature map of the first image; a first determining unit 1302, used to determine target feature points in the feature map; a second determining unit 1303 , used to determine a first topological relationship according to the target feature point, the target feature point is associated with a position where the first topological relationship changes, and the first topological relationship is used to indicate a lane line in the first image relationship between.
  • an acquiring unit 1301, configured to acquire a feature map of the first image configured to acquire a feature map of the first image
  • a second determining unit 1303 used to determine a first topological relationship according to the target feature point, the target feature point is associated with a position where the first topological relationship changes, and the first topological relationship is used to indicate a lane line in the first image relationship between.
  • the first determination unit 1302, the second determination unit 1303, the third determination unit and the fourth determination unit mentioned above may be different processor
  • each functional unit in the embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
  • the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the essence of the technical solution of this application or the part that contributes to some solutions or all or part of the technical solution can be embodied in the form of software products, and the computer software products are stored in a storage medium.
  • a computer device which may be a personal computer, a server, or a network device, etc.
  • a processor processor
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. .
  • the apparatus 1400 shown in FIG. 14 includes at least one processor 1410 , a memory 1420 , and optionally, a communication interface 1430 .
  • connection medium between the processor 1410 and the memory 1420 is not limited in this embodiment of the present application.
  • a communication interface 1430 is also included.
  • the processor 1410 communicates with other devices, data transmission can be performed through the communication interface 1430 .
  • the processor 1410 in FIG. 14 can execute instructions by calling the computer stored in the memory 1420, so that the device 1400 can execute the execution of the lane line detection device in any of the above method embodiments. Methods.
  • the embodiment of the present application also relates to a computer program product, which, when the computer program product is run on a computer, causes the computer to execute the steps implemented by the above-mentioned lane line detection device.
  • the embodiment of the present application also relates to a computer-readable storage medium, where program code is stored in the computer-readable storage medium, and when the program code is run on the computer, the computer is made to perform the steps implemented by the above-mentioned lane line detection device .
  • the embodiment of the present application also relates to a system-on-a-chip, where the system-on-a-chip includes a processor, configured to call a computer program or a computer instruction stored in a memory, so that the processor executes the method in any one of the above method embodiments.
  • the processor is coupled to the memory through an interface.
  • the chip system further includes a memory, where computer programs or computer instructions are stored.
  • the embodiments of the present application also relate to a processor, where the processor is configured to call a computer program or computer instruction stored in a memory, so that the processor executes the method in any one of the above method embodiments.
  • the processor mentioned in any of the above-mentioned places can be a general-purpose central processing unit, a microprocessor, a specific application-specific integrated circuit (application-specific integrated circuit, ASIC), or one or more for controlling any of the above-mentioned methods An integrated circuit for executing the program of the method in the embodiment.
  • the memory mentioned in any of the above can be read-only memory (read-only memory, ROM) or other types of static storage devices that can store static information and instructions, random access memory (random access memory, RAM), etc.
  • embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
  • computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions
  • the device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Traffic Control Systems (AREA)

Abstract

本申请实施例公开了一种车道线检测方法及装置，涉及自动驾驶技术领域。该方法包括：获取第一图像的特征图；在所述特征图中确定目标特征点；根据所述目标特征点确定第一拓扑关系，所述目标特征点关联所述第一拓扑关系发生变化的位置，所述第一拓扑关系用于指示所述第一图像中的车道线之间的关联关系。该方法通过识别特征图中的目标特征点来确定车道线拓扑关系，有助于提升车道线拓扑关系的检测效率。

Description

一种车道线检测方法及装置
相关申请的交叉引用
本申请要求在2021年09月09日提交中华人民共和国知识产权局、申请号为202111055992.4、申请名称为“一种车道线检测方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及自动驾驶技术领域，特别涉及一种车道线检测方法及装置。
背景技术
车道线检测是自动驾驶辅助系统(Advanced Driver Assistance System，ADAS)的重要任务，是实现自适应巡航控制(Adaptive Cruise Control，ACC)、车道偏移预警系统(Lane Departure Warning System，LDWS)等的关键技术。在针对智能车辆或无人车辆的研究中，车道线检测是一个复杂且具有挑战性的课题。车道线作为道路的一个主要部分，起到为无人驾驶车辆提供参考、指导安全驾驶的作用。同时，车道线检测还可以后续用于实现道路定位、确定车辆和道路之间的相对位置、以及辅助车辆的决策规划。
当前已有许多基于传统图像处理实现车道线检测方法，这些方法可在场景明确、易于识别的道路上取得不错的效果，但是在一些设计中，如图1所示，在使用高精度地图建图的场景中，通常会从单帧图片中识别出每条车道线上的点，然后将车道线上的点投影到绝对坐标系中，再使用聚类拟合等后处理算法建立车道线的关联关系，以获得高精度地图中的车道拓扑图。该方法存在效率较低、普适应性较低、对其它数据依赖性较强等问题。而且，随着研究的深入，车道线检测任务所对应的场景越来越多样化，在复杂的场景下如何提升车道线检测效果仍为一个难题。
发明内容
本申请实施例提供了一种车道线检测方法及装置，有助于提升车道线检测效率。
第一方面，本申请实施例提供了一种车道线检测方法，该方法可用于车道线检测装置，该车道线检测装置可以部署在车辆侧或服务器侧，可以是独立装置，也可以是装置中的芯片或部件，还可以是软件模块，本申请实施例对该车道线检测装置的部署方式或产品形态均不作限定。
该方法包括：获取第一图像的特征图；在所述特征图中确定目标特征点；根据所述目标特征点确定第一拓扑关系，所述目标特征点关联所述第一拓扑关系发生变化的位置，所述第一拓扑关系用于指示所述第一图像中的车道线之间的关联关系。
通过上述方法，根据预定义的目标特征点，能够将复杂的车道线检测场景转换为简单场景，来确定第一图像中的车道线之间的关联关系。
结合第一方面，在一种可能的实现方式中，所述在所述特征图中确定目标特征点，包括：计算所述特征图中每个特征点为所述目标特征点的置信度；根据所述置信度，在所述 特征图中确定所述目标特征点。
通过上述方法，例如可以根据目标检测算法和置信度，在特征图的多个特征点中确定目标特征点。
结合第一方面，在一种可能的实现方式中，所述根据所述目标特征点确定第一拓扑关系，包括：根据所述目标特征点在所述特征图中的位置，对所述特征图进行切片划分，得到至少两个特征图分片；根据所述至少两个特征分片中的车道线的编码，确定所述第一拓扑关系。
通过上述方法，根据目标特征点将特征图划分为至少两个特征图分片，以分别对至少两个特征图分片中的车道线进行检测。
结合第一方面，在一种可能的实现方式中，所述方法还包括：根据所述目标特征点关联的位置，调整所述目标特征点所在车道线和/或相邻车道线的编码。
通过上述方法，根据每条车道线的编码对图像序列或属于同一图像的至少两个特征图分片中的车道线进行编码匹配，减少算法引入的参数，有助于增加车道线检测算法的鲁棒性。
结合第一方面，在一种可能的实现方式中，所述目标特征点关联以下任一位置：车道线停止位置、车道线分叉位置、或者车道线合并位置。
通过上述方法，可以根据车道拓扑关系的转换关系，预定义影响车道拓扑关系变换的位置点。应理解，此处仅是对若干可能的位置的示例说明而非任何限定，在其它实施例中，还可以有其它位置，在此不再赘述。
结合第一方面，在一种可能的实现方式中，所述第一图像属于一组图像序列，所述方法还包括：根据所述图像序列中多个图像中的车道线的编码，确定第二拓扑关系，所述第二拓扑关系用于指示所述图像序列中的车道线之间的关联关系。
通过上述方法，车道线检测装置可以根据一组图像序列，确定不同图像中的车道线之间的拓扑关系，提高车道线拓扑关系的检测效率。
结合第一方面，在一种可能的实现方式中，所述方法还包括：根据所述特征图确定相似矩阵，所述相似矩阵用于指示所述特征图中各个特征点的全局关联关系。
通过上述方法，车道线检测装置可以学习一帧图像的特征图中各个特征点之间的全局拓扑关系，以增强各个特征点之间的关联关系。
第二方面，本申请实施例提供了一种车道线检测装置，包括：获取单元，用于获取第一图像的特征图；第一确定单元，用于在所述特征图中确定目标特征点；第二确定单元，用于根据所述目标特征点确定第一拓扑关系，所述目标特征点关联所述第一拓扑关系发生变化的位置，所述第一拓扑关系用于指示所述第一图像中的车道线之间的关联关系。
结合第二方面，在一种可能的实现方式中，第一确定单元用于：计算所述特征图中每个特征点为所述目标特征点的置信度；根据所述置信度，在所述特征图中确定所述目标特征点。
结合第二方面，在一种可能的实现方式中，所述第二确定单元用于：根据所述目标特征点在所述特征图中的位置，对所述特征图进行切片划分，得到至少两个特征图分片；根据所述至少两个特征分片中的车道线的编码，确定所述第一拓扑关系。
结合第二方面，在一种可能的实现方式中，所述装置还包括：调整单元，用于根据所述目标特征点关联的位置，调整所述目标特征点所在车道线和/或相邻车道线的编码。
结合第二方面，在一种可能的实现方式中，所述目标特征点关联以下任一位置：车道线停止位置、车道线分叉位置、或者车道线合并位置。
结合第二方面，在一种可能的实现方式中，所述第一图像属于一组图像序列，所述装置还包括：第三确定单元，用于根据所述图像序列中多个图像中的车道线的编码，确定第二拓扑关系，所述第二拓扑关系用于指示所述图像序列中的车道线之间的关联关系。
结合第二方面，在一种可能的实现方式中，所述装置还包括：第四确定单元，用于根据所述特征图确定相似矩阵，所述相似矩阵用于指示所述特征图中各个特征点的全局关联关系。需要说明的是，上述第一确定单元、第二确定单元、第三确定单元、或者第四确定单元可以为不同处理器，也可以为相同处理器，本申请实施例对此不做限定。
第三方面，本申请实施例提供了一种车道线检测装置，包括：处理器和存储器；所述存储器用于存储程序；所述处理器用于执行所述存储器所存储的程序，以使所述装置实现如上第一方面以及第一方面任一可能设计所述的方法。
第四方面，本申请实施例提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有程序代码，当所述程序代码在计算机上运行时，使得计算机执行上述第一方面以及第一方面可能的设计所述的方法。
第五方面，本申请实施例提供了一种计算机程序产品，当所述计算机程序产品在计算机上运行时，使得所述计算机执行上述第一方面以及第一方面可能的设计所述的方法。
第六方面，本申请实施例提供了一种芯片系统，该芯片系统包括处理器，用于调用存储器中存储的计算机程序或计算机指令，以使得该处理器执行上述第一方面以及第一方面可能的设计所述的方法。
结合第六方面，在一种可能的实现方式中，该处理器可以通过接口与存储器耦合。
结合第六方面，在一种可能的实现方式中，该芯片系统还可以包括存储器，该存储器中存储有计算机程序或计算机指令。
第七方面，本申请实施例提供了一种处理器，该处理器用于调用存储器中存储的计算机程序或计算机指令，以使得该处理器执行上述第一方面以及第一方面可能的设计所述的方法。
本申请实施例在上述各方面提供的实现的基础上，还可以进行进一步组合以提供更多实现。
上述第二方面至第七方面中任一方面中的任一可能设计可以达到的技术效果，可以相应参照上述第一方面中的任一可能设计可以达到的技术效果描述，重复之处不予论述。
附图说明
图1为一种车道线检测方法示例；
图2示出了本申请实施例适用的应用场景的示意图；
图3示出了本申请实施例的车辆感知系统的示意图；
图4示出了本申请实施例的车道线检测装置的原理示意图；
图5a-图5c示出了本申请实施例的目标特征点关联的位置的示意图；
图6示出了本申请实施例的目标检测模块的原理示意图；
图7示出了本申请实施例的特征分割模块的原理示意图；
图8示出了本申请实施例的车道线检测模块的原理示意图；
图9示出了本申请实施例的车道线检测方法的流程示意图；
图10a-图10c示出了本申请实施例的车道线编码的示意图；
图11示出了本申请实施例的全局关系检测的示意图；
图12a-图12b示出了本申请实施例的显示方式的示意图；
图13示出了本申请实施例的车道线检测方法的示意图；
图14示出了本申请实施例的车道线检测方法的示意图。
具体实施方式
本申请实施例提供了一种车道线检测方法及装置，通过识别第一图像的特征图中的目标特征点，来确定第一拓扑关系，有助于提升车道线检测效率。其中，方法和装置是基于同一技术构思的，由于方法及装置解决问题的原理相似，因此装置与方法的实施可以相互参见，重复之处不再赘述。
需要说明的是，本申请实施例中的车道线检测方案可以应用于车联网，如车-万物(vehicle to everything，V2X)、车间通信长期演进技术(long term evolution-vehicle，LTE-V)、车辆-车辆(vehicle to vehicle，V2V)等。例如可以应用于具有驾驶移动功能的车辆，或者车辆中具有驾驶移动功能的其它装置。该其它装置包括但不限于：车载终端、车载控制器、车载模块、车载模组、车载部件、车载芯片、车载单元、车载雷达或车载摄像头等其他传感器，车辆可通过该其它装置实施本申请实施例提供的车道线检测方法。当然，本申请实施例中的车道线检测方案还可以用于除了车辆之外的其它具有移动控制功能的智能终端，或设置在除了车辆之外的其它具有移动控制功能的智能终端中，或设置于该智能终端的部件中。该智能终端可以为智能运输设备、智能家居设备、机器人等。例如包括但不限于智能终端或智能终端内的控制器、芯片、雷达或摄像头等其它传感器、以及其它部件等。
需要说明的是，本申请实施例中“至少一个”是指一个或者多个，“多个”是指两个或两个以上。“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B的情况，其中A，B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。“以下至少一项(个)”或其类似表达，是指的这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a,b,或c中的至少一项(个)，可以表示：a,b,c,a和b,a和c,b和c,或a和b和c，其中a,b,c可以是单个，也可以是多个。
以及，除非有特别说明，本申请实施例提及“第一”、“第二”等序数词是用于对多个对象进行区分，不用于限定多个对象的优先级或者重要程度。例如，第一拓扑关系、第二拓扑关系，只是为了区分不同的拓扑关系，而不是表示这两个拓扑关系的优先级或者重要程度等的不同。
下面结合附图及实施例介绍本申请实施例适用的应用场景。
图2示出了本申请实施例适用的应用场景的示意图。参阅图2所示，该应用场景可以包括车辆和服务端，该服务端可以为云端，云端可以包括云端服务器和/或云端虚拟机。服务端可以与车辆进行通信，以为车辆提供多种服务，例如空中升级(over the air，OTA)服务、高精地图服务、自动驾驶或辅助驾驶服务等。
车辆可以从云端下载高精地图数据来获得高精地图，为使用者提供更加准确的导航服 务。道路信息更新是非常频繁的，该服务不仅可以更加及时的将道路信息更新到地图中，还可以降低车辆本地对存储空间的需求。例如，对于大型城市或者地区，整套高精地图的数据量大，通过云端提供的高精地图服务，可以让车辆在行驶时实时地获取当前位置小范围区域的高精地图，且该区域的高精地图可以在不需要时从车辆上释放。
车辆可以与云端进行交互，以提升自动驾驶或辅助驾驶功能，从而提升车辆的安全性和出行效率。例如，车辆可以通过车身上安装的传感装置收集路面信息和周围车辆信息，并将收集到的信息上传到云端，云端基于收集的信息进行不同场景下驾驶算法的训练，并随着训练数据的更新不断优化驾驶算法，并更新到车辆，使得车辆的应对各种场景的自动驾驶能力不断提升。再如，对于感知装置所使用的基于神经网络的图像处理算法，该图像处理算法的训练可以在云端完成，并且随着训练数据的更新而更新；相应地，车辆可以从云端获取更新后的图像处理算法，从而可以提升感知装置的图像处理能力。再如，在恶劣天气下，车辆可以通过云端获取天气信息以及道路交通事故信息，从而辅助车辆进行规划，提升出行效率，且降低车辆发生事故的风险。或者，云端可以向车辆发送实时的道路信息，比如红绿灯信息，如此，车辆可以提前接收到前方路口的红绿灯变化间隔时间，并根据当前的车速计算出车辆通过所用时间，从而判断出合适并且安全的通过时机，以及规划好车辆的行驶速度，如此，不仅可以降低车辆能耗，还可以增加行车的安全性。
此外，车辆可以通过云端获取第三方的服务，例如在驾驶员授权的情况下，快递员可以通过一次性数字授权开启车辆的后备箱，将物品放置在车内，从而实现驾驶员不在场的情况下接收快递。
车辆可以通过无线通信的方式与云端交互信息，该无线通信可以遵循车辆所接入网络的无线协议，例如蜂窝网的V2X(C-V2X)通信，该蜂窝网例如为长期演进(long term evolution，LTE)无线网络或第五代(5th generation，5G)无线网络等。
该应用场景还可以包括路侧单元(road side unit，RSU)，路侧单元可以安装在路侧，可以与云端和车辆通信，与云端通信的路侧单元可以视为与车辆类似的终端装置，与车辆通信的路侧单元可以视为与车辆类似的终端装置，也可以视为车辆的服务端装置。路侧单元可以采用无线通信的方式与车辆或云端进行交互，与车辆通信可以采用专用短距离通讯(dedicated short range communication，DSRC)技术，也可以采用基于蜂窝网的V2X(C-V2X)通信，例如，基于LTE通信协议或基于5G通信协议。与云端的通信可以采用基于蜂窝网的V2X(C-V2X)通信，例如，基于LTE通信协议或基于5G通信协议。路侧单元可以为车辆提供服务，例如实现车辆身份识别，电子收费，电子扣分等。路侧单元可以安装传感装置，以实现对道路信息的采集，进而提供车路协同服务。路侧单元可以对接路侧交通牌(例如，电子红绿灯、或电子限速牌等)，以实现对红绿灯、或限速牌的实时控制，或者可以通过云端或直接将道路信息提供给车辆，以提升自动驾驶或辅助驾驶功能。
如前所述，车道线检测是自动驾驶辅助系统(Advanced Driver Assistance System，ADAS)的重要任务，是实现自适应巡航控制(Adaptive Cruise Control，ACC)、车道偏移预警系统(Lane Departure Warning System，LDWS)等的关键技术。在针对智能车辆或无人车辆的研究中，车道线检测是一个复杂且具有挑战性的课题。车道线作为道路的一个主要部分，起到为无人驾驶车辆提供参考、指导安全驾驶的作用。同时，车道线检测还可以后续用于实现道路定位、确定车辆和道路之间的相对位置、以及辅助车辆的决策规划。
本申请实施例中，如图3所示，车辆上可以安装多种传感器，例如摄像装置、激光雷 达、毫米波雷达、超声波传感器等中的一项或多项，以通过传感器获取车辆周围的环境信息，并对获取的信息进行分析和处理，实现例如障碍物感知、目标识别、车辆定位、路径规划、驾驶员监控/提醒等功能，从而提升车辆驾驶的安全性、自动化程度和舒适度。其中，车辆根据多种传感器获得的感知信息进行综合分析，还可以确定自车在当前道路的哪个车道，道路上的各个车道线之间的拓扑关系等，从而根据道路拓扑图，提升车辆的自动驾驶或辅助驾驶功能。
其中，摄像装置用于获取车辆所在环境的图像信息，目前车辆上可以安装多个摄像头以实现对更多角度的信息的获取。激光雷达是激光探测及测距(LightLaser Detection and Ranging，LiDAR)系统的简称，主要包括发射器，接收器和信号处理单元组成，发射器是激光雷达中的激光发射机构；发射器发射的激光照射到目标物体后，通过目标物体反射，反射光线会经由镜头组汇聚到接收器上。信号处理单元负责控制发射器的发射，以及处理接收器接收到的信号，并计算出目标物体的位置、速度、距离、和/或大小等信息。
毫米波雷达以毫米波作为探测介质，可以测量从毫米波雷达到被测物体之间的距离、角度和相对速度等。毫米波雷达根据其探测距离的远近可以分为长距雷达(Long Range Radar，LRR)、中距雷达(Mid-Range Radar，MRR)以及短距雷达(Short Range Radar，SRR)。LRR主要面向的应用场景包括主动巡航以及制动辅助等，LRR对探测的角域宽度要求不高，反应到天线上是对天线的3dB波束宽度要求较低。MRR/SRR主要面向的应用场景包括自动泊车，并道辅助以及盲点检测等，MRR/SRR对探测的角域宽度要求较高，反应到天线上是对天线的3dB波束宽度要求较高，且要求天线有较低的副瓣水平。波束宽度用于保证可探测角域范围，低副瓣用于减少地面反射的杂波能量，降低虚警概率，保证驾驶安全。LRR可以安装于车身前方，MRR/SRR可以安装于车的四角位置，共同使用可以实现对于车身四周360范围的覆盖。
毫米波雷达可以包括壳体，壳体内置有至少一片印制电路板(Printed circuit board，PCB)，例如可以包括电源PCB和雷达PCB，其中电源PCB可以提供雷达内部使用电压，也可以提供与其它设备通信的接口和安全功能；雷达PCB可以提供毫米波信号的收发和处理，其上集成有用于毫米波信号处理的元器件以及用于毫米波信号收发的天线(发射天线Tx和接收天线Rx)。天线可以微带阵列的方式形成于雷达PCB的背面，用于发射和接收毫米波。
超声波传感器，又可以称为超声波雷达，是利用超声波探测的传感装置，其工作原理是通过超声波发射装置向外发射超声波，通过接收装置接收经障碍物反射回来的超声波，根据超声波反射接收的时间差来测算距离。目前利用超声波传感器测算的距离可以用于提示车体到障碍物距离，辅助停车或减少不必要碰撞。应理解的是，上述传感器仅是对本申请实施例中车辆上可能配置的传感器的示例说明而非任何限定，在其他实施例中，传感器可以包括但不限于上述举例。
本申请实施例中，车道线检测装置可以是应用程序，可以安装或运行在车辆的芯片或部件中，或车辆上的手机、平板电脑等智能设备上。或者，该车道线检测装置可以是软件模块，可以部署在车辆的任一电子控制单元(electronic control unit，ECU)中。或者，该车道线检测装置可以是车辆中新增的硬件模块，该硬件模块中可以配置有相关判断逻辑或者算法，可以作为车辆中的一个ECU，通过汽车总线与其他ECU或者各种传感器进行信息传递，实现车道线检测，本申请实施例对该车道线检测装置的产品形态或部署方式等不 做限定。
实施时，该车道线检测装置例如可以从摄像装置获取待处理的单帧图像或一组图像序列。其中，针对单帧图像，例如表示为第一图像，该车道线检测装置可以获取第一图像的特征图，并在所述特征图中确定目标特征点，根据所述目标特征点确定第一拓扑关系，所述目标特征点关联所述第一拓扑关系发生变化的位置，所述第一拓扑关系用于指示所述第一图像中的车道线之间的关联关系。其中，由于车道线检测装置可以根据目标特征点将所述特征图划分为至少两个特征图分片，以根据所述至少两个特征图分片确定所述第一拓扑关系，从而可以将复杂的车道线检测场景转换为简单场景，以提升车道线检测效率。
针对一组图像序列(该图像序列中包含连续采集的多个图像，所述第一图像属于该组图像序列)，该车道线检测装置可以根据所述图像序列中多个图像中的车道线的编码，确定第二拓扑关系，所述第二拓扑关系用于指示所述图像序列中的车道线之间的关联关系。由此，仅通过依靠图像序列即可获得车道拓扑关系，由于检测过程中可以引入的参数减少、以及减少投影等中间过程带来的误差，有助于提升车道线检测方法的鲁棒性。
在一种可能的实现方式中，如图4所示，该车道线检测装置中可以配置有车道线检测网络和编码匹配模块，该车道线检测网络可以包括以下至少一项：神经网络(Backbone)模块、目标检测模块(Point Proposal Head)、特征分割模块、特征融合模块、车道线检测模块(Lane Head)、以及全局关系检测模块。
其中，该神经网络模块可以根据输入的单帧图像或者图像序列，学习一帧图像中的局部特征和全局拓扑特征，生成该帧图像的特征图。该目标检测模块可用于在该特征图中确定目标特征点，以便确定该帧图像中车道线拓扑关系发生变化的位置。该特征分割模块可以根据所述目标特征点在所述特征图中的位置，对所述特征图进行切片划分，得到至少两个特征图分片，经过解析后，确定所述至少两个特征图分片的特征点之间的关联关系，即局部关系(location relation)。该全局关系检测模块可用于针对完整的特征图输出全局的相似矩阵(Similarity Matrix)，以指示所述特征图中各个特征点的全局关联关系(即全局关系(global relation))，增强一帧图像中的车道线之间的关联关系。融合模块能够根据上述局部关系和全局关系，对特征图或特征图的至少两个特征图分片进行特征融合(fusion)，并将特征融合结果输入车道线检测模块。该车道线检测模块可用于在所述特征图或至少两个特征分片中检测车道线。编码匹配模块可用于对属于同一帧图像的至少两个特征图分片中的车道线进行编码匹配，或对一组图像序列中的多个图像中的车道线进行编码匹配。
本申请实施例中，车道线检测装置能够输出一帧图像(表示为第一图像)对应的以下结果：第一拓扑关系、每个特征图分片中的车道线位置和车道线编码、相似矩阵(所述相似矩阵用于指示所述特征图中各个特征点的全局关联关系)，以及第一图像所属的一组图像序列的第二拓扑关系，所述第二拓扑关系用于指示所述图像序列中的车道线之间的关联关系。上述结果能够被提供给前述的ACC、LDWS等系统，以便所述ACC、LDWS等系统根据所述第一拓扑关系和/或第二拓扑关系，提升车辆的自动驾驶或辅助驾驶功能。
需要说明的是，图4中仅是对车道线检测装置的功能介绍并非限定，在其他实施例中，该车道线检测装置还可以包括其他功能模块，或者该车道线检测装置的功能模块可以具有其他名称，本申请实施例对此不做限定。
为了便于理解，在介绍本申请实施例的车道线检测方法之前，首先对本申请实施例的 目标特征点、以及车道线检测装置的各个功能模块进行解释说明。
1、目标特征点
本申请实施例中，目标特征点关联于车道线拓扑关系发生变化时的位置，该位置也可称为关键位置。为便于区分，以第一拓扑关系表示第一图像中的车道线之间的关联关系，该目标特征点即关联所述第一拓扑关系发生变化的位置。
示例的，该目标特征点可关联以下任一位置：车道线停止(stop)位置、车道线分叉(split)位置、或者车道线合并(merge)位置。
如图5a所示，同一道路上存在两条并行车道，车道A和车道B，由于车道拓扑发生变化，车道A和车道B在前方汇聚为车道C，由此导致原本位于车道A和车道B之间的车道线ab在位置点c处终止，车道线拓扑关系发生变化，该位置点c即为车道线ab的停止位置。
如图5b所示，同一道路上存在车道D，由于车道拓扑发生变化，该车道D在前方以及右前方分叉成为两条车道，即车道E和车道F，由此导致车道D的右侧车道线d0在位置点d处分叉为车道E的右侧车道线ed、以及车道F的左侧车道线df，车道线拓扑关系发生变化，该位置点d即为车道线d0的停止位置。
如图5c所示，位于两条道路上的车道G和车道H，由于道路拓扑发生变化，该车道G和车道H汇聚为车道I，由此导致车道G原本的左侧车道线g0和车道H原本的右侧车道线h0在位置点g处汇聚，合并为车道I的左侧车道线i0，车道线拓扑关系发生变化，该位置点g即为车道线g0和车道线h0的合并位置。
在设计上述神经网络模型以及目标检测模型时，可以根据图5a-图5c所示的三种位置，定义目标特征点和训练模型，以便使用训练得到的神经网络模型对第一图像进行特征检测、以及使用训练得到的目标检测模型确定相应特征图中的目标特征点，以识别出所述目标特征点表示的车道线拓扑关系发生变化的位置，以便获得该第一图像中的车道线之间的关联关系，即第一拓扑关系。
需要说明的是，图5a-图5c所示的三种位置，是对本申请实施例中预定义的车道线拓扑关系变化位置的示例说明而非任何限定，在其他实施例中，可以根据业务需要或者场景需求或者真实道路拓扑关系等，定义所述目标特征点，本申请实施例对此不做限定。
2、神经网络模块
如图4所示，该神经网络(Backbone)模块可以包括卷积神经网络(Convolutional Neural Networks，CNN)或者变换(Transformer)神经网络等模型。神经网络模块的输入可以是单帧图像，也可以是图像序列。其中，对于图像序列，该图像序列中可以包含连续采集的多个图像，该图像序列的序列方向(即多个图像的变换方向)与车辆前进方向相同，神经网络模型每次可以处理该图像序列中的一帧图像。
为便于区分，本申请实施例中，将输入至该神经网络模型的单帧图像或者图像序列中当前需要处理的一帧图像，称为第一图像。对于所述第一图像，该神经网络模块可以对该第一图像进行特征提取，获得所述第一图像的特征图(feature map)。该车道线检测装置可以该特征图作为中间结果，基于该特征图进一步地进行车道线检测的后续步骤，以便输出第一图像对应的以下结果：第一拓扑关系、每个特征图分片中的车道线位置和车道线编码、相似矩阵(所述相似矩阵用于指示所述特征图中各个特征点的全局关联关系)，以及第一图像所属的一组图像序列的第二拓扑关系。
3、目标检测模块(Point Proposal Head)
该目标检测模块可用于计算第一图像的特征图中每个特征点为目标特征点的置信度，并根据所述置信度，在所述特征图中确定所述目标特征点。下文述及的各个表达式中，参数含义表示如下表1所示：
表1
Figure PCTCN2022116161-appb-000001
如图6所示，该目标检测模型可利用N×1维置信度图(Confidence map)(N为特征图中的单元格的总数量，N为大于等于的整数)，获得特征图中特征点为目标特征点的置信度，并通过遮罩(masking)筛选出置信度较高(例如置信度大于等于第一阈值)的特征点(即将置信度低于第一阈值的特征点视作背景)作为目标特征点。
其中，特征点的置信度损失函数例如可以如下表达式(1)和(2)所示：
Figure PCTCN2022116161-appb-000002
Figure PCTCN2022116161-appb-000003
其中，L exist表示存在损失对应的损失函数，该函数可应用于特征图中包含目标特征点的单元格；L noneexist表示不存在损失对应的损失函数，该函数可用于降低特征图中每个背景的置信度值。如果特征图中的某个特征点位置存在目标特征点，该特征点的置信度值可近似为1，如果某个特征点位置处不存在目标特征点，该特征点的置信度值为0。Gn为存在车道线的单元格。
该目标检测模块还可通过特征点的位置损失函数调整(fine-tune)获得每个输出单元格的特征点在UV坐标系下的位置。需要说明的是，本申请实施例中，UV坐标系可以是以图片(包括第一图像、特征图、任一特征图分片等)左上角为原点，水平方向为U坐标，垂直方向为V坐标，(u,v)为图片中特征点的坐标。
作为示例，该位置损失函数可以使用二范数计算与真值位置的偏差，如下表达式(3)所示：
Figure PCTCN2022116161-appb-000004
4、特征分割模块
本申请实施例中，该特征分割模块可以依据目标特征点在特征图中的横向位置，经由恒等变换，将所述特征图沿横向(垂直于车辆行进方向)切分为至少两个特征图分片，如 图7所示。该特征分割模块还可以将所述至少两个特征图分片经过映射(例如ROI Align)处理，统一每个特征图分片的输出尺寸。其中，所述恒等变换例如可以为引入的残差网络(等效网络)，以将预测的目标特征点这一信息传递至特征图的合适位置，保障能够正确地对特征图进行划分。
5、全局(globel)关系检测模块
本申请实施例中，该全局关系检测模块可以通过多点对多点学习车道线位置点的关系，以增强车道线的全局关系特征。
示例的，该全局关系检测模块可以采用相似矩阵来描述车道线的全局关系，同一车道线上的位置点可以统一采用相同元素值。例如，车道上的位置点属于同一条车道线时，相似矩阵中的相应元素可以置为1；车道上的位置点不属于同一条车道线时，相似矩阵中的相应元素可以置为2；非车道上的位置点在相似矩阵中的相应元素可以置为3。
示例的，该全局关系检测模块的损失函数可采用如下表达式(4)，相似矩阵可表示为(5)：
Figure PCTCN2022116161-appb-000005
Figure PCTCN2022116161-appb-000006
其中，L Globel表示特征图中各个特征点之间的全局关联关系，l(i，j)表示相似矩阵中第i行第j列的元素，C ij表示元素值，Np表示相似矩阵的维度，
Figure PCTCN2022116161-appb-000007
表示嵌入特征(embedding feature))；K 1、K 2为常量，可以为任何值，例如K 1＝1、K 2＝2。
6、特征融合模块
本申请实施例中，该特征融合模块能够基于全局关系检测模块的输出结果、以及特征分割模块的输出结果进行特征融合，之后输入至车道线检测模块。
7、车道线检测模块(Lane Head)
该车道线检测模块可用于检测特征图或特征图的任一个特征图分片中的特征点为车道线中心点的置信度，并根据所述置信度，在特征图或特征图分片中确定车道线，以及通过遮罩(masking)筛选出置信度较高(例如置信度大于等于第二阈值)的车道线中心点。
如图8所示，该车道线检测模型可利用Np×1维置信度图(Confidence map)，获得车道线的置信度，并通过遮罩(masking)筛选出置信度较高(例如置信度大于等于第二阈值)的车道线(即将置信度低于第二阈值的特征点视作背景)。
其中，车道线中心点的置信度损失函数例如可以如下表达式(6)和(7)所示：
Figure PCTCN2022116161-appb-000008
Figure PCTCN2022116161-appb-000009
其中，L exist表示存在损失对应的损失函数，该函数可应用于特征图或特征图分片中包含车道线中心点的单元格；L none_exist表示不存在损失对应的损失函数，该函数可用于降低特征图或特征图分片中每个背景的置信度值。如果特征图或特征图分片中的某个特征点位置存在车道线中心点，该特征点的置信度值近似为1，如果某个特征点位置处不存在车道 线中心点，该特征点的置信度值为0。
示例的，该车道线检测模块还可利用Np×1维语义预测(semantic prediction)确定车道线的编码，以及通过组类(group class)确定具有同一编码的车道线。其中，车道编码损失函数如下表达式(8)所示：
Figure PCTCN2022116161-appb-000010
其中，L encode表示车道线编码。
示例的，该车道线检测模块还可通过车道线中心点位置损失函数调整(fine-tune)每个输出单元格的车道线中心点在UV坐标系下的位置。需要说明的是，本申请实施例中，UV坐标系可以是以图片(包括第一图像、特征图、任一特征图分片)左上角为原点，水平方向为U坐标，垂直方向为V坐标。
作为示例，该车道线中心点位置损失函数可以使用二范数计算与真值位置的偏差，如下表达式(9)所示：
Figure PCTCN2022116161-appb-000011
需要说明的是，本申请实施例中述及的上述车道线检测网络的各个功能模块、以及编码匹配模块，均为预先学习和训练得到的，本申请实施例对该学习和训练过程不做限定。
下面结合方法流程图介绍本申请实施例的车道线检测方法。
图9示出了本申请实施例的车道线检测方法的流程示意图。其中，该方法可由前述的车道线检测装置实现，该车道线检测装置可以部署在车辆上或云端服务器中。参阅图9所示，该方法可以包括以下步骤：
S910：车道线检测装置获取第一图像的特征图。
本申请实施例中，第一图像可为一组图像序列中当前需要处理的一帧图像，该图像序列中包含连续采集的多个图像。
在一种可选的设计中，实施S910时，车道线检测装置可以依次以所述多个图像中的图像作为第一图像，并通过前述的神经网络模块获取所述第一图像的特征图。
S920：车道线检测装置在所述特征图中确定目标特征点。
在一个示例中，该车道线检测装置可通过前述的目标检测模块，计算所述特征图中每个特征点为所述目标特征点的置信度，并根据所述置信度，在所述特征图中确定所述目标特征点。示例的，所述目标特征点可以关联但不限于以下任一位置：车道线停止位置、车道线分叉位置、或者车道线合并位置。
S930：车道线检测装置根据所述目标特征点确定第一拓扑关系。
本申请实施例中，所述目标特征点可以是根据业务需求或者场景需求或者真实道路拓扑关系等预先定义的，所述目标特征点可关联第一拓扑关系发生变化的位置，所述第一拓扑关系可用于指示所述第一图像中的车道线之间的关联关系。示例地，上述业务需求可以包括但不限于是前述的高精地图业务的需求、或自动驾驶业务的需求、或辅助驾驶业务的需求，上述场景需求可以包括需要应用该高精地图业务、或自动驾驶业务、或辅助驾驶业务的场景，包括但不限于高精地图建图服务场景、导航服务场景、自动驾驶服务场景或辅助驾驶服务场景等。通过预定义目标特征点，可以较低的处理代价辅助确定车道线拓扑关系，从而辅助上述相关业务或相关服务的准确实施。
在一个示例中，实施S930时，所述车道线检测装置可以根据所述目标特征点在所述特征图中的位置，通过前述的特征分割模块对所述特征图进行切片划分，得到至少两个特征图分片，并通过车道线检测模块和编码匹配模块，根据所述至少两个特征分片中的车道线的编码，确定所述第一拓扑关系。可选的，该车道线检测装置还可以所述目标特征点关联的位置，调整所述目标特征点所在车道线和/或相邻车道线的编码。
本申请实施例中，为便于理解，可以车辆当前所处的车道为当前行驶车道，车道线检测装置在对检测到的车道线进行编码时，例如可以将当前行驶车道的左起第一条车道编码为-1，左起第二条车道编码为-2，依次类推；将当前行驶车道的右起第一条车道编码为1，右起第二条车道编码为2，依次类推，如图10a所示。
情形一：单帧图像的不同特征图分片之间的车道线匹配
如前所述，对于第一图像，该第一图像的特征图可以被车道线检测装置根据目标特征点划分为至少两个特征图分片，车道线检测装置可以为所述至少两个特征图分片中识别的车道线分别进行编码。如图10b所示，以虚线表示特征图沿横向(垂直于车辆行进方向)的切分位置，通过沿横向切分，可将单帧图像的特征图划分为若干个特征图分片，例如，特征图分片1和特征图分片2。对于每个特征图分片，车道线检测装置可以识别该特征图分片中的车道线，并根据车辆位置为识别出的车道线进行编码，例如特征图分片1中，车辆当前所在车道左侧的车道线可分别编码为-1、-2，车辆当前所在车道右侧的车道线可分别编码为1、2，特征图分片2中，车辆当前所在车道左侧的车道线可分别编码为-1、-2，车辆当前所在车道右侧的车道线可分别编码为1、2、3。需要说明的是，本申请实施例中，为了便于理解，图10a中示出每个特征图分片在其相应图像对应的分片区域。
对于同一帧图像对应的若干个特征图分片，对于不包含目标特征点关联的位置的车道线，可以根据不同特征图分片中车道线的编码，按照前后特征图分片的车道线编码匹配规则(即编码相同为同一车道线)进行统一归类，确定不同特征图分片中的车道线之间的关联关系。例如图10a中，特征图分片1中的车道线-2与特征图分片2中的车道线-2具有关联关系，特征图分片1中的车道线-1与特征图分片2中的车道线-1具有关联关系。
对于包含目标特征点关联的位置的车道线，可以按照目标特征点关联的位置类型对包含目标特征点联的位置的车道线或该车道线的相邻车道线的编码进行调整后，根据不同特征图分片中的车道线的编码，确定不同特征图分片中的车道线之间的关联关系。例如，合并位置和停止位置所在车道的右车道线的右侧相邻车道线的编码减1，分叉位置所在车道的右车道线的右侧车道线的编码减1。
如图10b所示，特征图分片1中，车道线1和车道线2包含目标特征点关联的车道线合并位置，该车道线1的编码可保持不变，该车道线2的编码减1后可调整为编码1，从而可确定特征图分片1中的车道线1、车道线2与特征图分片2中的车道线1具有关联关系。特征图分片1中，车道线3为位于车道线2右侧的相邻车道线，该车道线3的编码减1后可调整为车道线2，从而可确定特征图分片1中的车道线3与特征图分片2中的车道线2具有关联关系。
应理解，图10b所示的编码调整方式仅为示例，具体实施时，需要根据车辆在车道的实际位置以及车道拓扑的变化，调整包含目标特征点关联的位置的车道线或该车道线的相邻车道线的编码，在此不再赘述。
情形二：若车辆行进过程中存在变道行为，在车辆变道过程中将存在车辆压车道线的 情况，由于车辆位置变化，将导致采集的单帧图像或一组图像序列中对车道线的编码与匹配结果的变化，为准确获知不同特征图分片或不同图像中的车道线拓扑关系，在一种可能的设计中，车道线检测装置可将车辆所压车道线编码为0。该情形中，与上述情形一类似，车道线检测装置需要通过车道线检测模块和编码匹配模块，按照车辆行进方向和车辆变道方向，以车道线0为界，调整车道线0左侧和/或右侧其它车道线的编码后，将不同特征图分片或不同图像中，相同编号的车道线归为同一条。
如图10c所示，车辆在车道A内行驶，可变道至车道B内行驶，在车辆变道过程中会途径车道A和车道B之间的车道线，对于车辆变道过程中采集的一帧图像对应的若干个特征图分片，如图10c中的特征图分片1、特征图分片2、特征图分片3(或者一组图像序列中的多个图像，例如图像1、图像2、图像3)中的车道线，若存在车道线编码为0，车道线检测装置可以通过车道线检测模块和编码匹配模块调整相关车道线的编码，例如，在图10c的特征图分片2(或图像2)中，根据车辆压车道线0且向车道A的右侧车道B变道，可保持车道A左侧车道线的编码不变，并依次将车道线0右侧的其它车道线的编码加1，经过编码调整后，可确定特征图分片1和特征图分片2中的各个车道线之间的关联关系。或者，在图10c的特征图分片1(或图像1)中，根据车辆向车道A的右侧车道B变道，可保持车道A左侧车道线的编码不变，并依次将车道线0右侧的其它车道线的编码减1，经过编码调整后，可确定特征图分片1和特征图分片2中的各个车道线之间的关联关系。
同理，对于图10c中的特征图分片2(或图像2)和特征图分片3(或图像3)，由于车辆是从车道A向右变道至车道B的，在特征图分片3(或图像3)中，可以保持车道B右侧的车道线的编码不变，将车道B左侧的车道线的编码依次调整加1，经过编码调整后，可确定特征图分片2(或图像2)和特征图分片3(或图像3)中的各个车道线之间的关联关系。或者，也可在特征图分片2(或图像2)中，保持车道B右侧的车道线的编码不变，将车道B左侧的车道线的编码依次调整减1，经过编码调整后，可确定特征图分片2(或图像2)和特征图分片3(或图像3)中的各个车道线之间的关联关系。相似地，对于第一图像所属的一组图像序列，车道线检测装置还可以通过车道线检测模块和编码匹配模块，根据所述图像序列中多个图像中的车道线的编码，确定第二拓扑关系，所述第二拓扑关系用于指示所述图像序列中的车道线之间的关联关系。
情形三：对于一组图像序列中的多个图像，可以沿车辆行进方向，根据编码对车道上的位置点进行归类，例如前后图像中编码为1的位置点均属于车道线1。
情形四：若车辆行进过程中存在变道行为，在车辆变道过程存在压车道线的情况，车辆所压车道线的编码为0，该情形中，与上述情形二类似，需要按照车辆变道方向，以车道线0为界，调整车道线0左侧和/或右侧其它车道线的编码后，将一组图像序列中的不同图像中，相同编号的车道线归为同一条。需要说明的是，对于一组图像序列中的不同图像，可以采用与前述介绍中对属于同一帧图像的不同特征图分片相同或相似的处理方式，确定所述第二拓扑关系，详细实现可参见前述的相关描述，在此不再赘述。
此外，本申请实施例中，车道线检测装置还可以通过前述的全局关系检测模块，根据所述特征图确定相似矩阵，所述相似矩阵用于指示所述特征图中各个特征点的全局关联关系。
如图11所示，对于待处理的第一图像，经过神经网络模块获得的所述第一图像的特征图后，该特征图可以输入至预先学习和训练得到的全局关系检测模块，该全局关系检测模 块可以根据特征图中的特征点关联的各个车道线上的位置点，输出该特征图对应的相似矩阵。其中，该全局关系检测模块可以采用前述表达式(5)确定该相似矩阵，损失函数采用前述的表达式(4)和真值矩阵。
由此，通过上述车道线检测方法，车道线检测装置可以通过对获得的车辆周围的环境图像进行解析，根据目标特征点确定该图像中的车道线之间的关联关系，将复杂的车道线检测场景转换为简单场景，以提升车道线检测效率。并且，由于检测过程中可以引入的参数减少、以及减少投影等中间过程带来的误差，有助于提升车道线检测方法的鲁棒性。
此外，在实施上述S910-S930的车道线检测方法时，在车辆侧，车道线检测装置还可以在所述车辆的人机交互界面(Human–Machine Interaction，HMI)输出相关信息，例如车道线拓扑信息，包括但不限于车辆所在的当前车道、当前车道所属的道路包含的各个车道线、各个车道线的拓扑关系；根据车道线拓扑关系获得的高精地图或者导航信息；根据车道线拓扑关系获得的自动驾驶策略或辅助驾驶策略等，以便所述车辆侧的驾驶员可以方便地根据HMI输出的相关信息，实现对所述车辆的驾驶控制，或了解该车辆的自动驾驶控制过程。
示例地，图12a示出了一种车辆内部的结构示意图。其中，HMI可以为车机的屏幕(或称为中控显示屏幕或中控屏)102、104、105，该HMI上可以实时地输出第一画面，该第一画面中可以包含上述车道线拓扑信息、或根据车道线拓扑关系获得的高精地图或者导航信息、或根据车道线拓扑关系获得的自动驾驶策略或辅助驾驶策略等。
又示例地，图12b示出了本申请实施例适用的抬头显示(head up display，HUD)场景的示意图。其中，HUD技术又称平视显示技术，近年来逐步在汽车领域、航空航天领域以及航海领域获得了越来越广泛地应用。HUD装置中的图像投射装置可把前述的车道线拓扑信息、或根据车道线拓扑关系获得的高精地图或者导航信息、或根据车道线拓扑关系获得的自动驾驶策略或辅助驾驶策略等投影到挡风玻璃上，经过挡风玻璃的反射，在驾驶员视线正前方形成虚像，使得驾驶员无需低头就可以看到这些信息。相比于图12a中仪表盘、中控屏等需要驾驶员低头观察的显示方式，HUD减少了驾驶员低头观察时无法顾及路况、以及驾驶员视线变化带来的眼睛瞳孔变化可能引发的驾驶风险，是本申请实施例适用的一种更安全的车载显示方式。此外，为了不干扰路况，本申请实施例还适用于增强现实(augmented reality，AR)HUD(AR-HUD)，以将数字图像叠加在车外真实环境上，使得驾驶员获得增强现实的视觉效果，可用于AR导航、自适应巡航、车道偏离预警等，本申请实施例对此不做限定。
本申请实施例还提供了一种车道线检测装置，用于执行上述实施例中车道线检测装置所执行的方法，相关特征可参见上述方法实施例，在此不再赘述。
如图13所示，该装置1300可以包括：获取单元1301，用于获取第一图像的特征图；第一确定单元1302，用于在所述特征图中确定目标特征点；第二确定单元1303，用于根据所述目标特征点确定第一拓扑关系，所述目标特征点关联所述第一拓扑关系发生变化的位置，所述第一拓扑关系用于指示所述第一图像中的车道线之间的关联关系。具体实现方式，请参考图2至图12b所示实施例中的详细描述，这里不再赘述。需要说明的是，本申请实施例中，第一确定单元1302、第二确定单元1303以及前文中述及的第三确定单元、第四确定单元可以为不同的处理器，也可以为同一处理器，本申请实施例对此不做限定。
需要说明的是，本申请实施例中对单元的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。在本申请的实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对一些方案做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器(processor)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
在一个简单的实施例中，本领域的技术人员可以想到上述实施例中的车道线检测装置均可采用图14所示的形式。
如图14所示的装置1400，包括至少一个处理器1410、存储器1420，可选的，还可以包括通信接口1430。
本申请实施例中不限定上述处理器1410以及存储器1420之间的具体连接介质。
在如图14的装置中，还包括通信接口1430，处理器1410在与其他设备进行通信时，可以通过通信接口1430进行数据传输。
当车道线检测装置采用图14所示的形式时，图14中的处理器1410可以通过调用存储器1420中存储的计算机执行指令，使得装置1400可以执行上述任一方法实施例中车道线检测装置执行的方法。
本申请实施例还涉及一种计算机程序产品，当所述计算机程序产品在计算机上运行时，使得所述计算机执行上述车道线检测装置所实现的步骤。
本申请实施例还涉及一种计算机可读存储介质，所述计算机可读存储介质中存储有程序代码，当所述程序代码在计算机上运行时，使得计算机执行上述车道线检测装置所实现的步骤。
本申请实施例还涉及一种芯片系统，该芯片系统包括处理器，用于调用存储器中存储的计算机程序或计算机指令，以使得该处理器执行如上述任一方法实施例中的方法。
在一种可能的实现方式中，该处理器通过接口与存储器耦合。
在一种可能的实现方式中，该芯片系统还包括存储器，该存储器中存储有计算机程序或计算机指令。
本申请实施例还涉及一种处理器，该处理器用于调用存储器中存储的计算机程序或计算机指令，以使得该处理器执行如上述任一方法实施例中的方法。
其中，上述任一处提到的处理器，可以是一个通用中央处理器，微处理器，特定应用集成电路(application-specific integrated circuit，ASIC)，或一个或多个用于控制上述任一方法实施例中的方法的程序执行的集成电路。上述任一处提到的存储器可以为只读存储器(read-only memory，ROM)或可存储静态信息和指令的其他类型的静态存储设备，随机存取存储器(random access memory，RAM)等。
应明白，本申请的实施例可提供为方法、系统、或计算机程序产品。因此，本申请可 采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
显然，本领域的技术人员可以对本申请实施例进行各种改动和变型而不脱离本申请实施例范围。这样，倘若本申请实施例的这些修改和变型属于本申请权利要求及其等同技术的范围之内，则本申请也意图包含这些改动和变型在内。

Claims (18)

  1. 一种车道线检测方法，其特征在于，包括：
    获取第一图像的特征图；
    在所述特征图中确定目标特征点；
    根据所述目标特征点确定第一拓扑关系，所述目标特征点关联所述第一拓扑关系发生变化的位置，所述第一拓扑关系用于指示所述第一图像中的车道线之间的关联关系。
  2. 根据权利要求1所述的方法，其特征在于，所述在所述特征图中确定目标特征点，包括：
    计算所述特征图中每个特征点为所述目标特征点的置信度；
    根据所述置信度，在所述特征图中确定所述目标特征点。
  3. 根据权利要求1或2所述的方法，其特征在于，所述根据所述目标特征点确定第一拓扑关系，包括：
    根据所述目标特征点在所述特征图中的位置，对所述特征图进行切片划分，得到至少两个特征图分片；
    根据所述至少两个特征分片中的车道线的编码，确定所述第一拓扑关系。
  4. 根据权利要求3所述的方法，其特征在于，所述方法还包括：
    根据所述目标特征点关联的位置，调整所述目标特征点所在车道线和/或相邻车道线的编码。
  5. 根据权利要求1-4中任一项所述的方法，其特征在于，所述目标特征点关联以下任一位置：车道线停止位置、车道线分叉位置、或者车道线合并位置。
  6. 根据权利要求1-5中任一项所述的方法，其特征在于，所述第一图像属于一组图像序列，所述方法还包括：
    根据所述图像序列中多个图像中的车道线的编码，确定第二拓扑关系，所述第二拓扑关系用于指示所述图像序列中的车道线之间的关联关系。
  7. 根据权利要求1-6中任一项所述的方法，其特征在于，所述方法还包括：
    根据所述特征图确定相似矩阵，所述相似矩阵用于指示所述特征图中各个特征点的全局关联关系。
  8. 一种车道线检测装置，其特征在于，包括：
    获取单元，用于获取第一图像的特征图；
    第一确定单元，用于在所述特征图中确定目标特征点；
    第二确定单元，用于根据所述目标特征点确定第一拓扑关系，所述目标特征点关联所述第一拓扑关系发生变化的位置，所述第一拓扑关系用于指示所述第一图像中的车道线之间的关联关系。
  9. 根据权利要求8所述的装置，其特征在于，所述第一确定单元用于：
    计算所述特征图中每个特征点为所述目标特征点的置信度；
    根据所述置信度，在所述特征图中确定所述目标特征点。
  10. 根据权利要求8或9所述的装置，其特征在于，所述第二确定单元用于：
    根据所述目标特征点在所述特征图中的位置，对所述特征图进行切片划分，得到至少两个特征图分片；
    根据所述至少两个特征分片中的车道线的编码，确定所述第一拓扑关系。
  11. 根据权利要求10所述的装置，其特征在于，所述装置还包括：
    调整单元，用于根据所述目标特征点关联的位置，调整所述目标特征点所在车道线和/或相邻车道线的编码。
  12. 根据权利要求8-11中任一项所述的装置，其特征在于，所述目标特征点关联以下任一位置：车道线停止位置、车道线分叉位置、或者车道线合并位置。
  13. 根据权利要求8-12中任一项所述的装置，其特征在于，所述第一图像属于一组图像序列，所述装置还包括：
    第三确定单元，用于根据所述图像序列中多个图像中的车道线的编码，确定第二拓扑关系，所述第二拓扑关系用于指示所述图像序列中的车道线之间的关联关系。
  14. 根据权利要求8-13中任一项所述的装置，其特征在于，所述装置还包括：
    第四确定单元，用于根据所述特征图确定相似矩阵，所述相似矩阵用于指示所述特征图中各个特征点的全局关联关系。
  15. 一种车道线检测装置，其特征在于，包括：处理器和存储器；
    所述存储器用于存储程序；
    所述处理器用于执行所述存储器所存储的程序，以使所述装置实现如所述权利要求1-7中任一项所述的方法。
  16. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有程序代码，当所述程序代码在计算机上运行时，使得计算机执行如权利要求1-7中任一项所述的方法。
  17. 一种计算机程序产品，其特征在于，当所述计算机程序产品在计算机上运行时，使得所述计算机执行如权利要求1-7中任一项所述的方法。
  18. 一种处理器，其特征在于，所述处理器用于调用存储器中存储的计算机程序或计算机指令，以使得所述处理器执行如权利要求1-7中任一项所述的方法。
PCT/CN2022/116161 2021-09-09 2022-08-31 一种车道线检测方法及装置 WO2023036032A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111055992.4 2021-09-09
CN202111055992.4A CN115797899A (zh) 2021-09-09 2021-09-09 一种车道线检测方法及装置

Publications (1)

Publication Number Publication Date
WO2023036032A1 true WO2023036032A1 (zh) 2023-03-16

Family

ID=85473195

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/116161 WO2023036032A1 (zh) 2021-09-09 2022-08-31 一种车道线检测方法及装置

Country Status (2)

Country Link
CN (1) CN115797899A (zh)
WO (1) WO2023036032A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105701449A (zh) * 2015-12-31 2016-06-22 百度在线网络技术（北京）有限公司 路面上的车道线的检测方法和装置
CN107111741A (zh) * 2014-10-06 2017-08-29 大陆汽车有限责任公司 用于具有摄像机的机动车的方法，设备和系统
CN108268033A (zh) * 2016-12-30 2018-07-10 百度（美国）有限责任公司 使用基于图的车道变换指导来操作无人驾驶车辆的方法和系统
US20200302662A1 (en) * 2019-03-23 2020-09-24 Uatc, Llc System and Methods for Generating High Definition Maps Using Machine-Learned Models to Analyze Topology Data Gathered From Sensors
US20210001877A1 (en) * 2019-07-02 2021-01-07 DeepMap Inc. Determination of lane connectivity at traffic intersections for high definition maps

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107111741A (zh) * 2014-10-06 2017-08-29 大陆汽车有限责任公司 用于具有摄像机的机动车的方法，设备和系统
CN105701449A (zh) * 2015-12-31 2016-06-22 百度在线网络技术（北京）有限公司 路面上的车道线的检测方法和装置
CN108268033A (zh) * 2016-12-30 2018-07-10 百度（美国）有限责任公司 使用基于图的车道变换指导来操作无人驾驶车辆的方法和系统
US20200302662A1 (en) * 2019-03-23 2020-09-24 Uatc, Llc System and Methods for Generating High Definition Maps Using Machine-Learned Models to Analyze Topology Data Gathered From Sensors
US20210001877A1 (en) * 2019-07-02 2021-01-07 DeepMap Inc. Determination of lane connectivity at traffic intersections for high definition maps

Also Published As

Publication number Publication date
CN115797899A (zh) 2023-03-14

Similar Documents

Publication Publication Date Title
CN109920246B (zh) 一种基于v2x通信与双目视觉的协同局部路径规划方法
US11195028B2 (en) Real-time simultaneous detection of lane marker and raised pavement marker for optimal estimation of multiple lane boundaries
US11422561B2 (en) Sensor system for multiple perspective sensor data sets
US20190204834A1 (en) Method and apparatus for object detection using convolutional neural network systems
CN113678140A (zh) 接近的车辆的定位和识别
US20220373353A1 (en) Map Updating Method and Apparatus, and Device
CN110738121A (zh) 一种前方车辆检测方法及检测系统
CN111595357B (zh) 可视化界面的显示方法、装置、电子设备和存储介质
CN111508276B (zh) 基于高精地图的v2x逆向超车预警方法、系统和介质
CN113885062A (zh) 基于v2x的数据采集融合设备、方法和系统
CN114821507A (zh) 一种用于自动驾驶的多传感器融合车路协同感知方法
US11042759B2 (en) Roadside object recognition apparatus
CN110647801A (zh) 设置感兴趣区域的方法、装置、存储介质及电子设备
EP3896639A1 (en) Methods and systems for managing an automated driving system of a vehicle
US20230050706A1 (en) Associating perceived and mapped lane edges for localization
CN114550142A (zh) 基于4d毫米波雷达和图像识别融合的车位检测方法
CN114708723B (zh) 轨迹预测方法和装置
EP3223188A1 (en) A vehicle environment mapping system
CN113435224B (zh) 用于获取车辆3d信息的方法和装置
WO2021245515A1 (en) Detection of traffic safety mirrors and navigational response
WO2023036032A1 (zh) 一种车道线检测方法及装置
Oniga et al. A fast ransac based approach for computing the orientation of obstacles in traffic scenes
CN113771845A (zh) 预测车辆轨迹的方法、装置、车辆和存储介质
CN113611008B (zh) 一种车辆行驶场景采集方法、装置、设备及介质
CN115331482A (zh) 车辆预警提示方法、装置、基站及存储介质

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22866497

Country of ref document: EP

Kind code of ref document: A1