CN112949493A - Lane line detection method and system combining semantic segmentation and attention mechanism - Google Patents
Lane line detection method and system combining semantic segmentation and attention mechanism Download PDFInfo
- Publication number
- CN112949493A CN112949493A CN202110235668.4A CN202110235668A CN112949493A CN 112949493 A CN112949493 A CN 112949493A CN 202110235668 A CN202110235668 A CN 202110235668A CN 112949493 A CN112949493 A CN 112949493A
- Authority
- CN
- China
- Prior art keywords
- lane line
- lane
- attention mechanism
- inputting
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 41
- 238000001514 detection method Methods 0.000 title claims abstract description 27
- 238000000034 method Methods 0.000 claims abstract description 23
- 239000013598 vector Substances 0.000 claims description 13
- 238000010586 diagram Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000013527 convolutional neural network Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 8
- 230000009286 beneficial effect Effects 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 238000012805 post-processing Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000004140 cleaning Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30256—Lane; Road marking
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a lane line detection method and system combining semantic segmentation and attention mechanism, relating to the technical field of lane line detection and comprising the following steps: inputting the picture into a backbone network to extract lane line characteristics in the picture, inputting the characteristics extracted by the backbone network into a full connection layer, and obtaining local characteristics of each lane line after processing by the full connection layer; inputting the local features of each lane line into an attention mechanism module, acquiring the global features of each lane line, and superposing the global features and the local features to obtain final lane line parameters; and comparing the calculated mean square error loss of the lane lines with the real lane lines in the input picture, calculating the cross entropy loss of the mask map and calculating the gradient of the corresponding convolutional layer, updating the parameters of the convolutional layer and the full-connection layer by using a back propagation algorithm, and continuously iterating until the model converges. The method of the invention accelerates the convergence speed of the model and improves the robustness of the model under the condition of not increasing the operation amount.
Description
Technical Field
The invention relates to the technical field of lane line detection, in particular to a lane line detection method and system combining semantic segmentation and attention mechanism.
Background
The lane line detection is one of basic tasks in the field of assistant driving, and in an assistant driving system, the lane line position is accurately acquired, which is an important prerequisite for subsequent vehicle driving route planning and lane departure early warning; the driving assistance system captures a driving area in front of a currently driving vehicle through a vehicle camera, and can position the specific position of the lane line in the image according to the image clues of the lane line by utilizing a digital image processing technology.
The general flow of the traditional lane line detection system is based on camera calibration of monocular images, image cleaning), feature extraction, lane line model fitting, time domain integration and image physical space correspondence; the core lane line detection method relies on manually extracted features to identify, such as color features, structure tensor, lane line outline and the like, and the position of a lane line is obtained after screening; the algorithm based on the gradient change of the local area has poor effect under the condition of complex road conditions, the lane line is easy to miss detection under the conditions of lane line shielding, blurring, light reflection and the like, and the traditional method has insufficient robustness in complex scenes; and the traditional algorithm generally takes a long time and is difficult to run in real time in a low-end embedded device.
With the wide application of deep learning in the field of computer vision, the schemes for solving the problem of lane line detection by utilizing deep learning are gradually increased, and the current common method is to use semantic segmentation to treat each pixel point in an image as a two-classification problem of a background and a lane line, use a lane line pixel point set obtained by post-processing and fit the lane line pixel point set into a lane line equation; the method based on semantic segmentation can effectively solve the problem of lane line ambiguity generally, but under the condition of occlusion, the scheme still has missed detection; in addition, certain time is consumed for post-processing, and the performance of the whole lane line detection process is reduced. At present, how to effectively solve the problem of missing detection of a lane line in complex scenes such as blurring and shielding on the basis of accelerating the convergence speed of a model and improving the robustness of the model is a difficult problem to be solved by technical personnel in the field.
Disclosure of Invention
In view of the above, the present invention provides a lane line detection method and system combining semantic segmentation and attention mechanism, so as to solve the problems proposed in the background art, and effectively solve the problem of missing detection of lane lines in complicated scenes such as blur and occlusion on the basis of accelerating the model convergence speed and improving the model robustness.
In order to achieve the above object, an aspect of the present invention provides a lane line detection method combining semantic segmentation and attention mechanism, including the following steps:
inputting the picture into a backbone network to extract lane line characteristics in the picture, and acquiring a lane line characteristic diagram;
inputting the lane line feature map into a full-connection layer to obtain local features of each lane line;
and acquiring the global characteristic of each lane line by using the attention mechanism through the local characteristic, and superposing the global characteristic and the local characteristic to obtain the final lane line parameter.
Preferably, the lane line feature map is assisted by segmentation to obtain a mask map of each lane line;
comparing the final lane line parameters and the mask image with the real lane lines in the input image, and respectively calculating the mean square error loss of the lane lines and the cross entropy loss of the mask image;
calculating the gradient of the corresponding convolution layer according to the mean square error loss of the lane line and the cross entropy loss of the mask map, updating the parameters of the convolution layer and the full-connection layer by using a back propagation algorithm, and continuously iterating until the model converges.
Preferably, the local features include cubic curve equation coefficients a, b, c, d of each lane line, a confidence p of each lane line, a start point ordinate s, and an end point ordinate e.
Preferably, the specific calculation step of the mean square error loss is as follows:
setting N anchor points at equal intervals on a longitudinal axis of the input picture, and acquiring a longitudinal coordinate of each anchor point;
inputting the vertical coordinate of the anchor point into an actual parameter equation and a network prediction parameter equation to obtain the horizontal coordinate of the anchor point on the actual lane line and the horizontal coordinate of the predicted lane line;
making a difference between the abscissa of the predicted lane line and the abscissa of the actual lane line, adding the differences between the abscissas of all the anchor points to obtain the total offset of the lane line, and dividing the total offset by the number of the anchor points to obtain the mean square error loss of the lane line, wherein the specific formula is as follows:
wherein M is the number of target lane lines, N is the number of anchor points,
is the predicted abscissa of the ith anchor point of the jth lane line,
the actual abscissa of the ith anchor point representing the jth lane.
Preferably, the parameter equation of the network prediction is
y＝ax3+bx2+cx+d；
Wherein y represents the ordinate of the anchor point and x represents the abscissa of the anchor point.
Preferably, the calculation formula of the cross entropy loss of the mask map is as follows:
wherein w represents the width of the mask pattern, h represents the height of the mask pattern, yiActual tag, y, representing the ith positioniWhen 0 denotes that the position is background, yi1 denotes that the position is a lane line, piRepresenting the probability value of the predicted lane line.
Preferably, the attention mechanism comprises:
inputting each vector output by the full-connection layer, namely the parameter of a specific lane line, into another full-connection layer, and then connecting a softmax layer to obtain a weight between the specific lane line and other lane lines;
and multiplying the weight values with corresponding lane line vectors respectively and then adding the multiplied weights to obtain the specific global vector of one lane line.
By adopting the technical scheme, the method has the following beneficial technical effects: generally, the fully connected layer can also extract global information of the input picture, but since there are many mutual relationships between lane lines in the actual scene, for example, the relative position relationship between different lane lines, a common vanishing point, and the like. In order to utilize the global information composed of the constraints among different lane lines and improve the capture capability of the model for the global information, an attention mechanism module is introduced.
Preferably, the segmentation assistance comprises the following specific steps:
extracting a specified number of feature maps from each layer of the backbone network to obtain feature maps with different scales;
unifying the feature maps of different scales by utilizing a linear interpolation mode to obtain a feature map of a unified scale;
and stacking the characteristic graphs with the uniform scale, and obtaining a mask graph of the lane line in the input picture through a shallow convolutional neural network.
By adopting the technical scheme, the method has the following beneficial technical effects: the segmentation auxiliary module makes full use of the characteristics of the input picture, improves the number of the supervision signals, accelerates the convergence speed of the model, and improves the robustness of the model.
On the other hand, the invention provides a lane line detection system combining semantic segmentation and an attention mechanism, which comprises a backbone network module, a full connection layer module, an attention mechanism module and a segmentation auxiliary module; wherein,
inputting the picture into a backbone network module to extract lane line characteristics in the picture, and acquiring a lane line characteristic diagram;
inputting the lane line feature map into a full-connection layer module to obtain local features of each lane line;
inputting the local features into an attention mechanism module, acquiring global features of each lane line, and superposing the global features with the local features to obtain final lane line parameters;
inputting the lane line feature map into a segmentation auxiliary module to obtain a mask map of each lane line;
comparing the final lane line parameters and the mask image with the real lane lines in the input image, and respectively calculating the mean square error loss of the lane lines and the cross entropy loss of the mask image;
calculating the gradient of the corresponding convolution layer according to the mean square error loss of the lane line and the cross entropy loss of the mask map, updating the parameters of the convolution layer and the full-connection layer by using a back propagation algorithm, and continuously iterating until the model converges.
Preferably, when the trained model is used for lane line prediction, the segmentation assisting module does not participate in the prediction process.
By adopting the technical scheme, the method has the following beneficial technical effects: the better training effect is achieved without increasing the reasoning time.
The technical scheme can show that the invention discloses and provides a lane line detection method combining semantic segmentation and attention mechanism, and compared with the prior art, the lane line detection method has the following beneficial technical effects:
(1) an end-to-end algorithm for directly predicting a lane curve equation is provided, post-processing is not needed, and the speed is higher.
(2) The semantic segmentation module is introduced into a training frame, extra supervision signals are added to assist the training of the model, and the segmentation module is closed when deployment is carried out, so that the convergence speed of the model is accelerated and the robustness of the model is improved under the condition that the deployment reasoning time and extra calculation amount are not increased.
(3) An attention mechanism is introduced into a training frame, the extraction capability of the model on the global features of the lane line is improved, and the problem of missing detection of the lane line under complex scenes such as blurring and shielding is effectively solved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a schematic flow diagram of the process of the present invention;
FIG. 2 is a schematic diagram of a segmentation assistance module of the present invention;
FIG. 3 is a schematic illustration of the attention mechanism of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The first aspect of the embodiment of the invention discloses a lane line detection system combining semantic segmentation and an attention mechanism, which comprises a backbone network module, a full connection layer module, an attention mechanism module and a segmentation auxiliary module; as shown in fig. 1, the method comprises the following steps:
the method comprises the following steps: inputting the picture to a backbone network module to extract lane line characteristics in the picture and obtain a lane line characteristic diagram;
it should be noted that, according to actual needs, a suitable backbone network may be selected, such as Resnet, Darknet, hourgals, etc., and Resnet18 is used in this embodiment.
Step two: inputting the lane line feature map into the full-connection layer module to obtain the local features of each lane line;
further, the local features include cubic curve equation coefficients a, b, c, d of each lane line, a confidence p of each lane line, a start point ordinate s, and an end point ordinate e. Wherein p is a probability value of 0 or more and 1 or less.
Step three: inputting the lane line feature map into a segmentation auxiliary module to obtain a mask map of each lane line;
the polynomial equation coefficients of the lane lines are learned from features extracted from the trunk network by utilizing the full connection layer, and only the coefficients of the lane line equations are used as supervision signals when loss is calculated in the mode, namely, the learnable content of the network is less, the difficulty of model convergence is increased when the number of the supervision signals is less, overfitting of the model is easily caused, and the robustness of the model in different scenes is reduced. Therefore, the model can learn the polynomial equation coefficient of the lane line by using a learning mode of semantic segmentation, and meanwhile, the segmentation information of the lane line can be learned by using another branch.
As shown in fig. 2, the operation manner of the segmentation assistance module is as follows:
extracting a specified number of feature maps from each layer of the backbone network to obtain feature maps with different scales;
unifying the feature maps of different scales by utilizing a linear interpolation mode to obtain a feature map of a unified scale;
stacking feature maps with uniform scales, and obtaining a mask map of a lane line in an input picture through a shallow convolutional neural network; the mask map is a gray scale map in which the pixel value at the background position is 0 and the pixel value at the predicted lane line position is 1.
By adopting the technical scheme, the method has the following beneficial technical effects: the segmentation auxiliary module makes full use of the characteristics of the input picture, improves the number of the supervision signals, accelerates the convergence speed of the model, and improves the robustness of the model. In addition, when the trained model is used for lane line prediction, the segmentation auxiliary module can be closed, so that a better training effect is achieved without increasing inference time.
Step four: inputting the local features into an attention mechanism module to obtain the global features of each lane line;
in order to improve the capture capability of the model for global information, an attention mechanism module is introduced. Generally, the full link layer module can also extract global information of an input picture, but since there are many mutual relationships between lane lines in an actual scene, for example, the relative position relationship between different lane lines, a common vanishing point, and the like. Therefore, for each vector output by the fully-connected layer module, namely parameters of a specific lane line, including coefficients of a curve equation, a starting point and a final point of the lane line and confidence of the lane line, the vector consisting of the numbers is input into another fully-connected layer module, and then a softmax layer is connected to obtain a weight between the lane line and other lane lines; and finally, multiplying the weights by corresponding lane line vectors respectively and then adding the multiplied weights to obtain a global vector of the lane line, and adding the vector obtained by the original full-connection layer module to obtain final output. Taking lane line number 0 as an example, an example of the attention mechanism is shown in fig. 3.
Step five: comparing the parameters of the lane lines and the mask image with the real lane lines in the input image, and respectively calculating the mean square error loss of the lane lines and the cross entropy loss of the mask image;
specifically, the calculation steps of the mean square error loss of the lane line are as follows:
setting N anchor points at equal intervals on a longitudinal axis of an input picture, and acquiring a longitudinal coordinate of each anchor point;
inputting the ordinate of the anchor point into an actual parameter equation and a network prediction parameter equation to obtain the abscissa of the anchor point on an actual lane line and the abscissa of a predicted lane line;
making difference between the abscissa of the predicted lane line and the abscissa of the actual lane line, adding the differences between the abscissas of all the anchor points to obtain the total offset of the lane line, and dividing the total offset by the number of the anchor points to obtain the mean square error loss of the lane line, wherein the specific formula is as follows:
wherein M is the number of target lane lines, N is the number of anchor points,
is the predicted abscissa of the ith anchor point of the jth lane line,
the actual abscissa of the ith anchor point representing the jth lane.
Further, the calculation formula of the cross entropy loss of the mask map is as follows:
wherein w represents the width of the mask, h represents the height of the mask, yiActual tag, y, representing the ith positioniWhen 0 denotes that the position is background, yi1 denotes that the position is a lane line, piRepresenting the probability value of the predicted lane line.
It should be noted that the parameter equation of the network prediction is
y＝ax3+bx2+cx+d；
Wherein y represents the ordinate of the anchor point, x represents the abscissa of the anchor point, and a, b, c, d are cubic curve equation coefficients obtained from the fully-connected layer and are known quantities.
Step six: calculating the gradient of the corresponding convolutional layer according to the mean square error loss of the lane line and the cross entropy loss of the mask graph, updating the parameters of the convolutional layer and the full-connection layer by using a back propagation algorithm, and continuously iterating until the model converges.
Preferably, the attention mechanism module is applied in the following manner:
inputting each vector output by the fully-connected layer module, namely the parameter of a specific lane line, into another fully-connected layer module, and then connecting a softmax layer to obtain the weight between the specific lane line and other lane lines;
and multiplying the weights by corresponding lane line vectors respectively and then adding the weights to obtain a specific global vector of one lane line.
The second aspect of the embodiments of the present invention provides a lane line detection method combining semantic segmentation and attention mechanism, including the following steps:
inputting the picture into a backbone network to extract lane line characteristics in the picture, and acquiring a lane line characteristic diagram;
inputting the lane line feature map into the full-connection layer to obtain the local feature of each lane line;
and acquiring the global characteristic of each lane line by using the attention mechanism for the local characteristic, and superposing the global characteristic and the local characteristic to obtain the final lane line parameter.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A lane line detection method combining semantic segmentation and attention mechanism is characterized by comprising the following steps:
inputting the picture into a backbone network to extract lane line characteristics in the picture, and acquiring a lane line characteristic diagram;
inputting the lane line feature map into a full-connection layer to obtain local features of each lane line;
and acquiring the global characteristic of each lane line by using the attention mechanism through the local characteristic, and superposing the global characteristic and the local characteristic to obtain the final lane line parameter.
2. The method for lane line detection with semantic segmentation and attention mechanism according to claim 1, further comprising:
the lane line feature map is subjected to segmentation assistance to obtain a mask map of each lane line;
comparing the final lane line parameters and the mask image with the real lane lines in the input image, and respectively calculating the mean square error loss of the lane lines and the cross entropy loss of the mask image;
calculating the gradient of the corresponding convolution layer according to the mean square error loss of the lane line and the cross entropy loss of the mask map, updating the parameters of the convolution layer and the full-connection layer by using a back propagation algorithm, and continuously iterating until the model converges.
3. The method for lane line detection with semantic segmentation and attention mechanism according to claim 1, wherein the local features comprise cubic curve equation coefficients a, b, c and d of each lane line, and a confidence p, a start point ordinate s and an end point ordinate e of each lane line.
4. The method for detecting the lane line by combining the semantic segmentation and the attention mechanism according to claim 3, wherein the specific calculation steps of the mean square error loss of the lane line are as follows:
setting N anchor points at equal intervals on a longitudinal axis of the input picture, and acquiring a longitudinal coordinate of each anchor point;
inputting the vertical coordinate of the anchor point into an actual parameter equation and a network prediction parameter equation to obtain the horizontal coordinate of the anchor point on the actual lane line and the horizontal coordinate of the predicted lane line;
making a difference between the abscissa of the predicted lane line and the abscissa of the actual lane line, adding the differences between the abscissas of all the anchor points to obtain a total offset of the lane line, and dividing the total offset by the number of the anchor points to obtain a mean square error loss of the lane line, wherein the specific formula is as follows:
wherein M is the number of target lane lines, N is the number of anchor points,
is the predicted abscissa of the ith anchor point of the jth lane line,
the actual abscissa of the ith anchor point representing the jth lane.
5. The method for detecting lane lines by combining semantic segmentation and attention mechanism as claimed in claim 4, wherein the network prediction parameter equation is
y＝ax3+bx2+cx+d；
Wherein y represents the ordinate of the anchor point and x represents the abscissa of the anchor point.
6. The method for detecting lane lines by combining semantic segmentation and attention mechanism according to claim 2, wherein the cross entropy loss of the mask map is calculated by the formula:
wherein w represents the width of the mask pattern, h represents the height of the mask pattern, yiActual tag, y, representing the ith positioniWhen 0 denotes that the position is background, yi1 denotes that the position is a lane line, piRepresenting the probability value of the predicted lane line.
7. The method of claim 1, wherein the attention mechanism comprises:
inputting the parameters of the lane line output by the full connection layer into another full connection layer, and then connecting a softmax layer to obtain the weight between the lane line and other lane lines;
and multiplying the weight values with corresponding lane line vectors respectively and then adding the multiplied weights to obtain the global vector of the lane line.
8. The method for detecting lane lines by combining semantic segmentation and attention mechanism according to claim 2, wherein the specific steps of segmentation assistance comprise:
extracting a specified number of feature maps from each layer of the backbone network to obtain feature maps with different scales;
unifying the feature maps of different scales by utilizing a linear interpolation mode to obtain a feature map of a unified scale;
and stacking the characteristic graphs with the uniform scale, and obtaining a mask graph of the lane line in the input picture through a shallow convolutional neural network.
9. A lane line detection system combining semantic segmentation and attention mechanism is characterized by comprising a backbone network module, a full connection layer module, an attention mechanism module and a segmentation auxiliary module; wherein,
inputting the picture into a backbone network module to extract lane line characteristics in the picture, and acquiring a lane line characteristic diagram;
inputting the lane line feature map into a full-connection layer module to obtain local features of each lane line;
inputting the local features into an attention mechanism module, acquiring global features of each lane line, and superposing the global features with the local features to obtain final lane line parameters;
inputting the lane line feature map into a segmentation auxiliary module to obtain a mask map of each lane line;
comparing the final lane line parameters and the mask image with the real lane lines in the input image, and respectively calculating the mean square error loss of the lane lines and the cross entropy loss of the mask image;
calculating the gradient of the corresponding convolution layer according to the mean square error loss of the lane line and the cross entropy loss of the mask map, updating the parameters of the convolution layer and the full-connection layer by using a back propagation algorithm, and continuously iterating until the model converges.
10. The system of claim 9, wherein the segmentation assistance module does not participate in the prediction process when using the trained model for lane line prediction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110235668.4A CN112949493B (en) | 2021-03-03 | 2021-03-03 | Lane line detection method and system combining semantic segmentation and attention mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110235668.4A CN112949493B (en) | 2021-03-03 | 2021-03-03 | Lane line detection method and system combining semantic segmentation and attention mechanism |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112949493A true CN112949493A (en) | 2021-06-11 |
| CN112949493B CN112949493B (en) | 2024-04-09 |
Family
ID=76247406
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110235668.4A Active CN112949493B (en) | 2021-03-03 | 2021-03-03 | Lane line detection method and system combining semantic segmentation and attention mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112949493B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113705515A (en) * | 2021-09-03 | 2021-11-26 | 北京百度网讯科技有限公司 | Training of semantic segmentation model and generation method and equipment of high-precision map lane line |
| CN113739811A (en) * | 2021-09-03 | 2021-12-03 | 阿波罗智能技术(北京)有限公司 | Method and device for training key point detection model and generating high-precision map lane line |
| CN113807236A (en) * | 2021-09-15 | 2021-12-17 | 北京百度网讯科技有限公司 | Method, apparatus, device, storage medium and program product for lane line detection |
| CN114581887A (en) * | 2022-03-07 | 2022-06-03 | 上海人工智能创新中心 | Method, device and equipment for detecting lane line and computer readable storage medium |
| CN114581976A (en) * | 2022-02-24 | 2022-06-03 | 中山大学 | Face Quality Judgment Method Based on Local Contribution Variance |
| CN115063761A (en) * | 2022-05-19 | 2022-09-16 | 广州文远知行科技有限公司 | Lane line detection method, device, equipment and storage medium |
| CN115147811A (en) * | 2022-07-01 | 2022-10-04 | 小米汽车科技有限公司 | Lane line detection method and device and electronic equipment |
| CN115588177A (en) * | 2022-11-23 | 2023-01-10 | 荣耀终端有限公司 | Method of training lane line detection network, electronic device, program product, and medium |
| CN116503828A (en) * | 2023-04-24 | 2023-07-28 | 大连理工大学 | A LiDAR-based train track detection method |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180211117A1 (en) * | 2016-12-20 | 2018-07-26 | Jayant Ratti | On-demand artificial intelligence and roadway stewardship system |
| EP3576008A1 (en) * | 2018-05-30 | 2019-12-04 | Aptiv Technologies Limited | Image based lane marking classification |
| CN111178245A (en) * | 2019-12-27 | 2020-05-19 | 深圳佑驾创新科技有限公司 | Lane line detection method, lane line detection device, computer device, and storage medium |
| CN111242037A (en) * | 2020-01-15 | 2020-06-05 | 华南理工大学 | Lane line detection method based on structural information |
| CN111259796A (en) * | 2020-01-16 | 2020-06-09 | 东华大学 | A Lane Line Detection Method Based on Image Geometric Features |
| CN111460921A (en) * | 2020-03-13 | 2020-07-28 | 华南理工大学 | Lane line detection method based on multitask semantic segmentation |
| CN111553205A (en) * | 2020-04-12 | 2020-08-18 | 西安电子科技大学 | Method, system, medium and video surveillance system for vehicle re-identification without license plate information |
| CN111950467A (en) * | 2020-08-14 | 2020-11-17 | 清华大学 | Fusion network lane line detection method and terminal device based on attention mechanism |
| CN112307850A (en) * | 2019-08-01 | 2021-02-02 | 浙江商汤科技开发有限公司 | Neural network training method, lane line detection method, device and electronic equipment |
| CN112381101A (en) * | 2021-01-13 | 2021-02-19 | 南京理工大学 | Infrared road scene segmentation method based on category prototype regression |
-
2021
- 2021-03-03 CN CN202110235668.4A patent/CN112949493B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180211117A1 (en) * | 2016-12-20 | 2018-07-26 | Jayant Ratti | On-demand artificial intelligence and roadway stewardship system |
| EP3576008A1 (en) * | 2018-05-30 | 2019-12-04 | Aptiv Technologies Limited | Image based lane marking classification |
| CN112307850A (en) * | 2019-08-01 | 2021-02-02 | 浙江商汤科技开发有限公司 | Neural network training method, lane line detection method, device and electronic equipment |
| CN111178245A (en) * | 2019-12-27 | 2020-05-19 | 深圳佑驾创新科技有限公司 | Lane line detection method, lane line detection device, computer device, and storage medium |
| CN111242037A (en) * | 2020-01-15 | 2020-06-05 | 华南理工大学 | Lane line detection method based on structural information |
| CN111259796A (en) * | 2020-01-16 | 2020-06-09 | 东华大学 | A Lane Line Detection Method Based on Image Geometric Features |
| CN111460921A (en) * | 2020-03-13 | 2020-07-28 | 华南理工大学 | Lane line detection method based on multitask semantic segmentation |
| CN111553205A (en) * | 2020-04-12 | 2020-08-18 | 西安电子科技大学 | Method, system, medium and video surveillance system for vehicle re-identification without license plate information |
| CN111950467A (en) * | 2020-08-14 | 2020-11-17 | 清华大学 | Fusion network lane line detection method and terminal device based on attention mechanism |
| CN112381101A (en) * | 2021-01-13 | 2021-02-19 | 南京理工大学 | Infrared road scene segmentation method based on category prototype regression |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113705515B (en) * | 2021-09-03 | 2024-04-12 | 北京百度网讯科技有限公司 | Training of semantic segmentation model and generation method and device of high-precision map lane line |
| CN113739811A (en) * | 2021-09-03 | 2021-12-03 | 阿波罗智能技术(北京)有限公司 | Method and device for training key point detection model and generating high-precision map lane line |
| CN113705515A (en) * | 2021-09-03 | 2021-11-26 | 北京百度网讯科技有限公司 | Training of semantic segmentation model and generation method and equipment of high-precision map lane line |
| CN113739811B (en) * | 2021-09-03 | 2024-06-11 | 阿波罗智能技术(北京)有限公司 | Method and equipment for training key point detection model and generating high-precision map lane line |
| CN113807236A (en) * | 2021-09-15 | 2021-12-17 | 北京百度网讯科技有限公司 | Method, apparatus, device, storage medium and program product for lane line detection |
| CN113807236B (en) * | 2021-09-15 | 2024-05-17 | 北京百度网讯科技有限公司 | Method, device, equipment, storage medium and program product for lane line detection |
| CN114581976A (en) * | 2022-02-24 | 2022-06-03 | 中山大学 | Face Quality Judgment Method Based on Local Contribution Variance |
| CN114581887A (en) * | 2022-03-07 | 2022-06-03 | 上海人工智能创新中心 | Method, device and equipment for detecting lane line and computer readable storage medium |
| CN114581887B (en) * | 2022-03-07 | 2024-06-07 | 上海人工智能创新中心 | Method, device, equipment and computer readable storage medium for detecting lane line |
| CN115063761A (en) * | 2022-05-19 | 2022-09-16 | 广州文远知行科技有限公司 | Lane line detection method, device, equipment and storage medium |
| CN115147811A (en) * | 2022-07-01 | 2022-10-04 | 小米汽车科技有限公司 | Lane line detection method and device and electronic equipment |
| CN115588177B (en) * | 2022-11-23 | 2023-05-12 | 荣耀终端有限公司 | Method for training lane line detection network, electronic device, program product and medium |
| CN115588177A (en) * | 2022-11-23 | 2023-01-10 | 荣耀终端有限公司 | Method of training lane line detection network, electronic device, program product, and medium |
| CN116503828A (en) * | 2023-04-24 | 2023-07-28 | 大连理工大学 | A LiDAR-based train track detection method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112949493B (en) | 2024-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112949493A (en) | Lane line detection method and system combining semantic segmentation and attention mechanism | |
| CN109726627B (en) | Neural network model training and universal ground wire detection method | |
| CN107154024A (en) | Dimension self-adaption method for tracking target based on depth characteristic core correlation filter | |
| CN113313047B (en) | A lane line detection method and system based on lane structure prior | |
| CN113657560A (en) | Weak supervision image semantic segmentation method and system based on node classification | |
| CN112270691B (en) | Monocular video structure and motion prediction method based on dynamic filter network | |
| CN112819853A (en) | Semantic prior-based visual odometer method | |
| CN114140469A (en) | Depth hierarchical image semantic segmentation method based on multilayer attention | |
| CN114626445B (en) | Dam termite video identification method based on optical flow network and Gaussian background modeling | |
| CN113724379A (en) | Three-dimensional reconstruction method, device, equipment and storage medium | |
| CN109376641A (en) | A moving vehicle detection method based on UAV aerial video | |
| CN117523100A (en) | Three-dimensional scene reconstruction method and device based on neural network and multi-view consistency | |
| CN113989612A (en) | Remote sensing image target detection method based on attention and generation countermeasure network | |
| CN113496194B (en) | Information processing device, information processing method, vehicle, information processing server, and recording medium | |
| CN112215766B (en) | Image defogging method combining image restoration and image enhancement and convolution network thereof | |
| CN118570439A (en) | An attention-guided hybrid dual-branch spatial decomposition neural network method for infrared dim small target detection | |
| CN115272423B (en) | Method and device for training optical flow estimation model and readable storage medium | |
| CN116665097A (en) | Self-adaptive target tracking method combining context awareness | |
| CN113723409A (en) | Gated imaging semantic segmentation method and device | |
| CN112907464A (en) | Underwater thermal disturbance image restoration method | |
| CN113012072A (en) | Image motion deblurring method based on attention network | |
| Chen et al. | GADO-Net: An improved AOD-Net single image dehazing algorithm | |
| CN119027462B (en) | An optical flow estimation method based on improved FlowNetS | |
| CN117275069B (en) | End-to-end head gesture estimation method based on learnable vector and attention mechanism | |
| CN116503603B (en) | Training method of inter-class shielding target detection network model based on weak supervision semantic segmentation and feature compensation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |