Recognition of Lane Borders
Recognition of Lane Borders
Recognition of Lane Borders
net/publication/276353051
CITATIONS READS
0 24
1 author:
Mohamed Zahran
Benha University
22 PUBLICATIONS 27 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
The Nile Delta - Contributed Book Project published by Springer International Publishing in 2017 View project
All content following this page was uploaded by Mohamed Zahran on 18 May 2015.
ABSTRACT
1. INTRODUCTION
Recently, the automation of road extraction from digital imagery has become an
area of considerable interest. The desire to provide and update data for
geographic information systems (GIS) has moved forward the research on this
issue. For urban planning applications like traffic flow analysis and simulation,
estimation of air and noise pollution, street maintenance, etc., there is a high
demand for accurate and detailed information about the road network. A survey
on 3D City Models by the European Organization for Experimental
1
Photogrammetric Research revealed that most of the participants have stated that
information about the road network is of their greatest interest [4].
Aerial images are a valuable source that provides detailed information about
urban features, including roads and their elements such as lanes and road
markings. Thus, despite of existing GIS data, detailed road data can be extracted
with high accuracy by analyzing high resolution aerial images. Examples of road
data acquired from aerial images are number of lanes, number of vehicles, queue
lengths, and vehicle velocities necessary for estimation of traffic flow conditions
on high capacity roads. However, manual procedure required for extracting
desired information is expensive and time-consuming. Thus, there is a big
economic desire to automate the extraction of roads from aerial and satellite
imagery, and there is a lot of research in this field, too. Valuable attempts are
shown in [2,4,5,7,10]. Nevertheless, fully automatic road extraction is still an
unsolved problem.
Two different approaches for automatic road extraction have been developed
during the past years. The first approach makes use of various scales to detect
roads elements and employs local grouping criteria and also context
information. This approach is limited to gray scale imagery with high
resolutions, where roads appear as uniform regions. The second approach deals
with the connectivity of roads utilizing the information derived from multi-
spectral satellite imagery. Here, roads are modeled as lines. Also, lines detected
in multiple channels are merged together. The two approaches can be combined
to take advantage of their strengths and to support overcoming their
shortcomings [1].
Lines and edges are important objects for many applications. They occur as the
low-level primitives of choice to extract from images. Roads, railroads, and
rivers, for example can be regarded as lines in aerial or satellite images. The
2
extraction of those types of objects is very crucial for the data capture to update
GIS. The width of an extracted line is essential for applications in which it is
required to characterize the type of a road, for example. Edges are also
necessary in numerous other applications, e.g., building extraction and object
recognition [9].
Extracting lines commonly begins with detecting local edges which are then
aggregated into more globally defined lines using various grouping criteria.
Local edges are detected using an edge operator. Many edge detection methods
are described in the literature. There are several differential edge detctors which
are either first difference operators or second difference operators. Two popular
edge detectors are the Laplacian of the Gaussian (LoG) and Canny’s detectors.
The LoG is a differencing operator that detects edges at the zero crossings of the
image that is smoothed first with the Gaussian function. Zero crossings are the
locations at which the sign changes. Canny’s detector is similar to the LoG but
belongs to the class of directional derivatives [8].
Canny’s edge detector is used in this paper to find edges in the test image
patches. Here, the image patch is convolved with the Gaussian function. The
first directional derivative of the convolved image is computed. Upon that the
gradient magnitude at each point of the convoluted image is calculated. Non-
maximum suppression is performed in the direction of the gradient. The
resulting edge image is subjected to a threshold in order to eliminate false edges.
At last, a fine to coarse technique can be applied to mark additional edges.
3. SHAPE DESCRIPTORS
3
Many popular methods of describing shape boundary are also available. In
tangential representations (ψ-s), the tangent ψ angle is encoded as a function of
arc length s. This is similar to differential chain codes, but don’t have to be
pixel-to-pixel. In radial representations (r-s), the distance s from the center is
encoded as a function of arc length s [8]. The Discrete Fourier Transform (DFT)
is an accurate shape descriptor that is invariant to geometric transformations. It
gives a sequence of complex coefficients called Fourier descriptors representing
the shape in the frequency domain, where lower frequencies indicate general
shape and higher frequencies denote shape details.
Although the shape descriptors using only boundary points can be used for plane
figures, descriptors utilizing interior points, as well, are likely to be more robust.
This is because small changes in area may cause considerable changes in the
boundary, especially with regard to figures with uneven boundaries.
Straightforward shape measures of a plane figure can be found from the figure
perimeter and the figure area.
The approach starts with applying Canny’s edge detector to find edges in the
image under consideration. The detected edge points are aggregated into edge
segments utilizing both gradient magnitude and orientation. Due to the fact that
4
the stripes of interest have a uniform white region against a dark background,
their edges would have closed shapes. Thus, extracted edge segments that have
no closed shapes are ignored and excluded from subsequent processes.
On the other hand, closed edge segments that start and end at the same point are
considered. For each of those segments, the number of segment points and the
coordinates of each segment point are determined. Accordingly, segment
perimeter and segment area are calculated. Another shape measure can also be
proposed here. It is the maximum of the stripe extent in the row direction and its
extent in the column direction. Since the stripes have a rectangular shape, with
the length is much larger than the width, the maximum extent of stripe is nearly
equal to one half of its perimeter, both are in pixel units.
In order to extract the desired stripes, the relevant shape measures are to be
applied to all identified closed shapes. A pre-assessment of the stripe shape as
appear in the image might be obtained by scaling real strip dimensions or by
direct measurement of stripe dimensions on the image. Having sets of stripes
where each set has a certain size, the pre-assessment process is considered for all
sets. By connecting the centroids of extracted stripes, lane borders, and highway
width and centerline can be determined.
5. EXPERIMENTATION
Two image patches, extracted from a large-scale digital aerial image and shown
in Figures (1) and (5), are used for the research experimentation. Each of the two
patches exhibit part of a highway network. The edge segments are extracted by
applying Canny’s edge detector and the edge tracking algorithm to the two
image patches, using MATLAB software package. Figures (2) and (6) illustrate
the resulted binary edge patches containing extracted edge segments in the two
image patches.
It can be noted that there are two sets of stripes in each of the two patches. The
two sets are of dissimilar stripe lengths, although they have almost similar width
of nearly 4 pixels. The short stripes have perimeters in the ranges of thirties and
forties of pixels, whereas the long stripes are of perimeters in the ranges of
seventies and eighties of pixels. Since the perimeter (P) of a stripe is twice the
sum of its length (L) and its width (W), the stripe area (A) can be set roughly
equal to W(P/2-W).
5
Prototype software has been developed and applied to the resulted binary edge
patches. The software extracts edge segments of closed figures and computes the
perimeter of each figure by summing up the number of figure points. The figure
area is estimated from the pixel coordinates of figure points. The extent in the
row direction, the extent in the column direction, and the maximum of the two
extents are also determined for each closed figure.
In order to identify the stripes of interest, the software considers each closed
figure that satisfies the criteria of perimeter and area as a candidate. The
perimeter criteria mean that the perimeter of candidate figures has to be either in
a range of 30 pixels to 50 pixels or in a range of 70 pixels to 90 pixels. The first
range is appropriate for picking the short strips, whereas the other range
supports extracting long strips. The area criteria mean that the area of candidate
figures has to be either in a range of 4(30/2-4)-T pixels to 4(50/2-4)+T pixels or
in a range of 4(70/2-4)-T pixels to 4(90/2-4)+T pixels. T is a tolerance of few
pixels, say 10 pixels. Again the two different area ranges are used for the
extraction of short and longer strips, respectively.
Most stripes existing in the two image patches are successfully recognized using
the criteria of stripe perimeter and stripe area as applied on the extracted closed
figures. Figures (3) and (7) exhibit the resulted binary image patches containing
extracted stripes. Figures (4) and (8) illustrate an enlarged view of the stripes
contained in the rectangles drawn in Figures (3) and (7). Tables (1) and (2) list
the shape attributes of the stripes detected in the first and second image patches,
respectively. The listed characteristics are the perimeter (P), the area (A), the
ratio A/P2, P/2, Maximum extent (Max Ext.) and the centroid coordinates (XC
and YC) of each of those stripes.
It can be realized from the figures of Table (1) that a total of 28 short stripes and
8 long stripes are extracted from the first image patch. The perimeters of short
stripes range from 42 pixels to 46 pixels, whereas the perimeters of long stripes
range from 77 pixels to 88 pixels. The areas of those stripes vary from 65 pixels
to 70 pixels for short stripes, and from 135.5 pixels to 153 pixels for long
stripes.
6
On the other hand, figures of Table (2) reveal that a total of 144 short stripes and
6 long stripes are detected in the second image patch. The perimeters of short
stripes vary from 31 pixels to 47 pixels, and the perimeters of long stripes (6)
vary from 72 pixels to 88 pixels. The relevant stripe areas range from 47.5 pixels
to 72.5 pixels for short stripes, and from 116.5 pixels to 149 pixels for long
stripes.
The ratio A/P2 of a stripe is a scale invariant measure. However, other irrelevant
closed figurers in the image patch do have A/P2 values that are close to the
corresponding values of stripes. Thus, the A/P2 measure is found ineffective on
its own in extracting the stripes of interest.
In contrast, it is noticed that the maximum extent of any of the extracted stripes
is found almost equal to one half of its perimeter. This is obvious by looking at
the records of P/2 and Max Ext. in the two tables. The difference between those
two parameters for any of the stripes has never surpassed one pixel. Such a
correlation has been found invariant regardless the figure size, position and
orientation. In fact, it has been used as an additional constraint, in the developed
software to discard irrelevant objects that are not elongated but have closed
shapes.
It has been also found that there are some stripes that exist in the two image
patches but have not been recognized. Those stripes either do not have closed
figures or do have insignificant size. The first category includes stripes having
gaps along their boundaries or consisting of contiguous partial stripes. The other
category consists of fractional stripes of limited size compared with the common
size of short stripes. Considering such stripe size might lead to extracting some
other irrelevant objects of closed figures.
At last, since the orientation of the longer edges of stripes are almost invariable,
it can be used as an additional constraint to discriminate stripes from other
unrelated objects. The discrete boundary coding can be used here to describe the
stripe boundary in terms of chain codes, each corresponds to a segment
direction.
7. CONCLUSIONS
7
extracted edge segments of closed figures are only considered. The related shape
measures are computed for and applied to all identified closed figures to extract
the desired stripes. In relation to the results of experiments carried out on two
image patches showing a part of a road network, the following conclusions can
be made:
Most stripes existing in the two image patches are successfully recognized
utilizing the perimeter and area measures as applied to edge segments of
closed figures.
Prior information about the dimensions of existing stripes is necessary
before starting automatic recognition. Such information can be obtained
from sample measurements on the ground or on the image.
The relationship between the maximum extent of a stripe and its perimeter
can be employed to discard irrelevant objects that have no such elongation
as the stripes have.
Orientation of the stripe edges can be a valuable attribute in
discriminating stripes from other irrelevant objects. Discrete boundary
coding is a popular method through which orientation is parameterized.
As the resolution of the digital image gets higher, the stripes become
sharper and also more unique as their width in pixels gets larger. This
would let the proposed approach to perform better.
REFERENCES
[1] Baumgartner, A., Steger, C., Mayer, H., Eckstein, W. and Ebner, H. (1999)
“Automatic Road Extraction Based on Multi-Scale, Grouping, and Context.”
Photogrammetric Engineering & Remote Sensing, Vol. 65, No.7, pp. 777–785.
[2] Eker, O. and Seker, D. (2004) “Semi-automatic Road Extraction from
Digital Aerial Photographs.” International Archives of Photogrammetry and
Remote Sensing, Vol. 35, Part B3.
[3] Haralic, R. M. and Shapiro, L. G. (1992) “Computer and Robot Vision.”
Addison-Wesley, Inc., Reading, MA.
[4] Hinz, S. and Baumgartner, A. (2000) “Road Extraction In Urban Areas
Supported By Context Objects.” International Archives of Photogrammetry and
Remote Sensing, Vol. 33, Part B3, pp. 405-412.
[5] McKeown, D., McGlone, C., Cochran, S., Hsieh, Y., Roux, M., Shufelt, J.
(1996) “Automatic Cartographic Feature Extraction Using Photogrammetric
Principles” Digital Photogrammetry: An Addendum to the Manual of
Photogrammetry, ASPRS, Maryland.
[6] Nevatia R. (1982) “Machine Perception.” Prentice-Hall, Inc., Englewood
Cliffs, New Jeresy.
8
[7] Poz, A., Vale, G., and Zanin, I. (2004) “Automatic Road Segment Extraction
by Grouping Road objects.” International Archives of Photogrammetry and
Remote Sensing, Vol. 35, Part B3.
[8] Schenk T. (1999) “Digital Photogrammetry” Laurelville, OH: TerraScience.
[9] Steger, C. (2000) “Subpixel-Precise Extraction Of Lines And Edges.”
International Archives of Photogrammetry and Remote Sensing, Vol. 33, Part
B3, pp. 141-156.
[10] Trinder, J. and Wang, Y. (1998) “Knowledge-Based Road Interpretation in
Aerial Images.” International Archives of Photogrammetry and Remote Sensing,
Vol. 32, part 4, pp. 635–640.
9
Figure 1: The First Image Patch
10
Figure 3: Border Stripes Detected in the First Image Patch
11
Figure 5: The Second Image Patch
12
Figure 7: Border Stripes Detected in the Second Image Patch
13
Table 1: Shape Attributes of Stripes Detected in the First Image Patch.
14
Table 2: Shape Attributes of Stripes Detected in the Second Image Patch.
15
Table 2, continued
16
Table 2, continued
17
Table 2, continued
18