KR20060134719A - Method for recognizing parking site of vehicle - Google Patents

Abstract

A method for recognizing a parking area for vehicles is provided to improve operation speed of a parking area searching system by restricting an inspection range by using a bird's eye view. A parking area recognizer(110) recognizes 3D position information of a parked vehicle from a 3D stereo vision camera and outputs the recognized result. A path generator(120) determines a path toward the parking area according to the 3D position information. A vehicle status sensor(130) includes a speed sensor, a steering angle sensor, a yaw rate sensor, and an accelerometer. A vehicle position estimator(140) estimates the present position of the vehicle from the sensed result of the vehicle status sensor. An HMI(Human Machine Interface,150) displays the recognized result from the parking area recognizer and displays a control signal according to the manipulation of a user. A path tracking controller(160) tracks a parking path according to the estimated vehicle position and the input signal from the HMI and outputs a steering angle command. A steering unit(170) adjusts the steering angle according to the steering angle command from the path tracking controller.

Description

Recognizing parking site of vehicle

1 is a view showing an example of an apparatus to which a parking section recognition method of a vehicle according to the present invention is applied;

2 is a flowchart of a method for recognizing a parking section of a vehicle according to the present invention.

3 is a view showing an image picked up by the camera of FIG.

FIG. 4 is a diagram illustrating pixel classification results of an image picked up by the parking section recognition unit of FIG. 1; FIG.

FIG. 5 is a diagram illustrating a stereo matching result of a classification result in the parking section recognition unit of FIG. 1; FIG.

FIG. 6 is a view illustrating a road / obstacle separation result in the parking section recognition unit of FIG. 1; FIG.

FIG. 7 is a diagram illustrating a bird's eye view detected from a disparity map of parking site marking in the parking section recognition unit of FIG.

FIG. 8 is a diagram illustrating an obstacle depth map from an obstacle disparity map in the parking partition recognition unit of FIG. 1. FIG.

FIG. 9 is a diagram illustrating a seed point input by a user, a guideline, and an obstacle histogram according to FIG. 2. FIG.

FIG. 10 is a view showing parking zone display results and input image results according to FIG.

Explanation of symbols for main parts of the drawings

110: parking section recognition unit 120: route planning generation unit

130: vehicle attitude sensor 140: vehicle position estimation unit

150: human machine interface unit 160: path tracking control unit

170: steering unit

The present invention relates to a vehicle, and more particularly to a method for recognizing a parking section of a vehicle.

In general, women and rowers are under pressure to park backwards between vehicles.

Therefore, many vehicles employ an ultrasonic parking assistance system that warns the driver of the stopping distance from the obstacle.

For example, Toyota and Aishin Seiki introduced a rear guide monitor to assist drivers by investigating the driving course predicted by images of rear vision cameras. Asin Seiki's next generation of devices is expected to include environmental awareness to give the driver the best possible vision. They use wheel speed sensors, structures from driving technology and visual cameras, such as Intermediate View Reconstruction (IVR) technology, to create virtualized images from the best viewer point.

The automatic parking system will automate parking operation with automatic steering control and automatic braking control.

This automatic parking system consists of three elements. : Automatic steering and braking system using a path generation including the measurement of the position of the target position, performing the planned trajectory, Human Machine Interface (HMI) receives the driver's input and provides visual information of the ongoing parking progress do.

Positioning of the target location can be performed in a variety of ways, namely complete manual indication, GPS infrastructure and vision based positioning of available parking compartments.

Toyota's Intelligent Parking Assist (IPA) is a semi-automatic parking system with braking control left at the driver's responsibility. Toyota's IPA has developed a position measurement of the target position by HMI. It shows the possible target position in the image of the rear vision camera, and the driver can change the target position using direction control buttons such as up, down, left, right and rotation.

Although semi-automatic parking systems have been commercialized, complete manual instructions are too boring and complex for daily use.

Thus, it is natural that the need for vision-based position measurement of available weld compartments is rapidly increasing.

Accordingly, the present invention has been made to solve the above problems, by increasing the operating speed by limiting the irradiation range of the parking compartment using a bird's eye view and an obstacle depth map. It is an object of the present invention to provide a method for recognizing a parking section of a vehicle to be resistant to noise such as stain, waste, and shadow.

In order to achieve the above object, there is provided a method of recognizing a parking compartment of a vehicle according to the present invention, including: imaging a left image and a right image of a parking compartment by a 3D stereo vision camera; Detecting a vertical edge by performing a left pixel and a right image of the captured parking section on a pixel-by-pixel basis, and stereo matching the detected vertical edge detecting a disparity map by stereo matching and disparity map of the detected disparity map through a plane surface constraint separating the map and the obstacle disparity map, and z, x world coordinate points from the disparity map of the separated parking partition representation. Detecting a mapped bird's eye view; and an obstacle depth map mapped from z-to-x world coordinate points from an opterkle disparity map of the separated parking compartment representation. A method for recognizing a parking section of a vehicle, the method comprising: detecting a guideline by huff transforming a bird's eye view of the detected parking section display; Detecting an obstacle histogram by projecting a guideline onto the detected obstacle depth map; Setting a search range in the bird's eye view based on the detected optocle histogram and a seed point input from a user; And recognizing the parking section through template matching in the set search range.

The parking section recognition step is characterized in that the matching of the template by the inner rectangle and the outer rectangle in the set search range is another feature.

Other objects, features and advantages of the present invention will become apparent from the following detailed description of embodiments taken in conjunction with the accompanying drawings.

Hereinafter, a preferred embodiment of a method for recognizing a parking section of a vehicle according to the present invention will be described.

1 is a view showing an example of an apparatus to which a vehicle parking partition recognition method according to the present invention is applied, and includes a three-dimensional stereo vision camera (unsigned), a parking partition recognition unit 110, and a route plan generation unit 120. ), A vehicle attitude sensor 130, a vehicle position estimating unit 140, a human machine interface unit 150, a path tracking control unit 160, and a steering unit 170.

FIG. 2 is a flowchart illustrating a method for recognizing a parking section of a vehicle according to the present invention, FIG. 3 is a view showing an image captured by the camera of FIG. 1, and FIG. 4 is a captured image in the parking section recognition unit of FIG. FIG. 5 is a diagram illustrating a pixel classification result, and FIG. 5 is a diagram illustrating a stereo matching result of the classification result in the parking section recognition unit of FIG. 1.

FIG. 6 is a diagram illustrating a road / obstacle separation result in the parking section recognition unit of FIG. 1, and FIG. 7 is a disparity map of the parking section display in the parking section recognition unit of FIG. 1. FIG. 8 is a view illustrating a bird's eye view detected from a disparity map of parking site marking, and FIG. 8 is selected from an obstacle disparity map in the parking section recognition unit of FIG. 1. It is a figure which shows an obstacle depth map.

FIG. 9 is a view illustrating a seed point position input by a user, a guideline and an histogram histogram according to FIG. 2, and FIG. 10 is a parking section display result and input according to FIG. A diagram showing image results.

Referring to the accompanying drawings, a method for recognizing a parking compartment of a vehicle according to the present invention configured as described above will be described in detail as follows.

The following system helps to park the vehicle semi-automatically / automatically by recognizing the parking area.

First, as shown in FIG. 1, the parking section recognition unit 110 recognizes the three-dimensional parking section from the parking section image captured by the three-dimensional stereo vision camera (unsigned) and the three-dimensional location information of the parked vehicle. The parking lot recognition result is output. The camera is a Point Gray Research Bumblebee camera installed in the rear of the test vehicle.

Then, the route plan generation unit 120 plans and outputs the route according to the 3D position information of the parked vehicle output from the parking section recognition unit 110.

In addition, the vehicle attitude sensor 130 detects and outputs a wheel speed, a steering angle, a turning speed, and an acceleration of the vehicle from a wheel speed sensor, a steering angle sensor, a yaw rate sensor, and an acceleration sensor not shown.

Then, the vehicle position estimator 140 estimates and outputs the position of the vehicle from the wheel speed, the steering angle, the turning speed, and the acceleration output from the vehicle attitude sensor 130.

In addition, the human machine interface unit 150 displays a parking lot recognition result output from the parking compartment recognition unit 110 and interfaces a control signal according to a user's manipulation.

The route tracking control unit 160 then outputs the route plan output from the route plan generator 120, the vehicle position estimate output from the vehicle position estimator 140, and the user input signal interfaced from the human machine interface unit 150. According to the parking path to track the steering angle command output.

Accordingly, the steering unit 170 adjusts the steering angle according to the steering angle command output from the path tracking control unit 160.

This steering adjustment enables semi-automatic or automatic parking.

At this time, the parking section recognition unit 110 recognizes the parking section as follows according to the parking section recognition method according to the present invention shown in FIG.

First, from (a) left image and (b) right image, which are stereo images of a typical parking compartment captured from the three-dimensional stereo vision camera of FIG. 3, left as shown in Table 1 below. As shown in Table 2 below, the pixel class is represented by the difference in intensity between one pixel of each of the left image and the right image and the four neighboring pixels around the pixel. Will be detected.

Each detected image has 640 * 480 resolution and 24-bit color information, and each image is modified by a Point Gray Research Triclops rectification library.

Table 1

Table 2

는 표1의 하나의 픽셀과 그 주위의 4개의 이웃 픽셀들을 하기한 수학식1에 의해 검출되게 되는데, 부드러운 면의 픽셀은 제로 클래스(zero class)로 분류되고, 에지 픽셀(edge pixel)은 넌-제로 클래스(non-zero class)로 클래서피케이션되게 되며, 쓰레스홀드(threshold) T의 효과를 줄이기 위해 히스토그램 이퀄라이제이션(histogram equalization) 또는 어뎁티브 쓰레스홀드(adaptive threshold)가 사용될 수 있다.Here, the pixel class of Table 2

Is detected by Equation 1 below, with one pixel of Table 1 and four neighboring pixels surrounding it, wherein the smooth-sided pixels are classified as zero class, and the edge pixels are non-zero. It will be clasified into a non-zero class, and histogram equalization or adaptive thresholds can be used to reduce the effect of threshold T.

(Equation 1)

는 그레이 벨루(gray value)이다.here,

Is a gray value.

Such pixel classification results (pixel classification classified as vertical edge) and (b) pixels classified as vertical edge (pixel) classified as vertical edge (Fig. 4). edge). Here, 13.7% of the total pixels are classified as horizontal edges and 7.8% as vertical edges.

Only by the pixel classified as vertical edge of the detected pixel classified as horizontal edge and pixel classified as vertical edge. Stereo matching is performed, wherein stereo matching is performed by (a) an investigated point of the left image, as shown in FIG. 5, and (b) a chores of the right image. Steps through class comparison, class similarity, color similarity and maximum similarity detection as detecting the funding point. It consists of step-by-step test sequences.

Only correspondence candidates that passed in the previous test step will be examined in the next test. The irradiation range of the pixel is limited to a horizontal line with -35 to 35 disparity assuming that the Bumblebee vertical alignment is correct.

That is, the correspondence test is performed on the irradiation pixel and the pixels having the same class.

Subsequently, class similarity is a measure of how similar a candidate pixel is to a pixel irradiated in a 3 * 3 class window, which is measured by Equation 2 below.

(Equation 2)

here,

Subsequently, color similarity is a measure of how similar a candy date pixel is to a pixel irradiated in a 5 * 5 color window, which is measured by Equation 3 below.

(Equation 3)

here,

Next, maximum similarity detection is a result of class similarity and color similarity, which is a result of Equation 4 below. If the highest total similarity is lower than some threshold, then the irradiated pixel is ignored as having failed to find the corresponding point.

(Equation 4)

In general, the pixels on the road surface are d (x, y), which is the plane surface constraint, that is, the y coordinate of the pixel is the pixel disparity of pixel as shown in equation (5). Satisfies a linear relationship with

(Equation 5)

는 포컬 렝스(focal length)이고,

는 틸트 앵글(tilt angle)이다.Where B is the baseline, H is the height,

Is the focal length,

Is the tilt angle.

Obstacles of pixels For example, adjacent vehicles do not follow a constraint. Therefore, the disparity map that is the result of the stereo matching is divided into two disparity maps, as shown in Fig. 6: (a) an obstacle disparity map. And (b) a disparity map of parking site marking.

와

좌표 포인트로 매핑하여 주차 구획 표시의 버즈 아이 뷰(bird's eye view)를 검출하고, 옵스터클 디스퍼러티 맵(obstacle disparity map)으로부터

와

좌표 포인트로 매핑하여 옵스터클 뎁스 맵(obstacle depth map)을 검출하게 되는데(S1, S2), 이러한 과정은 다음과 같다.This disparity map of parking site marking

Wow

Map to coordinate points to detect a bird's eye view of the parking parcel indication, and from the obstacle disparity map

Wow

Mapping to the coordinate point to detect the obstacle depth map (Obstacle depth map) (S1, S2), this process is as follows.

는 하기한 방정식 (6-1) 과 같은 디스퍼러티(disparity)에 역 비례하게 된다.The disparity between the camera and the object

Is inversely proportional to the disparity as shown in equation (6-1) below.

(Equation 6-1)

The above-mentioned plane surface constraint can be simplified as shown in equation (6-2) below. P ₁ and P ₂ are constant parameters of the camera environment.

(Equation 6-2)

here,

사이의 관계는 하기한 방정식 (6-3), (6-4)와 같이 간략화될 수 있다.The y coordinate of the pixel on the road surface

The relationship between can be simplified as shown in the following equations (6-3) and (6-4).

(Equation 6-3)

(Equation 6-4)

와 픽셀의 x 좌표 사이의 관계는 3각 측량에 의해 하기한 방정식 (6- 5)와 같이 정의된다.

The relationship between and the x coordinate of the pixel is defined by triangulation as shown in equation (6-5) below.

(Equation 6-5)

와

의 알오아이(ROI:Region Of Interest)로의 값을 복사하여 이루어진다. 주차 구획 표시의 다른 칼라를 갖는 픽셀들은 아스팔트와 잔디와 같이 구조의 노이즈를 제거하는 것으로 무시된다.Using this relationship, as shown in Figures 7 (a) and (b), the disparity map of parking site marking is converted to a bird's eye view. The bird's eye view comes from the disparity map of parking site marking

Wow

This is done by copying the value into the ROI (Region Of Interest). Pixels with different colors of parking compartment markings are ignored to remove noise from the structure, such as asphalt and grass.

와

로 좌표 포인으로 매핑된 검출되게 된다(S2). 도8 (a) 및 (b)에 도시된 바와 같이, 옵스터클 디스퍼러티 맵(obstacle disparity map)에서 픽셀에 상응하는 월드 좌표 포인트(

와

)는 상기한 방정식 (6-1)과 (6-5)에 의해 결정될 수 있다. The obstacle depth map then projects the disparity information of the pixels that dissatisfied the plane surface constraint.

Wow

It is detected that the mapped to the coordinate point as (S2). As shown in Fig. 8 (a) and (b), the world coordinate point corresponding to the pixel in the obstacle disparity map (

Wow

) Can be determined by the above equations (6-1) and (6-5).

A guideline is detected by hough transforming a bird's eye view of the parking partition display detected in step S1 (S3).

Subsequently, the guideline is projected onto the obstacle disparity map detected in step S2 to detect an obstacle histogram (S4).

Here, since stereo matching does not perform sub pixel resolution for real-time performance, pixels of the disparity map are contributed to a vertical array in an obstacle depth map. contribute) The elements of the obstacle depth map accumulate contributions of corresponding disparity map pixels. Noise on the disparity map can be eliminated by removing elements of the obstacle depth map under a certain threshold. In general, noise in the disparity map does not produce a peak on the obstacle depth map. Thus, free space is the continuous portion of an obstacle histocle under a certain threshold.

Subsequently, as shown in FIG. 9 (a), the bidirectional search range is set from the seed point input from the user in the obstacle histacle detected in step S4. (S5).

Here, the position of the own vehicle is limited to -40 to 40 degrees related to the longitudinal direction of the parking area. Therefore, the peak of the Hough transform in each of these ranges is the guideline shown in Fig. 9B. In the direction of the guideline, the survey area of the parking compartment center is 20% of the center of the free space. In other directions orthogonal to the guideline direction, the initial estimate of the parking compartment center is a distant position from the guideline to half the size of the template. The irradiation range in the orthogonal direction is 10 pixels, and each irradiation range is 10 degrees.

Subsequently, the parking section is recognized by template matching by an inner lectangle and an outer rectangle in the search range set in step S5 (S6).

That is, the final template matching uses a template consisting of two rectangles inferred from the specification regarding the parking compartment drawing. Template matching measures how many pixels of the parking compartment representation exist between two rectangles, ie, inner and outer rectangles. Fig. 10 (a) shows the result on the bird's eye view of the parking compartment display. Figure 10 (b) projects the results for a bird's eye view of the input image. Since the irradiation range is narrowed by the obstacle depth map, template matching successfully detects the correct position in spite of stain, waste and shadow. In addition, template matching, the bottleneck of the localization process, consumes less time. The total calculation time is about 400-500 msec at 1 G㎐ of PC.

As described above, the vehicle parking section recognition method according to the present invention increases the operating speed by limiting the irradiation range of the parking section using a bird's eye view, and maintains a stain and waste. And noise such as shadows.

Those skilled in the art will appreciate that various changes and modifications can be made without departing from the spirit of the present invention.

Therefore, the technical scope of the present invention should not be limited to the contents described in the embodiments, but should be defined by the claims and their equivalents.

Claims (2)

Imaging a left image and a right image of the parking compartment by means of a three-dimensional stereo vision camera; and performing a left image and a right image of the captured parking compartment by pixel Detecting a vertical edge by classifying the unit, detecting a disparity map by stereo matching the detected vertical edge, and detecting the vertical edge. Separating the disparity map of the parking compartment mark from the disparity map and the obstacle disparity map through a plane surface constraint; Detecting a bird's eye view mapped to a z, x world coordinate point from the disparity map of the separated parking compartment representation; In teokeul parking disc recognition method of a vehicle compartment comprising the step of detecting the option requester greater depth maps (depth obstacle map) mapped to the z, x coordinate points from World peoreo Ti map, Huff transforming a bird's eye view of the detected parking segment indication to detect guidelines; Detecting an obstacle histogram by projecting a guideline onto the detected obstacle depth map; Setting a search range in the bird's eye view based on the detected optocle histogram and a seed point input from a user; And performing a template matching on the set search range to recognize a parking section. The method of claim The parking section recognition step of the vehicle parking section recognition method characterized in that for performing the template matching by the inner rectangle and the outer rectangle (tangle) in the set search range.

Images