JP6824759B2 - Bound line detection device, lane marking system, and lane marking method - Google Patents

Description

The present invention relates to a technique for detecting a lane marking indicating a parking lot.

A parking support device has been proposed in which a parking line indicating a parking lot is detected from a camera-captured image and a parking target position is set based on the detected parking line (see, for example, Patent Document 1). In the parking support device proposed in Patent Document 1, after the vehicle once sets the parking target position based on each end of the two lane markings, the vehicle is separated from the two lane markings by a predetermined distance. In this state, the detection range (area of interest) is not set, and as shown in FIG. 10, the vehicle 101 is in a state of advancing to the back side of the parking lot, and each end 104 of the two lane markings 102 and 103 and 105 is included in the detection ranges (regions of interest) 106 and 107. The detection range (area of interest) is the range in which the parking support device can detect the parking zone.

JP-A-2016-124398

In the parking support device proposed in Patent Document 1, after the parking target position is once set, the parking target position is not corrected until the ends 104 and 105 of the two lane markings 102 and 103 are detected again. Therefore, the amount of correction at the time when the respective ends 104 and 105 of the two marking lines 102 and 103 are detected again becomes large, and there is a possibility that the correction becomes difficult or impossible.

Further, for example, in a parking area or a service area of an expressway, as shown in FIG. 11, the ends 104 and 105 of the two lane markings 102 and 103 may be displaced from each other along the direction of the lane markings 102 and 103. is there. In this case, if the vehicle 101 does not advance further into the parking lot than the position shown in FIG. 11, the detection ranges (areas of interest) in which the ends 104 and 105 of the two lane markings 102 and 103 detect the parking lot. ) Since it is not included in 106 and 107, it is highly possible that the amount of correction of the parking target position at the time when the respective ends 104 and 105 of the two marking lines 102 and 103 are detected again becomes larger.

In view of the above problems, the present invention includes the vehicle side ends of the lane markings in the area even when the vehicle side ends of the two lane markings indicating the parking lot are displaced along the direction of the lane markings. It is an object of the present invention to provide a lane marking detection technique capable of setting a region of interest.

The lane marking detection device according to the present invention includes a detection unit that detects a first lane marking and a second lane marking from a captured image obtained by a camera that captures the periphery of the vehicle, and a vehicle of the first lane marking. The first reference line having an end point corresponding to the side end portion and parallel to the first division line and the end point corresponding to the vehicle side end portion of the second division line are provided on the second division line. A generation unit that generates a second parallel reference line, a position calculation unit that calculates a target parking position based on the end points and directions of the first reference line, and the end points and directions of the second reference line, and the above. The first reference line based on the target parking position, the average direction of the first reference line and the second reference line, and the direction between the end points of the first reference line and the second reference line. The detection unit includes the end point coordinates of the first reference line and the end point coordinate calculation unit for calculating the end point coordinates of the second reference line, and the detection unit includes the end point coordinates of the first reference line calculated by the end point coordinate calculation unit and the above. This is a configuration (first configuration) in which the region of interest is set based on the end point coordinates of the second reference line. The average direction of a plurality of lines can be defined by a vector obtained by normalizing the sum of the normalized direction vectors of each line. For example, the normalized direction vector indicating the average direction of the first reference line and the second reference line includes the normalized direction vector indicating the direction of the first reference line and the second reference. It is a vector obtained by normalizing the sum with the normalized direction vector indicating the direction of the line.

Further, in the lane marking detection device having the first configuration, the end point coordinate calculation unit has a predetermined length from the target parking position toward the vehicle side in the average direction of the first reference line and the second reference line. The distant midpoint coordinates are calculated, and the first reference line is directed toward the first reference line side from the midpoint coordinates in a direction perpendicular to the average direction of the first reference line and the second reference line. Calculate the first coordinates that are half the distance between the end point of the reference line and the second reference line, and in the direction perpendicular to the average direction of the first reference line and the second reference line. A second coordinate that is half the distance between the end point of the first reference line and the second reference line from the point coordinate toward the second reference line side is calculated, and the second coordinate is calculated. A straight line extending in the direction between the end points of the first reference line and the second reference line, and extending in the average direction of the first reference line and the second reference line through the first coordinates. A third coordinate that is an intersection with a straight line is calculated, and a straight line that passes through the first coordinate and extends in a direction between each end point of the first reference line and the second reference line and the second coordinate. The fourth coordinate, which is the intersection of the first reference line and the straight line extending in the average direction of the second reference line, is calculated, and the distance between the third coordinate and the target parking position is the first. When the distance between the coordinate of 4 and the target parking position is smaller, the third coordinate is set as the end point coordinate of the first reference line, the second coordinate is set as the end point coordinate of the second reference line, and the above. When the distance between the third coordinate and the target parking position is larger than the distance between the fourth coordinate and the target parking position, the fourth coordinate is set as the end point coordinate of the second reference line, and the first There may be a configuration (second configuration) in which the coordinates of are used as the end point coordinates of the first reference line. When the distance between the third coordinate and the target parking position is equal to the distance between the fourth coordinate and the target parking position, the third coordinate is used as the end point coordinate of the first reference line. The second coordinate may be used as the end point coordinate of the second reference line, the fourth coordinate may be used as the end point coordinate of the second reference line, and the first coordinate may be used as the end point coordinate of the first reference line. It may be the end point coordinates of.

Further, in the lane marking detection device having the first or second configuration, the region of interest is the first interest set based on the end point coordinates of the first reference line calculated by the end point coordinate calculation unit. The first area of interest and the second area of interest include a region and a second area of interest set based on the end point coordinates of the second reference line calculated by the end point coordinate calculation unit. May be a configuration (third configuration) separated from each other.

Further, in the lane marking detection device having the third configuration, the top view area of the first area of interest and the top view area of the second area of interest are smaller than the top view area of the vehicle (fourth). The configuration of) may be used.

Further, the lane marking detection device having any of the first to fourth configurations includes an output unit that outputs information related to the first reference line and the second reference line to the outside (fifth). Configuration) may be used.

Further, in the lane marking detection device having the fifth configuration, the output unit has the coordinates of the target parking position and the average direction of the first reference line and the second reference line with respect to the front-rear direction of the vehicle. The angle may be output to the outside as information related to the first reference line and the second reference line (sixth configuration).

The lane marking system according to the present invention has a configuration (seventh configuration) including a camera for photographing the periphery of the vehicle and a lane marking device having any of the first to sixth configurations.

The lane marking detection method according to the present invention includes a detection step of detecting a first lane marking and a second lane marking from a captured image obtained by a camera that captures the periphery of the vehicle, and a vehicle of the first lane marking. The first reference line having an end point corresponding to the side end portion and parallel to the first division line and the end point corresponding to the vehicle side end portion of the second division line are provided on the second division line. A generation step of generating a parallel second reference line, a position calculation step of calculating a target parking position based on the end points and directions of the first reference line, and the end points and directions of the second reference line, and the above. The first reference line based on the target parking position, each direction of the first reference line and the second reference line, and the direction between each end point of the first reference line and the second reference line. End point coordinates and end point coordinates of the second reference line are calculated, and end point coordinates of the first reference line and end point coordinates of the second reference line calculated in the end point coordinate calculation step. It is a configuration (eighth configuration) including a step of setting an area of interest based on the above.

According to the lane marking detection technique of the present invention, not only the average direction of the target parking position and the first reference line and the second reference line, but also the direction between each end point of the first reference line and the second reference line. The endpoint coordinates of the first reference line and the endpoint coordinates of the second reference line are calculated based on the above, and the region of interest is set based on the endpoint coordinates of the first reference line and the endpoint coordinates of the second reference line. To. As a result, even if the vehicle-side ends of the two lane markings indicating the parking lot are displaced along the direction of the lane, the area of interest in which the vehicle-side end of the lane is included in the area can be set. Can be done.

The figure which shows the configuration example of the lane marking device Top view of the parking lot Flow chart showing an operation example of the lane marking ECU Top view showing the relationship between the lane marking line and the reference line Flowchart showing the calculation procedure of the target parking position Top view showing the relationship between the reference line and the coordinate candidates of the target parking position Flowchart showing the calculation procedure of the end point coordinates of the reference line Top view showing the relationship between the target parking position and the end point coordinates of the reference line Top view showing the relationship between the reference line and the region of interest Top view showing the relationship between the reference line and the region of interest when the end point coordinates of the reference line are not corrected. Top view showing the relationship between the lane marking and the region of interest in Patent Document 1. Top view showing the relationship between the lane marking and the region of interest in Patent Document 1.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. In the following description, the direction in which the vehicle travels straight from the driver's seat to the steering wheel is referred to as "forward direction". Further, the direction in which the vehicle travels straight from the steering wheel to the driver's seat is referred to as "rear direction". Further, the direction from the right side to the left side of the driver facing forward, which is the direction perpendicular to the straight traveling direction and the vertical straight line of the vehicle, is referred to as "left direction". Further, the direction from the left side to the right side of the driver facing forward, which is the direction perpendicular to the straight traveling direction and the vertical straight line of the vehicle, is referred to as "right direction".

Further, in the present embodiment, a coordinate system (world coordinate system) with a specific point of the vehicle equipped with the lane marking device as the origin is used. The specific point of the vehicle is that it is separated from the front end of the vehicle in the rear direction by the effective length (the length obtained by subtracting the rear overhang from the total length of the vehicle) and is in the center of the vehicle in the left-right direction. The front-rear direction of the vehicle is the Z-axis direction (the rear direction is the positive direction of the Z-axis), and the left-right direction of the vehicle is the X-axis direction (the left direction is the positive direction of the X-axis).

<1. Configuration of lane marking device>
FIG. 1 is a diagram showing a configuration example of a lane marking detection device. The lane marking ECU (Electric Control Unit) 1, which is an example of the lane marking device, is connected to the photographing unit 2 and is also connected to the vehicle control ECU 3 or the like via an in-vehicle network 4 such as CAN (Controller Area Network). To. Then, the lane marking detection system 1 is configured by the lane marking ECU 1 and the photographing unit 2.

The photographing unit 2 includes three cameras 21 to 23. The camera 21 is provided at the rear end of the vehicle. Therefore, the camera 21 is also referred to as a back camera 21. The optical axis of the back camera 21 is along the front-rear direction of the vehicle in a top view. The back camera 21 captures the rear direction of the vehicle. The mounting position of the back camera 21 is preferably at the center of the left and right of the vehicle, but may be slightly deviated from the center of the left and right in the left and right direction.

The camera 22 is provided on the left door mirror of the vehicle. Therefore, the camera 22 is also referred to as a left side camera 22. The optical axis of the left side camera 22 is along the left-right direction of the vehicle in a top view. The left side camera 22 photographs the left direction of the vehicle. The camera 23 is provided on the right door mirror of the vehicle. Therefore, the camera 23 is also referred to as a right side camera 23. The optical axis of the right side camera 23 is along the left-right direction of the vehicle in a top view. The right side camera 23 captures the right direction of the vehicle.

In the present embodiment, the lane marking ECU 1 acquires the captured images of the cameras 21 to 23, but the lane marking ECU 1 excludes only the captured images of the back camera 21 except for the left side camera 22 and the right side camera 23 from the photographing unit 2. May be configured to acquire.

The lane marking ECU 1 includes a video acquisition unit 10, a detection unit 11, a generation unit 12, a position calculation unit 13, and an end point coordinate calculation unit 14. The lane marking ECU 1 can be composed of, for example, a control unit and a storage unit. The control unit is a computer including a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory). The storage unit non-volatilely stores the computer programs and data required for the division line detection ECU 1 to operate as the image acquisition unit 10, the detection unit 11, the generation unit 12, the position calculation unit 13, and the end point coordinate calculation unit 14. .. As the storage unit, for example, EEPROM, flash memory, or the like can be used.

The image acquisition unit 10 continuously acquires analog or digital captured images from the cameras 21 to 23 at a predetermined cycle (for example, a 1/30 second cycle). Then, when the acquired time-continuous captured image (acquired video) is analog, the video acquisition unit 10 converts the analog captured image into a digital captured image (A / D conversion).

The detection unit 11 detects the first division line and the second division line indicating the parking zone from the captured image output from the image acquisition unit 10 by using image processing such as edge extraction. The first lane marking and the second lane marking are drawn with white lines or the like on the road surface of the parking lot. In the top view of the parking lot illustrated in FIGS. 2A to 2D, the portion surrounded by the gray dotted line D1 is detected as the first division line, and the portion surrounded by the gray dotted line D2 is the second. Detected as a lane marking.

In the first detection, the detection unit 11 sets the entire area of the captured image as the region of interest (the range in which the division line can be detected), and in the second and subsequent detections, the first reference calculated by the endpoint coordinate calculation unit 14. The region of interest is set based on the end point coordinates of the line and the end point coordinates of the second reference line.

The generation unit 12 generates a first reference line corresponding to the first lane marking and a second reference line corresponding to the second lane marking.

The position calculation unit 13 calculates the target parking position based on the end points and directions of the first reference line and the end points and directions of the second reference line.

The end point coordinate calculation unit 14 calculates the end point coordinates of the first reference line and the end point coordinates of the second reference line. The details of the calculation method will be described later.

The vehicle control ECU 3 controls, for example, the direction and speed of the vehicle according to the operation by the driver, and controls, for example, the direction of the vehicle according to the information sent from the lane marking ECU 1 via the vehicle-mounted network 4.

The generation unit 12 and the position calculation unit 13 of the lane marking ECU 1 have a function of outputting information related to the first reference line and the second reference line to the vehicle control ECU 3. In the present embodiment, the generation unit 12 outputs the angle of the average direction of the first reference line and the second reference line with respect to the front-rear direction of the vehicle, and the position calculation unit 13 outputs the coordinates of the target parking position to the vehicle control ECU 3. To do. Thereby, for example, when the vehicle is moved to the parking lot, the vehicle control ECU 3 can automatically control the direction of the vehicle (steering angle).

<2. Operation of lane marking device>
FIG. 3 is a flowchart showing an operation example of the lane marking ECU 1 which is an example of the lane marking detection device. When the operation start event of the lane marking ECU 1 occurs, the lane marking ECU 1 starts the operation of the flowchart shown in FIG. Examples of the operation start event of the lane marking ECU 1 include the fact that the position of the shift lever of the vehicle is switched to the reverse range. The photographing unit 2 also starts operating at the same time as the operation of the lane marking ECU 1 starts.

First, the detection unit 11 attempts to detect the first lane marking and the second lane marking (step S1). As described above, in the first detection, the entire area of the captured image is the region of interest, and in the second and subsequent detections, the end point coordinates of the first reference line and the second reference calculated by the end point coordinate calculation unit 14. The region of interest is set based on the end point coordinates of the line.

When the first lane marking and the second lane marking are detected, the generation unit 12 generates a first reference line corresponding to the first lane marking and a second reference line corresponding to the second lane marking. (Step S2). Specifically, the generation unit 12 has an end point corresponding to the vehicle-side end of the first lane, and is parallel to the first lane, and the vehicle-side end of the first reference line and the second lane. Generates a second reference line that has endpoints corresponding to the portions and is parallel to the second section line. Therefore, for example, as shown in FIG. 4A, the vehicle-side end E1 of the first lane and the vehicle-side end E2 of the second lane are in the directions of the first lane and the second lane. When not deviated to, the end point EP1 of the first reference line LN1 and the end point EP2 of the second reference line LN2 are not deviated in the directions of the first lane marking and the second lane marking. On the other hand, for example, as shown in FIG. 4B, the vehicle-side end E1 of the first lane and the vehicle-side end E2 of the second lane are in the directions of the first lane and the second lane. When deviated to, the end point EP1 of the first reference line LN1 and the end point EP2 of the second reference line LN2 are deviated in the directions of the first lane marking and the second lane marking.

In step S3 following step S2, the position calculation unit 13 calculates the target parking position based on the end points and directions of the first reference line and the end points and directions of the second reference line. The specific processing contents of step S3 will be described with reference to the flowchart shown in FIG. 5 and the top view shown in FIG.

First, the position calculation unit 13 has an effective length L0 (the length obtained by subtracting the rear overhang from the total length of the vehicle) from the end point EP1 of the first reference line toward the opposite side of the vehicle in the direction of the first reference line. The first coordinate A1 that is separated is calculated (step S3_1). Next, the position calculation unit 13 calculates the second coordinate A2 which is separated from the end point EP2 of the second reference line in the direction of the second reference line by the effective length L0 toward the side opposite to the vehicle (step). S3_2).

Next, the position calculation unit 13 determines the end points EP1 and the second reference line of the first reference line from the first coordinate A1 toward the second reference line side in the direction perpendicular to the direction of the first reference line. The third coordinate A3, which is half the distance W from, is calculated, and the distance W is calculated from the first coordinate A1 in the direction perpendicular to the direction of the first reference line toward the side opposite to the second reference line. The fourth coordinate A4, which is half apart from the above, is calculated (step S3_3). Further, the position calculation unit 13 calculates the fifth coordinate A5, which is half the distance W from the second coordinate A2 toward the first reference line side in the direction perpendicular to the direction of the second reference line. , Calculate the sixth coordinate A6, which is half the distance W from the second coordinate A2 in the direction perpendicular to the direction of the second reference line toward the side opposite to the first reference line (step S3_4). ..

Then, the position calculation unit 13 selects the two coordinates that minimize the distance between the two points from the third coordinates A3 to the sixth coordinates A6 (step S3_5), and is closer to the vehicle among the two selected coordinates. The coordinates (the coordinates with the smaller distance from the origin) are set as the coordinates of the target parking position (step S3_6), and the calculation process of the coordinates of the target parking position is completed.

By processing the flowchart shown in FIG. 5, the coordinates of the target parking position are set to the back by the effective length L0 in the depth direction of the parking lot from the vehicle side end (entrance of the parking lot) of the first and second lanes. It can be entered and located at the center of the first and second lane markings.

Returning to FIG. 3, the operation after the calculation process of the coordinates of the target parking position is completed will be described. In step S4 following step S3, the end point coordinate calculation unit 14 calculates the end point coordinates of the first reference line and the end point coordinates of the second reference line. The specific processing contents of step S4 will be described with reference to the flowchart shown in FIG. 7 and the top view shown in FIG.

First, the end point coordinate calculation unit 14 calculates the midpoint coordinate Pw that is separated from the target parking position A0 toward the vehicle side by the effective length L0 in the average direction of the first reference line LN1 and the second reference line LN2 ( Step S4-1).

Next, the end point coordinate calculation unit 14 first refers from the midpoint coordinate Pw toward the first reference line LN1 side in a direction perpendicular to the average direction of the first reference line LN1 and the second reference line LN2. The end point coordinates P1 of the first reference line LN1 which is half the distance W between the end point EP1 of the line LN1 and the second reference line LN2 are calculated (step S4_2).

Next, the end point coordinate calculation unit 14 increases the distance W from the midpoint coordinate Pw toward the second reference line LN2 in a direction perpendicular to the average direction of the first reference line LN1 and the second reference line LN2. The end point coordinates P2 of the second reference line LN2 that is separated are calculated (step S4_3).

Next, the end point coordinate calculation unit 14 passes through the end point coordinates P2 of the second reference line LN2 and extends in the direction between the end points of the first reference line LN1 and the second reference line LN2 (between the end points EP1 and EP2). The corrected endpoint coordinates of the first reference line LN1, which is the intersection of the straight line and the straight line extending in the average direction of the first reference line LN1 and the second reference line LN2 through the endpoint coordinates P1 of the first reference line LN1. Calculate P1'(step S4_4).

Next, the endpoint coordinate calculation unit 14 has a straight line extending in the direction between the endpoints of the first reference line LN1 and the second reference line LN2 through the endpoint coordinates P1 of the first reference line LN1 and the second reference. Calculate the corrected endpoint coordinates P2'of the second reference line LN2, which is the intersection of the first reference line LN1 and the straight line extending in the average direction of the second reference line LN2 through the endpoint coordinates P2 of the line LN2 (step). S4_5).

Then, the end point coordinate calculation unit 14 calculates the distance L1 between the corrected end point coordinates P1'of the first reference line LN1 and the coordinates A0 of the target parking position (step S4_6), and corrects the second reference line LN2. The distance L2 between the end point coordinates P2'and the target parking position coordinate A0 is calculated (step S4_7), and it is determined whether or not the distance L1 is equal to or less than the distance L2 (step S4_8). In addition, unlike this embodiment, the case may be divided according to whether or not the distance L1 is smaller than the distance L2.

When the distance L1 is less than or equal to the distance L2, the corrected endpoint coordinates P1'of the first reference line LN1 are adopted as the endpoint coordinates of the first reference line LN1, and the endpoint coordinates P2 of the second reference line LN2 are used as the second. It is adopted as the end point coordinates of the reference line LN2 of (step S4_9), and the calculation process of the end point coordinates is completed.

On the other hand, when the distance L1 is larger than the distance L2, the corrected endpoint coordinates P2'of the second reference line LN2 are adopted as the endpoint coordinates of the second reference line LN2, and the endpoint coordinates P1 of the first reference line LN1 are used. It is adopted as the end point coordinates of the reference line LN1 of 1 (step S4_10), and the calculation process of the end point coordinates is completed.

According to the calculation process of the end point coordinates described above, not only in the average direction of the target parking position and the first reference line and the second reference line, but also in the direction between the end points of the first reference line and the second reference line. The end point coordinates of the first reference line and the end point coordinates of the second reference line are calculated based on the above. Then, in the second and subsequent detections of the first lane marking and the second lane marking, the region of interest is set based on the end point coordinates of the first reference line and the end point coordinates of the second reference line. As a result, even if the vehicle-side ends of the two lane markings indicating the parking lot are displaced along the direction of the lane, the area of interest in which the vehicle-side end of the lane is included in the area can be set. Can be done.

Further, according to the calculation process of the end point coordinates described above, the vehicle-side end of the first lane and the vehicle-side end of the second lane are displaced in the directions of the first lane and the second lane. The end point coordinates can be calculated by the same calculation process without distinguishing between the case where the case is and the case where the position is not deviated.

Further, according to the calculation process of the end point coordinates described above, only the one having the smaller distance from the target parking position among the two corrected end point coordinates is adopted as the end point coordinates (see steps S4_8 to S4_10). This makes it consistent with the coordinate determination process of the target parking position in step S3_6 described above. Specifically, the distance from the intersection of the straight line perpendicular to the first reference line and the first reference line through the coordinates of the target parking position to the corrected end point coordinates of the first reference line exceeds the effective length L0. If not, the corrected end point coordinates of the first reference line are adopted, and the correction of the second reference line from the intersection of the straight line perpendicular to the second reference line and the second reference line passing through the coordinates of the target parking position. When the distance to the end point coordinates does not exceed the effective length L0, the corrected end point coordinates of the second reference line are adopted. As a result, the end point coordinates of the first reference line can be set to a position close to the vehicle side end of the first lane line as aimed, and the end point coordinates of the second reference line can be set to the vehicle side of the second lane line as aimed. Can be set near the edge.

Returning to FIG. 3, the operation after the calculation process of the end point coordinates is completed will be described. In step S5 following step S4, the detection unit 11 sets the region of interest based on the end point coordinates of the first reference line and the end point coordinates of the second reference line calculated by the end point coordinate calculation unit 14. As a result, the range of the region of interest can be narrowed, so that erroneous detection by the detection unit 11 can be reduced.

As shown in FIG. 9A, for example, the detection unit 11 sets a first region of interest R1 including the end point coordinates P1 or P1'of the first reference line LN1 calculated by the end point coordinate calculation unit 14, and the end point coordinates. A second region of interest R2 including the end point coordinates P2 or P2'of the second reference line LN2 calculated by the calculation unit 14 is set.

In the example shown in FIG. 9A, the first region of interest R1 and the second region of interest R2 are separated from each other. As a result, the range of the region of interest can be further narrowed, so that erroneous detection by the detection unit 11 can be further reduced. Then, in order to effectively reduce the false detection in the detection unit 11, the top view area of the first interest region R1 and the top view area of the second interest region R2 are at least larger than the top view area of the vehicle. It is desirable to keep it small. Further, in the example shown in FIG. 9A, the top view shape of the first region of interest R1 is rectangular, the longitudinal direction of the first region of interest R1 is made to coincide with the direction of the first reference line LN1, and the second region of interest R1. The top view shape of R2 is rectangular, and the longitudinal direction of the second region of interest R2 coincides with the direction of the second reference line LN2. By devising this shape, the top view area of the first area of interest R1 and the top view area of the second area of interest R2 are reduced.

If the above steps S4_4 to S4_10 are not performed, the end point coordinates of the first reference line LN1 calculated by the end point coordinate calculation unit 14 will always be the coordinates P1, and the end point coordinates calculation unit 14 will calculate the second. Since the end point coordinates of the reference line LN2 of 2 are always the coordinates P2, the state shown in FIG. 9B may occur, and the detection unit 11 may not be able to detect the first lane marking in the second and subsequent detections. There is.

In step S6 following step S5, the lane marking ECU 1 confirms whether or not the end event of the lane marking ECU 1 has occurred. The operation end event of the lane marking ECU 1 includes, for example, that the distance between the coordinates of the target parking position and the origin is equal to or less than a predetermined value, which is substantially zero, and the position of the shift lever of the vehicle is changed from the reverse range to another range. It can be mentioned that it has been switched.

If the operation end event of the lane marking ECU 1 has not occurred, the process returns to step S1. On the other hand, if the operation end event of the lane marking ECU 1 has occurred, the lane marking ECU 1 ends the operation of the flowchart shown in FIG.

Instead of executing step S6, the lane marking ECU 1 constantly monitors whether or not the operation end event of the lane marking ECU 1 has occurred during the operation of the flowchart, and the operation end event of the lane marking ECU 1 occurs. If so, the operation of the flowchart may be terminated immediately. In this case, when the process of step S5 is completed, the process may return to step S1.

<3. Notes>
In addition to the above-described embodiment, various technical features disclosed in the present specification can be modified in various ways without departing from the spirit of the technical creation. That is, it should be considered that the above-described embodiment is exemplary in all respects and is not restrictive, and the technical scope of the present invention is not the description of the above-mentioned embodiment but the scope of claims. It is shown and should be understood to include all modifications that fall within the meaning and scope of the claims.

For example, the lane marking ECU 1 outputs information related to the first reference line and the second reference line to the display device, and the display device superimposes the first reference line and the second reference line on the camera image. May be displayed. This superimposed display allows the driver of the vehicle to clearly recognize the parking lot.

Further, under the assumption that the direction of the first reference line and the direction of the second reference line are completely the same, the direction of the first reference line or the direction of the second reference line is set as the first reference. The division line detection ECU 1 may execute various processes by regarding the average direction of the line and the second reference line.

1 lane marking ECU
10 Video acquisition unit 11 Detection unit 12 Generation unit 13 Position calculation unit 14 End point coordinate calculation unit 2 Imaging unit 21-23 Camera 3 Vehicle control ECU
4 In-vehicle network

Claims (8)

A detection unit that detects the first section line and the second section line from the captured image obtained by the camera that captures the surroundings of the vehicle.
It has an end point corresponding to the vehicle side end of the first section line, and has a first reference line parallel to the first section line and an end point corresponding to the vehicle side end of the second section line. A generator that generates a second reference line parallel to the second section line,
A position calculation unit that calculates a target parking position based on the end points and directions of the first reference line and the end points and directions of the second reference line.
The first reference based on the target parking position, the average direction of the first reference line and the second reference line, and the direction between the end points of the first reference line and the second reference line. It is provided with an end point coordinate calculation unit for calculating the end point coordinates of the line and the end point coordinates of the second reference line.
The detection unit sets a region of interest based on the end point coordinates of the first reference line and the end point coordinates of the second reference line calculated by the end point coordinate calculation unit. .. The end point coordinate calculation unit
The midpoint coordinates that are separated by a predetermined length from the target parking position toward the vehicle side in the average direction of the first reference line and the second reference line are calculated.
The end point of the first reference line and the second reference from the midpoint coordinates in a direction perpendicular to the average direction of the first reference line and the second reference line toward the first reference line side. Calculate the first coordinate, which is half the distance from the line,
The end point of the first reference line and the second reference from the midpoint coordinates in a direction perpendicular to the average direction of the first reference line and the second reference line toward the second reference line side. Calculate the second coordinate, which is half the distance from the line,
A straight line extending in the direction between the end points of the first reference line and the second reference line passing through the second coordinates, and the first reference line and the second reference line passing through the first coordinates. Calculate the third coordinate, which is the intersection with the straight line extending in the average direction of the line,
A straight line extending in the direction between the end points of the first reference line and the second reference line passing through the first coordinates, and the first reference line and the second reference line passing through the second coordinates. Calculate the fourth coordinate, which is the intersection with the straight line extending in the average direction of the line,
When the distance between the third coordinate and the target parking position is smaller than the distance between the fourth coordinate and the target parking position, the third coordinate is set as the end point coordinate of the first reference line, and the third coordinate is used. Let the coordinates of 2 be the end point coordinates of the second reference line.
When the distance between the third coordinate and the target parking position is larger than the distance between the fourth coordinate and the target parking position, the fourth coordinate is set as the end point coordinate of the second reference line, and the second. The lane marking detection device according to claim 1, wherein the coordinates of 1 are set to the end point coordinates of the first reference line. The region of interest is a first region of interest set based on the end point coordinates of the first reference line calculated by the end point coordinate calculation unit, and the second reference calculated by the end point coordinate calculation unit. Includes a second region of interest, which is set based on the end point coordinates of the line,
The lane marking detection device according to claim 1 or 2, wherein the first region of interest and the second region of interest are separated from each other. The lane marking detection device according to claim 3, wherein the top view area of the first area of interest and the top view area of the second area of interest are each smaller than the top view area of the vehicle. The lane marking detection device according to any one of claims 1 to 4, further comprising an output unit that outputs information related to the first reference line and the second reference line to the outside. The output unit sets the coordinates of the target parking position and the angle of the average direction of the first reference line and the second reference line with respect to the front-rear direction of the vehicle to the first reference line and the second reference line. The lane marking detection device according to claim 5, which outputs information related to the reference line of the above to the outside. A camera that shoots the area around the vehicle,
A lane marking detection system comprising the lane marking detection device according to any one of claims 1 to 6. A detection process that detects the first section line and the second section line from the captured image obtained by the camera that photographs the surroundings of the vehicle, and
It has an end point corresponding to the vehicle side end of the first section line, and has a first reference line parallel to the first section line and an end point corresponding to the vehicle side end of the second section line. A generation step of generating a second reference line parallel to the second section line, and
A position calculation step of calculating a target parking position based on the end points and directions of the first reference line and the end points and directions of the second reference line.
The first reference based on the target parking position, each direction of the first reference line and the second reference line, and the direction between the end points of the first reference line and the second reference line. The end point coordinate calculation step of calculating the end point coordinates of the line and the end point coordinates of the second reference line, and
A lane marking detection method comprising a step of setting an area of interest based on the end point coordinates of the first reference line and the end point coordinates of the second reference line calculated in the end point coordinate calculation step.