JP2010245802A - Control support device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a control support device capable of reducing loads for control support. <P>SOLUTION: In the control support device, each of cameras C_1-C_4 is provided on a vehicle in an obliquely downward posture and captures a field around the vehicle. A CPU 12p converts a field image output from each of the cameras C_1-C_4 to a bird's-eye view image, and extracts an edge from the converted bird's-eye view image. The CPU 12p also detects the angle information of a line drawn by the extracted edge corresponding to a plurality of positions on the extracted edge, and executes control support processing such as warning corresponding to predetermined angle information for which the number of times of detection exceeds a reference in the detected angle information. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a steering assistance device, and in particular, assists a pilot's operation based on an object scene image output from a camera that is provided in a slanting downward posture on a moving object and captures the object scene around the moving object. The present invention relates to a steering assist device.

  An example of this type of device is disclosed in Patent Document 1. According to this background art, the road surface image output from the camera mounted in the front part of the vehicle is converted into an overhead view image by the overhead view conversion unit. The intensity profile creation unit integrates the luminance in the direction orthogonal to the road width direction at a plurality of positions in the road width direction of the road surface appearing in the overhead image. The lane marker detection unit detects the position of the lane marker installed on the road surface based on the integrated luminance distribution detected by the intensity profile creation unit. Thus, an alarm can be promptly generated when the vehicle departs from the lane.

JP 2004-145852 A

  However, in the background art, it is necessary to integrate the luminance in the direction orthogonal to the road width direction at a plurality of positions in the road width direction, and there is a concern about overload.

  Therefore, a main object of the present invention is to provide a steering assistance device that can reduce a load for steering assistance.

  The steering assist device according to the present invention (10: reference numeral corresponding to the embodiment; the same applies hereinafter) is provided on the moving body (100) in an obliquely downward posture and captures an object scene around the moving body (C_1 ~ C_4) conversion means (S5) for converting the scene image output to the bird's-eye image, extraction means (S15) for extracting an edge from the bird's-eye image converted by the conversion means, and by the edge extracted by the extraction means Detection means (S17 to S25) for detecting the angle information of the drawn line corresponding to a plurality of positions on the edge extracted by the extraction means, and the number of detections out of the angle information detected by the detection means exceeds the reference Processing means (S31, S35, S51, S53) for executing a steering support process corresponding to the specific angle information is provided.

  Preferably, the steering support process includes a warning process (S35) for generating a warning toward the pilot.

  Preferably, the steering support process includes an exclusion process (S53) that excludes an edge that draws a line having specific angle information from a discrimination target for obstacle detection.

  Preferably, the detection means includes a creation means (S21) for creating a histogram representing the distribution state of the angle information, and the processing means executes the steering support process with reference to the histogram created by the creation means.

  Preferably, the imaging unit includes a plurality of cameras (C_1 to C_4) each outputting an object scene image, and updates the target camera for extraction processing of the extraction unit in a cyclic manner between the plurality of cameras. Means (S13, S39, S41) are further provided.

  The steering support method according to the present invention is a steering support method executed by the steering support device (10), and is provided on the moving body (100) in an obliquely downward posture and captures an object scene around the moving body. A conversion step (S5) for converting the object scene image output from the means (C_1 to C_4) into a bird's-eye image, an extraction step (S15) for extracting an edge from the bird's-eye image converted by the conversion step, and an extraction step. The detection step (S17 to S25) for detecting the angle information of the line drawn by the edge corresponding to a plurality of positions on the edge extracted by the extraction step, and the number of detections among the angle information detected by the detection step Processing steps (S31, S35, S51, S53) for executing the steering support process corresponding to the specific angle information exceeding the reference are provided.

  According to the present invention, the angle information detected by the detecting means coincides with each other when a plurality of positions on the edge drawing a straight line are designated. Further, the number of detections of the same angle information increases as the length of the straight line drawn by the edge increases. Therefore, the steering assist process is executed when a linear road surface paint having a length exceeding the length corresponding to the reference is captured by the imaging unit. The angle information of the line drawn by the edge is detected at a plurality of positions on the edge, and the steering support process is executed corresponding to the angle information whose number of detection exceeds the reference. As a result, the load for steering assistance can be reduced.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

It is a block diagram which shows the basic composition of this invention. It is a block diagram which shows the structure of one Example of this invention. (A) is an illustrative view showing a state of looking at the front of the vehicle, (B) is an illustrative view showing a state of looking at the right side of the vehicle, and (C) shows a state of looking at the back of the vehicle. It is an illustration figure, (D) is an illustration figure which shows the state which looked at the left side surface of the vehicle. It is an illustration figure which shows an example of the visual field caught by the some camera attached to the vehicle. (A) is an illustrative view showing an example of a bird's-eye view image based on the output of the previous camera, (B) is an illustrative view showing an example of a bird's-eye view image based on the output of the right camera, and (C) is an output of the left camera. It is an illustration figure which shows an example of the bird's-eye view image based on, and (D) is an illustration figure which shows an example of the bird's-eye view image based on the output of a back camera. FIG. 6 is an illustrative view showing one example of an all-around bird's-eye image based on the bird's-eye image shown in FIGS. 5 (A) to 5 (D). It is an illustration figure which shows an example of the steering assistance image displayed by a display apparatus. It is an illustration figure which shows the angle of the camera attached to the vehicle. It is an illustration figure which shows the relationship between a camera coordinate system, the coordinate system of an imaging surface, and a world coordinate system. It is a perspective view which shows an example of the road surface paint of a vehicle and its vicinity. It is an illustration figure which shows another example of an all-around bird's-eye view image. (A) is an illustration figure which shows a part of reproduction image, (B) is an illustration figure which shows a part of difference image corresponding to the reproduction image shown to (A). FIG. 13 is an illustrative view showing one example of a normal vector defined on the difference image shown in FIG. It is a histogram which shows an example of the distribution state of the normal vector with respect to an angle direction. It is a flowchart which shows a part of operation | movement of CPU applied to the FIG. 2 Example. It is a flowchart which shows a part of other operation | movement of CPU applied to the FIG. 2 Example. FIG. 11 is a flowchart showing still another portion of behavior of the CPU applied to the embodiment in FIG. 2; It is a flowchart which shows a part of operation | movement of CPU applied to another Example. It is a perspective view which shows an example of a vehicle, the obstacle of the vicinity, and road surface paint. It is an illustration figure which shows another example of an all-around bird's-eye view image.

[Basic configuration]
Referring to FIG. 1, the steering assist device of the present invention is basically configured as follows. The imaging unit 1 is provided on the moving body in an obliquely downward posture, and captures an object scene around the moving body. The conversion unit 2 converts the object scene image output from the imaging unit 1 into a bird's-eye view image. The extraction unit 3 extracts an edge from the bird's-eye view image converted by the conversion unit 2. The detection unit 4 detects angle information of a line drawn by the edge extracted by the extraction unit 3 corresponding to a plurality of positions on the edge extracted by the extraction unit 3. The processing unit 5 executes the steering support process corresponding to the specific angle information whose number of detection exceeds the reference among the angle information detected by the detection unit 4.

  Therefore, the angle information detected by the detection unit 4 matches each other when a plurality of positions on the edge drawing a straight line are designated. Further, the number of detections of the same angle information increases as the length of the straight line drawn by the edge increases. Therefore, the steering support process is executed when a linear road surface paint having a length exceeding the length corresponding to the reference is captured by the imaging unit 1.

The angle information of the line drawn by the edge is detected at a plurality of positions on the edge, and the steering support process is executed corresponding to the angle information whose number of detection exceeds the reference. As a result, the load for steering assistance can be reduced.
[Example]

  The steering assistance device 10 of this embodiment shown in FIG. 2 includes four cameras C_1 to C_4. The cameras C_1 to C_4 output scene images P_1 to P_4 every 1/30 seconds in response to the common vertical synchronization signal Vsync. The output scene images P_1 to P_4 are captured by the image processing circuit 12. The captured object scene images P_1 to P_4 are written in the work areas F1 to F4 of the SDRAM 12m, respectively.

  Referring to FIGS. 3A to 3D, the steering assist device 10 is mounted on a vehicle 100 that travels on the ground. Specifically, the camera C_1 is installed in a substantially center of the front portion of the vehicle 100 in a posture facing obliquely downward in front of the vehicle 100. The camera C_2 is installed in a posture that faces obliquely downward to the right of the vehicle 100, approximately in the center in the width direction on the right side of the vehicle 100 and on the upper side in the height direction.

  The camera C_3 is installed in an approximately center in the width direction of the rear portion of the vehicle 100 and on the upper side in the height direction with the posture facing the diagonally downward rearward direction of the vehicle 100. The camera C_4 is installed on the left side of the vehicle 100 substantially in the center in the width direction and on the upper side in the height direction with the posture facing the diagonally lower left side of the vehicle 100.

  FIG. 4 shows a bird's-eye view of the vehicle 100 and the surrounding ground. According to FIG. 4, the camera C_1 has a field of view VW_1 that captures the front of the vehicle 100, the camera C_2 has a field of view VW_2 that captures the right direction of the vehicle 100, and the camera C_3 has a field of view VW_3 that captures the rear of the vehicle 100. The camera C_4 has a field of view VW_4 that captures the left direction of the vehicle 100. Also, the visual fields VW_1 and VW_2 have a common visual field VW_12, the visual fields VW_2 and VW_3 have a common visual field VW_23, the visual fields VW_3 and VW_4 have a common visual field VW_34, and the visual fields VW_4 and VW_1 have a common visual field VW_41.

  Returning to FIG. 2, the CPU 12p provided in the image processing circuit 12 generates the bird's-eye view image BEV_1 shown in FIG. 5A based on the object scene image P_1 stored in the work area F1, and the work area F2 is displayed in the work area F2. Based on the stored scene image P_2, a bird's-eye view image BEV_2 shown in FIG. 5B is generated. The CPU 12p also generates the bird's-eye view image BEV_3 shown in FIG. 5C based on the object scene image P_3 stored in the work area F3, and based on the object scene image P_4 stored in the work area F4. A bird's-eye view image BEV_4 shown in (D) is generated. Bird's-eye view images BEV_1 to BEV_4 are also stored in work areas F1 to F4.

  The bird's-eye view image BEV_1 corresponds to an image captured by the virtual camera looking down the visual field VW_1 in the vertical direction, and the bird's-eye view image BEV_2 corresponds to an image captured by the virtual camera looking down the visual field VW_2 in the vertical direction. The bird's-eye view image BEV_3 corresponds to an image captured by a virtual camera looking down the visual field VW_3 in the vertical direction, and the bird's-eye view image BEV_4 corresponds to an image captured by a virtual camera looking down the visual field VW_4 in the vertical direction.

  According to FIGS. 5A to 5D, the bird's-eye image BEV_1 has a bird's-eye coordinate system X1 and Y1, the bird's-eye image BEV_2 has a bird's-eye coordinate system X2 and Y2, and the bird's-eye image BEV_3 has a bird's-eye coordinate system. The bird's-eye view image BEV_4 has a bird's-eye coordinate system X4 / Y4.

  Subsequently, the CPU 12p rotates and / or moves the bird's-eye view images BEV_2 to BEV_4 with reference to the bird's-eye view image BEV_1 in order to combine the bird's-eye view images BEV_1 to BEV_4. The coordinates of the bird's-eye view images BEV_2 to BEV_4 are converted on the work areas F2 to F4 so as to draw the all-around bird's-eye view image shown in FIG.

  In FIG. 6, the overlapping area OL_12 corresponds to an area that reproduces the common visual field VW_12, and the overlapping area OL_23 corresponds to an area that reproduces the common visual field VW_23. The overlapping area OL_34 corresponds to an area that reproduces the common visual field VW_34, and the overlapping area OL_41 corresponds to an area that reproduces the common visual field VW_41.

  Further, the unique area OR_1 corresponds to an area that reproduces a part of the visual field VW1 excluding the common visual fields VW_41 and VW_12, and the unique area OR_2 reproduces a part of the visual field VW2 except the common visual fields VW_12 and VW_23. Corresponds to the area to be The unique area OR_3 corresponds to an area that reproduces a part of the visual field VW3 excluding the common visual fields VW_23 and VW_34, and the unique area OR_4 reproduces a part of the visual field VW4 except the common visual fields VW_34 and VW_41. Corresponds to the area to be

  The display device 14 installed in the driver's seat of the vehicle 100 defines the section BK1 where the overlapping areas OL_12 to OL_41 are located at the four corners, and displays some bird's-eye images belonging to the defined section BLK1 in each of the work areas F1 to F4. Read from. The display device 14 further combines the read bird's-eye images with each other, and pastes a graphic image G1 simulating the upper part of the vehicle 100 at the center of the all-round bird's-eye image obtained thereby. As a result, the steering assistance image shown in FIG. 7 is displayed on the monitor screen.

  Next, how to create the bird's-eye view images BEV_1 to BEV_4 will be described. However, since the bird's-eye images BEV_1 to BEV_4 are all created in the same manner, the creation procedure of the bird's-eye image BEV3 will be described as a representative of the bird's-eye images BEV_1 to BEV_4.

  Referring to FIG. 8, camera C_3 is disposed rearward and obliquely downward at the rear of vehicle 100. If the depression angle of the camera C_3 is “θd”, the angle θ shown in FIG. 8 corresponds to “180 ° −θd”. The angle θ is defined in the range of 90 ° <θ <180 °.

  FIG. 9 shows the relationship between the camera coordinate system X · Y · Z, the coordinate system Xp · Yp of the imaging surface S of the camera C_3, and the world coordinate system Xw · Yw · Zw. The camera coordinate system X, Y, Z is a three-dimensional coordinate system with the X, Y, and Z axes as coordinate axes. The coordinate system Xp · Yp is a two-dimensional coordinate system having the Xp axis and the Yp axis as coordinate axes. The world coordinate system Xw · Yw · Zw is a three-dimensional coordinate system having the Xw axis, the Yw axis, and the Zw axis as coordinate axes.

  In the camera coordinate system X, Y, Z, the optical center of the camera C3 is defined as the origin O, the Z axis is defined in the optical axis direction, the X axis is defined in the direction perpendicular to the Z axis and parallel to the ground, and A Y axis is defined in a direction orthogonal to the Z axis and the X axis. In the coordinate system Xp / Yp of the imaging surface S, with the center of the imaging surface S as the origin, the Xp axis is defined in the horizontal direction of the imaging surface S, and the Yp axis is defined in the vertical direction of the imaging surface S.

  In the world coordinate system Xw / Yw / Zw, the intersection of the vertical line passing through the origin O of the camera coordinate system XYZ and the ground is defined as the origin Ow, and the Yw axis is defined in the direction perpendicular to the ground. An Xw axis is defined in a direction parallel to the X axis of Z, and a Zw axis is defined in a direction orthogonal to the Xw axis and the Yw axis. The distance from the Xw axis to the X axis is “h”, and the obtuse angle formed by the Zw axis and the Z axis corresponds to the angle θ described above.

  When coordinates in the camera coordinate system X, Y, and Z are expressed as (x, y, z), “x”, “y”, and “z” are X-axis components in the camera coordinate system X, Y, and Z, respectively. A Y-axis component and a Z-axis component are shown. When the coordinates in the coordinate system Xp / Yp of the imaging surface S are expressed as (xp, yp), “xp” and “yp” respectively represent the Xp-axis component and the Yp-axis component in the coordinate system Xp / Yp of the imaging surface S. Show. When coordinates in the world coordinate system Xw · Yw · Zw are expressed as (xw, yw, zw), “xw”, “yw”, and “zw” are Xw axis components in the world coordinate system Xw · Yw · Zw, The Yw axis component and the Zw axis component are shown.

A conversion formula between the coordinates (x, y, z) of the camera coordinate system X, Y, Z and the coordinates (xw, yw, zw) of the world coordinate system Xw, Yw, Zw is expressed by Formula 1.

Here, assuming that the focal length of the camera C_3 is “f”, the coordinates (xp, yp) of the coordinate system Xp / Yp of the imaging surface S and the coordinates (x, y, z) of the camera coordinate system X / Y / Z are The conversion formula between is expressed by Equation 2.

Also, Equation 3 is obtained based on Equation 1 and Equation 2. Equation 3 shows a conversion formula between the coordinates (xp, yp) of the coordinate system Xp / Yp of the imaging surface S and the coordinates (xw, zw) of the two-dimensional ground coordinate system Xw / Zw.

  Also, a bird's-eye coordinate system X3 / Y3, which is a coordinate system of the bird's-eye image BEV_3 shown in FIG. 5C, is defined. The bird's-eye coordinate system X3 / Y3 is a two-dimensional coordinate system having the X3 axis and the Y3 axis as coordinate axes. When the coordinates in the bird's-eye view coordinate system X3 / Y3 are expressed as (x3, y3), the position of each pixel forming the bird's-eye view image BEV_3 is represented by the coordinates (x3, y3). “X3” and “y3” respectively indicate an X3 axis component and a Y3 axis component in the bird's eye view coordinate system X3 · Y3.

Projection from the two-dimensional coordinate system Xw / Zw representing the ground onto the bird's-eye coordinate system X3 / Y3 corresponds to so-called parallel projection. If the height of the virtual camera, that is, the virtual viewpoint is “H”, the conversion formula between the coordinates (xw, zw) of the two-dimensional coordinate system Xw · Zw and the coordinates (x3, y3) of the bird's-eye coordinate system X3 · Y3 is , Represented by Equation (4). The height H of the virtual camera is determined in advance.

Further, based on Equation 4, Equation 5 is obtained, Equation 5 is obtained based on Equation 5 and Equation 3, and Equation 7 is obtained based on Equation 6. Equation 7 corresponds to a conversion formula for converting the coordinates (xp, yp) of the coordinate system Xp / Yp of the imaging surface S into the coordinates (x3, y3) of the bird's eye coordinate system X3 / Y3.

  The coordinates (xp, yp) of the coordinate system Xp / Yp of the imaging surface S represent the coordinates of the object scene image P_3 captured by the camera C_3. Accordingly, the object scene image P_3 from the camera C3 is converted into the bird's-eye view image BEV_3 by using Equation 7. Actually, the object scene image P_3 is first subjected to image processing such as lens distortion correction, and then converted into a bird's-eye view image BEV_3 by Equation 7.

  When the road surface paint 200 such as a pedestrian crossing is drawn on the ground around the vehicle 100 in the manner shown in FIG. 10, the all-around bird's-eye view image shown in FIG. 11 is created corresponding to the section BK1. .

  In the following description, a part of the all-around bird's-eye view image shown in FIG. 11 that is reproduced corresponding to the unique area OR_1 shown in FIG. 6 is defined as “reproduced image REP_1”, and the unique area shown in FIG. A part of the image reproduced corresponding to OR_2 is defined as “reproduced image REP_2”. Similarly, of the bird's-eye view image shown in FIG. 11, a part of the image reproduced corresponding to the unique area OR_3 shown in FIG. 6 is defined as “reproduced image REP_3” and corresponds to the unique area OR_4 shown in FIG. A part of the reproduced image is defined as “reproduced image REP_4”.

  In the image processing circuit 12, the variable M is set to each of “1” to “4” in response to the vertical synchronization signal Vsync, and the following processing is executed corresponding to each numerical value.

  First, a difference image DEF_M representing a difference between frames of the reproduced image REP_M is created by the difference calculation process. As a result, for the reproduced image REP_2 shown in FIG. 12A, a differential image DEF_2 shown in FIG. 12B is created.

  When the vehicle 100 is moving, an alignment process for performing an alignment considering the movement of the vehicle 100 between the difference image REP_L of the previous frame and the difference image REP_L of the current frame is executed prior to the difference calculation process. The Therefore, a bird's-eye view image representing a planar pattern such as the road surface paint 200 is in principle matched between frames by the alignment process. However, in reality, an error in conversion processing into a bird's-eye view image and an error in alignment between frames occur, and thus a high-luminance component corresponding to an edge (contour) of the road surface paint 200 appears in the difference image DEF_2.

  Subsequently, a certain coordinate on the edge appearing in the difference image DEF_M is designated, and a normal vector is assigned to the designated coordinate. The normal vector is a vector orthogonal to a line that draws an edge at designated coordinates, and corresponds to angle information at the designated coordinates of a line that draws an edge. The coordinate designation and the normal vector assignment are executed by paying attention to a plurality of coordinates on the edge. Therefore, for the differential image DEF_2 shown in FIG. 12B, a plurality of normal vectors are assigned in the manner shown in FIG.

  The distribution state in the angular direction of the normal vector thus assigned is represented by a histogram. When a normal vector is assigned as shown in FIG. 13, the histogram shows the characteristics shown in FIG. According to FIG. 14, the number of normal vector assignments increases rapidly at two angles respectively corresponding to two straight lines that describe the edge of the road surface paint.

  When the histogram is completed, the number of normal vector assignments corresponding to the angle θ_N (N: 1, 2,... Nmax) is detected as “T_N”. The detected number of assignments T_N is compared with the reference value REF, and when an angle θ_N that satisfies T_N> REF is detected, a warning is generated from the speaker 16 for the steering assist process. Note that the difference between the angles θ_N and θ_N−1 corresponds to, for example, 1 °.

  Specifically, the CPU 12p executes a plurality of tasks including an image creation task shown in FIG. 15 and a road surface paint detection task shown in FIGS. 16 to 17 in parallel. Note that control programs corresponding to these tasks are stored in the flash memory 18 (see FIG. 2).

  Referring to FIG. 15, when the vertical synchronization signal Vsync is generated, the process proceeds from step S1 to step S3, and the object scene images P_1 to P_4 are captured from the cameras C_1 to C_4, respectively. The captured scene images P_1 to P_4 are stored in the work areas F1 to F4, respectively. In step S5, bird's-eye view images BEV_1 to BEV_4 are created based on the captured object scene images P_1 to P_4. In step S7, coordinate conversion is performed on the bird's-eye images BEV_2 to BEV_4, and the bird's-eye images BEV_1 to BEV_4 are combined with each other. On the monitor screen of the display device 16, a part of the all-around bird's-eye view image combined by coordinate transformation and the graphic image G <b> 1 multiplexed thereon are displayed as a steering assistance image. When the process of step S7 is completed, the process returns to step S1.

  Referring to FIG. 16, in step S11, it is determined whether or not the vertical synchronization signal Vsync is generated. When the determination result is updated from NO to YES, the variable M is set to “1” in step S13. In step S15, a difference calculation process is executed to create a difference image DEF_M representing a difference between the reproduced image REP_M of the previous frame and the difference image REP_M of the current frame. In the difference image DEF_M, a high luminance component corresponding to the edge (contour component) of the object captured by the camera C_M appears.

  In step S17, a certain coordinate on the edge appearing in the difference image DEF_M is designated, and in step S19, a normal vector is assigned to the designated coordinate. In step S21, a histogram is created or updated with reference to the angle of the normal vector assigned in step S19. The created or updated histogram shows the distribution state of the number of normal vector assignments in the angular direction.

  In step S23, it is determined whether or not the coordinate designation on the edge is completed. If the determination result is NO, other coordinates on the edge are designated in step S25, and then the process returns to step S19. If a determination result is YES, it will progress to Step S27. Accordingly, the processing in steps S19 to S21 is executed while paying attention to a plurality of coordinates on the edge.

  In step S27, the variable N is set to “1”. In step S29, referring to the histogram, the number of normal vector assignments corresponding to the angle θ_N is detected as “T_N”. In step S31, it is determined whether or not the detected number of allocations T_N exceeds the reference value REF. In step S33, it is determined whether or not the variable N has reached the maximum value Nmax.

  If “YES” in the step S31, the process proceeds to a step S35, and a warning is generated from the speaker 16 for the steering assist process. When the process of step S35 is completed, the process proceeds to step S39. If “NO” in the step S31 and “YES” in the step S33, the process proceeds to a step S39 as it is. If both steps S31 and S33 are NO, the variable N is incremented in step S37, and then the process returns to step S29. As the variable N is incremented, the angle θ_N is increased by 1 °, for example.

  In step S39, it is determined whether or not the variable M has reached “4”. If YES, the process directly returns to step S11. If NO, the variable M is incremented in step S41 and then returns to step S15.

  As can be seen from the above description, each of the cameras C_1 to C_4 is provided on the vehicle 100 in an obliquely downward posture and captures an object scene around the vehicle 100. The CPU 12p converts the object scene image output from each of the cameras C_1 to C_4 into a bird's-eye image (S5), and extracts an edge from the converted bird's-eye image (S15). The CPU 12p also detects angle information of a line drawn by the extracted edge corresponding to a plurality of positions on the extracted edge (S17 to S25), and the number of detection times out of the detected angle information exceeds the reference. A steering support process such as a warning is executed in response to the specific angle information (S31, S35).

  Accordingly, the detected angle information matches each other when a plurality of positions on the edge drawing a straight line are designated. Further, the number of detections of the same angle information increases as the length of the straight line drawn by the edge increases. Therefore, the steering support process is executed when a linear road surface paint having a length exceeding the length corresponding to the reference is captured by any of the cameras C_1 to C_4.

  The angle information of the line drawn by the edge is detected at a plurality of positions on the edge, and the steering support process is executed corresponding to the angle information whose number of detection exceeds the reference. As a result, the load on the steering assist process can be reduced.

  In this embodiment, a pedestrian crossing is assumed as a road surface paint drawn in a straight line. However, as a road surface paint, a lane marker on a highway or a parking space marker on a parking lot can be assumed.

  In this embodiment, a warning is audibly generated using a speaker, but a warning may be generated visually using a monitor screen.

  Further, in this embodiment, a warning is generated in order to assist the maneuvering. However, in order to ensure the three-dimensional obstacle detection accuracy, the edge of the image representing the straight road surface paint is obstructed. The process of excluding from the determination target for the detection may be executed as the steering support process. In this case, preferably, the process shown in FIG. 18 is executed instead of the process shown in FIG.

  In addition, the same step number is assigned to the same process as the process shown in FIG. 17 among the processes shown in FIG. 18, and redundant description is omitted as much as possible. In addition, a known technique is adopted for the process of detecting an obstacle based on the bird's-eye view image, and a detailed description regarding the obstacle detection process is omitted.

  Referring to FIG. 18, if “YES” in the step S31, the angle θ_N is set in a register (not shown). If “YES” is determined in the step S33, the process proceeds to a step S53 so as to exclude the edge component corresponding to the angle set in the register from the determination target for the obstacle detection. Such a process corresponds to a steering support process. When the steering support process is completed, the process proceeds to step S39.

  As a result, when the obstacle 300 exists around the vehicle 100 as shown in FIG. 19, an image representing the obstacle 300 can be detected with high accuracy from the bird's-eye view image shown in FIG. 20.

  The following are notes on the above-described embodiment. This annotation can be arbitrarily combined with the above-described embodiment as long as there is no contradiction.

  Coordinate conversion for generating a bird's-eye view image from a captured image as described in the embodiment is generally called perspective projection conversion. Instead of using the perspective projection conversion, a bird's-eye view image may be generated from the captured image by a known plane projection conversion. When using planar projective transformation, a homography matrix (coordinate transformation matrix) for converting the coordinate value of each pixel on the captured image into the coordinate value of each pixel on the bird's-eye view image is obtained in advance at the stage of camera calibration processing. . A method for obtaining a homography matrix is known. And when performing image conversion, what is necessary is just to convert a picked-up image into a bird's-eye view image based on a homography matrix. In any case, the captured image is converted into the bird's-eye image by projecting the captured image onto the bird's-eye image.

DESCRIPTION OF SYMBOLS 10 ... Steering assistance apparatus C_1-C_4 ... Camera 12 ... Image processing circuit 12p ... CPU
12m ... SDRAM
DESCRIPTION OF SYMBOLS 14 ... Display apparatus 16 ... Speaker 16 ... Flash memory 100 ... Vehicle

Claims (6)

Conversion means for converting the object scene image output from the image pickup means provided in the moving body in an obliquely downward posture and capturing the object scene around the moving object into a bird's-eye view image;
Extraction means for extracting an edge from the bird's-eye view image converted by the conversion means;
Detection means for detecting angle information of a line drawn by the edge extracted by the extraction means corresponding to a plurality of positions on the edge extracted by the extraction means, and among the angle information detected by the detection means An operation support apparatus comprising processing means for executing operation support processing in response to specific angle information whose number of detections exceeds a reference.   The operation support apparatus according to claim 1, wherein the operation support process includes an alarm process for generating an alarm toward the operator.   The steering support apparatus according to claim 1, wherein the steering support process includes an exclusion process that excludes an edge that draws a line having the specific angle information from a discrimination target for obstacle detection. The detection means includes a creation means for creating a histogram representing a distribution state of the angle information,
The steering support device according to any one of claims 1 to 3, wherein the processing means executes the steering support processing with reference to a histogram created by the creation means. The imaging means includes a plurality of cameras each outputting the object scene image,
The steering assist device according to any one of claims 1 to 4, further comprising an update unit that circularly updates a camera of interest for the extraction process of the extraction unit among the plurality of cameras. A steering assistance method executed by the steering assistance device,
A conversion step of converting the object scene image output from the imaging means that captures the object scene around the moving object and is provided in an obliquely downward posture to the moving object, to a bird's eye view image;
An extraction step of extracting an edge from the bird's-eye view image converted by the conversion step;
A detection step for detecting angle information of a line drawn by the edge extracted by the extraction step corresponding to a plurality of positions on the edge extracted by the extraction step, and among the angle information detected by the detection step An operation support method comprising a processing step of executing an operation support process in response to specific angle information whose number of detection exceeds a reference.

Images