JP6822427B2 - Map change point detector - Google Patents

Description

The present invention relates to a map change point detection device that detects a change point of a road map.

A technique of running a measuring trolley to measure the position of a white line with high accuracy is known. The measuring trolley described in Patent Document 1 includes an odometry device, three GPS (Global Positioning System), three gyro sensors, a laser radar, a camera, and a computer. The white line position measured by the measuring trolley is used for a high-precision map. Road markings such as white lines are accurately drawn on high-precision maps. Patent Document 2 is a document that discloses a top view conversion process.

JP-A-2007-218705 Japanese Unexamined Patent Publication No. 2009-21734

New roads may open. When new roads open, high-precision maps need to be updated. Existing roads may also undergo changes that may require updating the precision map. For example, the rightmost lane will be changed to a right turn lane.

High-precision maps are preferably updated promptly in response to actual road changes. In the following, the change point will be the position where the actual road has changed with respect to the road shown on the high-precision map.

The change point can be detected by running the measuring trolley frequently and comparing the created high-precision map with the map obtained by the measuring trolley. However, since the measuring trolley is provided with the various devices described above, it is expensive, and it is difficult to prepare a large number of measuring trolleys and frequently travel on various roads. That is, it is difficult to quickly detect the change point by the measuring trolley.

Devices equipped with cameras, such as drive recorders, are increasingly being installed in general vehicles. Therefore, it is conceivable to detect the change point using an image captured by a camera mounted on a general vehicle. However, it is not easy to detect a change point using an image captured by a camera mounted on a general vehicle. The reason for this is as follows.

Even if the road width in the image captured by the camera is actually the same, the farther the road is, the narrower the road is. Therefore, the image captured by the camera cannot be simply collated with the image of the road portion drawn on the high-precision map.

In addition, the camera mounted on a general vehicle is provided to image an obstacle existing in the vicinity of the vehicle on which the camera is mounted, such as another vehicle. Naturally, the image captured by the camera is included in the image. , Various obstacles that are not on the road surface are shown. In order to detect the change point with high accuracy, it is preferable to remove obstacles unnecessary for detecting the change point from the image captured by the camera.

However, the resolution of the camera mounted on a general vehicle is not higher than that mounted on a measuring vehicle. Therefore, there is a possibility that obstacles cannot be detected and removed from the image with high accuracy.

For these reasons, it has not been easy to accurately detect the change point using an image captured by a low-resolution camera mounted on a general vehicle.

The present disclosure has been made based on this circumstance, and an object of the present invention is to provide a map change point detecting device capable of accurately detecting a change point even by using a camera having a low resolution. It is in.

The above object is achieved by a combination of the features described in the independent claims, and the sub-claims provide further advantageous specific examples. The reference numerals in parentheses described in the claims indicate, as one embodiment, the correspondence with the specific means described in the embodiments described later, and do not limit the technical scope.

One of the disclosures to achieve the above objectives is
A camera (110) installed in the vehicle that captures the surroundings of the vehicle,
A bird's-eye view conversion unit (S6) that converts the image captured by the camera into a bird's-eye view image,
A map storage unit (150) that stores a road map (151) including a road surface map, and
A collation processing unit (164) that collates the bird's-eye view image with the road surface map to determine whether or not there is a change point that is a position where the actual road surface has changed with respect to the road surface map.
It is a map change point detection device including a collation area specifying unit (163) that determines an area in the vehicle width direction used for collation from the bird's-eye view image based on the number of road marking lines that can be detected from the bird's-eye view image.
The other disclosure to achieve the above objectives is
A camera (110) installed in the vehicle that captures the surroundings of the vehicle,
A bird's-eye view conversion unit (S6) that converts the image captured by the camera into a bird's-eye view image,
A map storage unit (150) that stores a road map (151) including a road surface map, and
A collation processing unit (164) that collates the bird's-eye view image with the road surface map to determine whether or not there is a change point that is a position where the actual road surface has changed with respect to the road surface map.
Obstacle removal processing unit that removes obstacles included in the bird's-eye view image by specifying the color of the object included in the road surface map and converting the color other than the specified color to black. With S7)
The collation processing unit is a map change point detection device that collates the bird's-eye view image after the obstacle is removed with the road surface map .

This map change point detection device converts the image around the vehicle captured by the camera into a bird's-eye view image. By converting to a bird's-eye view image, the road width becomes a constant width image like an actual road regardless of the distance from the vehicle. Further, by converting to a bird's-eye view image, it is possible to remove various objects existing on a place other than the road surface, such as a building and a tree. Since this bird's-eye view image is used as an image to be collated with the road surface map, the change point can be detected accurately even if the resolution is low.

It is a block diagram which shows the structure of the road map information update system 100 of 1st Embodiment. It is a flowchart which shows the process which the image processing unit 162 of FIG. 1 executes. It is a figure which shows the front image 180 which the camera 110 takes. It is a figure which shows the corrected front image 181 generated in S2 of FIG. It is a figure which shows the relationship of the real image height Y, the ideal image height Y0, the incident angle θ, and the focal length f. It is a figure which shows the relationship between each point (u, v) of the image before correction, and the orthogonal coordinates (u0, v0) of an ideal image. It is a figure which shows the front image 180. It is a figure which shows the distortion correction image 185. It is a figure explaining the image before distortion correction. It is a figure explaining the image after volume distortion correction. It is a figure which shows the relationship between the lane marking 182 and the vanishing point 183. It is a figure which shows the bird's-eye view image before removing an obstacle. It is a figure which shows the bird's-eye view image after removing an obstacle. It is a flowchart which shows the process which the collation area identification part 163 executes. It is a figure explaining the method of calculating the white line intensity G. It is a figure explaining the method of detecting the center of a white line from the white line intensity G. It is a figure which shows the relationship between the detected line segment pattern, and a traveling lane. It is a flowchart which shows the process which the collation processing unit 164 executes. It is a figure which shows the high-precision map before distance conversion. It is a figure which shows the high-precision map after distance conversion. It is a figure which shows the enlargement / reduction process executed in S22 of FIG. It is a figure which shows the projective transformation performed in S22 of FIG. It is a figure explaining the scoring executed in S23 of FIG. It is a block diagram which shows the structure of the road map information update system 200 of 2nd Embodiment. It is a flowchart which shows the process which the map generation part 265 of FIG. 20 executes. It is a figure which shows the bird's-eye view image for collation processing generated by the image processing unit 162. It is a figure explaining the image extracted in S31 of FIG. 21 from FIG. It is a flowchart which shows the process which image processing unit 162 executes in 3rd Embodiment. It is a flowchart which shows the detailed processing of S4A of FIG. It is a vanishing point region image 386 one frame before. It is a vanishing point region image 386 obtained in the latest frame. It is a figure explaining the pitch fluctuation amount δy. This is a bird's-eye view image without pitch correction. This is a bird's-eye view image with pitch correction. It is a flowchart which shows the process which the collation processing unit 164 executes in 4th Embodiment. It is a figure which shows the relationship between the first collation area 401 and the first partial area 402. It is a figure which shows the relationship between the 1st partial region 402a and the 2nd partial region 403.

Hereinafter, embodiments will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a road map information updating system 100 having a function as a map change point detecting device.

[overall structure]
The road map information update system 100 includes a camera 110, a parameter storage unit 120, a GPS receiver 130, a behavior detection sensor 140, a map storage unit 150, a calculation unit 160, and a stored data storage unit 170. The road map information updating system 100 is mounted on the vehicle 1.

The camera 110 is attached so as to be able to image the front of the vehicle 1, and images the front of the vehicle 1 in the periphery of the vehicle 1. In the following, the image captured by the camera 110 will be referred to as the front image 180. FIG. 3 shows an example of the front image 180.

In the present embodiment, the camera 110 is a camera included in the drive recorder. If the vehicle 1 is equipped with a drive recorder, the road map information updating system 100 does not need to be separately provided with a camera 110. Since it is the camera 110 included in the drive recorder, the camera 110 is provided with a wide-angle lens and has a wide angle of view.

The parameter storage unit 120 stores the parameters of the camera 110 required for image processing. Specifically, this parameter is the focal length f of the camera 110, the yaw angle rx of the camera optical axis, and the tilt angle ry. The focal length f of the camera 110 is a value determined by the product and is stored in advance in the parameter storage unit 120.

On the other hand, the yaw angle rx and the tilt angle ry of the camera optical axis change depending on the mounting posture of the camera 110. Therefore, in the present embodiment, the front image 180 is analyzed to calculate the yaw angle rx and the tilt angle ry. Then, the calculation result is stored in the parameter storage unit 120. A writable storage unit is used in the parameter storage unit 120 so that the calculation result can be stored.

The GPS receiver 130 receives a navigation signal transmitted by a navigation satellite included in GPS (Global Positioning System), which is one of the satellite navigation systems, and sequentially calculates the current position based on the received navigation signal.

The behavior detection sensor 140 is a sensor that detects the behavior of the vehicle 1. Accelerometers, vehicle speed sensors, and gyro sensors are specific examples of the behavior detection sensor 140.

A high-precision map 151 is stored in the map storage unit 150. The high-precision map 151 is a map measured by traveling a measuring trolley, and road markings such as road markings are shown in a detailed state like an image. In addition, the high-precision map 151 also shows in detail three-dimensional objects above the road surface, such as utility poles, guardrails, and road signs. For convenience of the following description, the road surface portion of the high-precision map 151 will be referred to as a road surface map. Further, the high-precision map 151 including the road surface map corresponds to a road map.

The arithmetic unit 160 is a computer equipped with a CPU, ROM, RAM, etc., and the CPU can use the temporary storage function of the RAM as a non-transitory tangible storage medium such as a ROM. Execute the stored program. As a result, the calculation unit 160 functions as a vehicle position detection unit 161, an image processing unit 162, a collation area identification unit 163, and a collation processing unit 164. When these functions are executed, the method corresponding to the program is executed.

A part or all of the functional blocks included in the arithmetic unit 160 may be realized by using one or a plurality of ICs (in other words, as hardware). Further, a part or all of the functional blocks included in the arithmetic unit 160 may be realized by a combination of software execution by the CPU and hardware members.

The storage data storage unit 170 is a storage unit that stores the change point detected by the processing of the collation processing unit 164 and the road information regarding the change point. The change point is the position where the actual road is changed with respect to the road shown in the high-precision map 151. The road information regarding the change point is information indicating what kind of change has occurred on the road at the change point. For example, the characters of the road marking newly added to the road are the road information regarding the change point.

Next, the processing of each unit included in the calculation unit 160 will be described.

[Processing of vehicle position detection unit 161]
The vehicle position detection unit 161 sequentially detects the current position of the vehicle 1 by hybrid navigation that combines satellite navigation and self-contained navigation. The satellite navigation is a method of estimating the current position based on the signal supplied from the GPS receiver 130, and the self-contained navigation is a method of detecting the relative trajectory of the vehicle position based on the vehicle behavior detected by the behavior detection sensor 140. Is. Not limited to this, the current position of the vehicle 1 may be sequentially detected only by satellite navigation. The vehicle position detection unit 161 outputs the detected current position to the image processing unit 162.

[Processing of image processing unit 162]
The image processing unit 162 processes the front image 180 sequentially captured by the camera 110 to generate a bird's-eye view image. The process executed by the image processing unit 162 will be described with reference to the flowchart shown in FIG. The image processing unit 162 periodically executes the process shown in FIG. 2 in the activated state of the road map information updating system 100.

In step (hereinafter, step is omitted) S1, the front image 180 captured by the camera 110 is input. In S2, the process of correcting the distortion of the front image 180 input in S1 is performed to generate the corrected front image 181 illustrated in FIG. S2 corresponds to the distortion correction processing unit.

Since the front image 180 is captured by a wide-angle lens, a large amount of distortion occurs. When the distortion is large, the detection accuracy of the change point decreases. Therefore, the distortion of the front image 180 is corrected. The distortion here is a phenomenon in which the image is drawn to the center of the image according to the angle of incidence on the camera 110, and is called distortion. In the case of a wide-angle lens, a barrel-shaped image is obtained in which the corners of the image are drawn to the vertical center or the horizontal center of the image due to distortion.

Image projection methods include normal projection and equidistant projection, and distortion correction processing corresponding to the projection method is performed. In the present embodiment, distortion correction processing corresponding to equidistant projection is performed. The equidistant projection relational expression is expressed by Equation 1. In Equation 1, Y is the real image height, Y0 is the ideal image, f is the focal length, and θ is the angle of incidence. FIG. 5 illustrates the relationship between the real image height Y, the ideal image height Y0, the incident angle θ, and the focal length f.

Equation 2 is obtained by expressing the orthogonal coordinates (u, v) of each point of the uncorrected image using the real image height Y and the angle φ in the polar coordinate system.

The Cartesian coordinates (u0, v0) of the ideal image are represented by Equation 3.

FIG. 6 shows the relationship between each point (u, v) of the uncorrected image, the orthogonal coordinates (u0, v0) of the ideal image, and the polar coordinates. In the distortion correction process, first, (u, v) is converted into polar coordinates (Y, φ) by Equation 2. Next, the polar coordinate Y is replaced with the ideal image height Y0 obtained from Equation 1. Then, the Cartesian coordinates (u0, v0) are obtained from Equation 3.

The image obtained by the processing up to this point is referred to as a distortion correction image 185. FIG. 7 shows a front image 180 different from that of FIG. 3, and FIG. 8 shows a distortion correction image 185 obtained by correcting the front image 180 of FIG. 7.

As can be seen from the relationship between Y and Y0 in FIG. 6, the distortion correction image 185 shown in FIG. 8 is an image in which a three-dimensional object projected on the edge of the image is pulled toward the edge of the image. This is called a volume distortion image.

In the distortion correction process, a process for correcting a volume distortion image is also performed. In order to correct the volume distortion image, as shown in FIG. 10, a non-linear conversion is performed so that the peripheral portion of the image is closer to the center of the image. This conversion is called volume distortion correction. The volume distortion image correction is a correction that converts the straight line shown in FIG. 9 into the curve shown in the equation 48 of FIG. Further, by this correction, the size in the left-right direction of the image is reduced to the broken line shown in FIG. 10, and the shrinkage rate is higher toward the left and right edges of the image.

The image after the volume distortion image correction is the corrected front image 181 shown in FIG. In the corrected front image 181 the roads and bridges are almost straight, and the building projected on the peripheral edge of the image also stands straight.

In S3, a straight line is detected from the corrected front image 181. In order to detect a straight line, an edge detection process for detecting the shading of an image is performed. The detected edges are represented by a set of points. A straight line such as a road marking line is detected by Hough transforming the set of points at this edge.

In S4, the vanishing point 183 of the corrected front image 181 is detected. S4 corresponds to the vanishing point detection unit. In order to detect the vanishing point 183, first, a parallel line extending near the center in the corrected front image 181 is extracted from the straight line detected in S3. As shown in FIG. 4, when four lane markings 182 can be detected as straight lines, for example, two lane markings 182 for dividing the lane in which the vehicle 1 travels are used. The point where these two lane markings 182 intersect is referred to as a vanishing point 183. FIG. 11 illustrates the relationship between the lane marking 182 and the vanishing point 183.

In S5, the orientation of the camera 110 is detected. S5 corresponds to a camera orientation detection unit. The orientation of the camera 110 means the orientation of the optical axis of the camera 110. Since the camera 110 is a camera included in the drive recorder, it is often attached after the vehicle is purchased, and the orientation of the camera 110 can be changed even after the attachment. Therefore, the orientation of the camera 110 is different for each vehicle 1. Therefore, in this S5, the orientation of the camera 110 is detected.

Specifically, the orientation of the camera 110 is a yaw angle rx and a tilt angle ry. The yaw angle rx and the tilt angle ry can be expressed by Equations 4 and 5, respectively. In Equation 4, as shown in FIG. 11, Δx is the distance between the image center 184 and the vanishing point 183 in the corrected front image 181 in the x direction. In Equation 5, Δy is the distance in the y direction between the image center 184 and the vanishing point 183 in the corrected front image 181 as shown in FIG.

When the orientation of the camera 110 can be detected in S5, then in S6, the top view conversion process (that is, bird's-eye view conversion) is performed on the corrected front image 181 to generate a bird's-eye view image having a viewpoint above the vehicle 1. .. S6 corresponds to the bird's-eye view conversion unit.

The top view conversion process is a process of converting the optical axis of the camera 110 in the direction from upward to downward. The top view conversion process is described in Patent Document 2 and the like.

When converting the corrected front image 181 into a bird's-eye view image, the portion of the corrected front image 181 that can be seen even when viewed from above the vehicle 1 is converted into a bird's-eye view image. That is, the road surface and its surroundings in the corrected front image 181 are converted into a bird's-eye view image. The range to be converted into the bird's-eye view image is determined based on the vanishing point 183 and the angle of view of the camera 110.

In S7, obstacles are removed from the bird's-eye view image generated in S6. S7 corresponds to the obstacle removal processing unit. By making the bird's-eye view image, among the obstacles shown in the corrected front image 181, the obstacles shown outside the range to be converted into the bird's-eye view image are removed. As a result, many obstacles that cause a decrease in the detection accuracy of the change point can be removed. However, the bird's-eye view image may also show objects other than road-installed objects, such as preceding vehicles and falling objects. Therefore, a process of removing obstacles from the bird's-eye view image is performed.

The method of removing obstacles is to extract the specified color and make the other colors black. The colors to be specified are white and orange, which are the colors corresponding to the road markings and road markings that you do not want to remove.

FIG. 12 shows a bird's-eye view image showing the preceding vehicle which is an obstacle, and FIG. 13 shows an image after executing S7 to remove the obstacle. By converting the portion of the preceding vehicle to black, the preceding vehicle is removed in the image of FIG. Obstacles smaller than a certain size may be restored to their original colors, assuming that they do not interfere with the subsequent processing.

In S8, the overall brightness of the bird's-eye view image is adjusted so that the portion having the highest brightness in the bird's-eye view image becomes the brightness of white. The reason for adjusting the brightness is that the brightness of the bird's-eye view image is affected by the surrounding environment at the time of camera imaging, such as the weather, time zone, and inside the tunnel, so if the brightness is not adjusted, there is a high risk of erroneous detection of change points. Because. The bird's-eye view image after the brightness adjustment in S8 is the bird's-eye view image for collation processing.

In S9, the current position supplied from the vehicle position detection unit 161 and the bird's-eye view image for the collation processing obtained by the processing of S8 are output to the collation area identification unit 163.

[Processing of collation area identification unit 163]
The collation area specifying unit 163 identifies the collation area from the bird's-eye view image for collation processing generated by the image processing unit 162. The collation area specifying unit 163 detects a road lane marking from the bird's-eye view image for collation processing, and determines the range of the collation region in the vehicle width direction based on the number of the road lane markings.

FIG. 14 shows a process executed by the collation area specifying unit 163. Detailed processing of the collation area specifying unit 163 will be described with reference to FIG. The collation area specifying unit 163 executes the process shown in FIG. 14 when the bird's-eye view image for the collation process is generated.

In FIG. 14, in S11, the white line detection process is executed. In the white line detection process, the following filter processing is performed on each pixel of the bird's-eye view image to quantify the white line-likeness. Let this value be the white line intensity G.

The white line intensity G is a value obtained by dividing the sum of the pixel values within the range of the assumed white line width S centering on the pixel of interest (m, n) by the area. The pixels of interest (m, n) are pixels for calculating the white line intensity G, and the white line intensity G is calculated in order for all the pixels.

The white line strength G is represented by the formulas 6, 7, and 8. Equation 6 shows the left white line intensity _GL . The numerator of the first term on the right side of Equation 6 is the sum of the pixel values within the range of the assumed white line width S, and the numerator of the second term on the right side of Equation 6 is the sum of the pixel values outside the range on the left side of the assumed white line width S. Is.

Equation 7 shows a right side white line intensity G _R. The numerator of the first term on the right side of the equation 7 is the sum of the pixel values within the range of the assumed white line width S as in the first term of the right side of the equation 6. The numerator of the second term on the right side of Equation 7 is the sum of the out-of-range pixel values on the right side of the assumed white line width S. FIG. 15 shows the range of the total pixel values calculated by the numerator of the first term and the second term on the right side of the formulas 6 and 7, respectively.

The clearer the white line on the image, the larger the value of the first term on the right side and the smaller the value of the second term on the right side in Equations 6 and 7. Therefore, Equations 6 and 7 show the white line strength G.

Equation 8, calculates the square root of sum of squares to put together a left white line intensity G _L and the right white line intensity G _R into a single value. In image processing, a method using the sum of square roots as a method for calculating the edge strength is common.

Note that FIG. 15 shows the white line strength G in the horizontal direction of the screen. In S11, in addition to the white line strength G in the horizontal direction of the screen, the white line strength G in the vertical direction of the screen is also calculated.

After calculating the white line intensity G, the center of the white line is then detected. In order to detect the center of the white line, the horizontal white line strength G searches for a strong part by one line in the vertical direction, and the horizontal white line strength G searches for a strong part by one line in the vertical direction. And. FIG. 16 shows the magnitude of the white line strength G in the horizontal direction line by line. The peak shown in FIG. 16 is centered on the white line.

In S12, the traveling lane is recognized. In the traveling lane recognition process, the portions where the center of the white line detected in S11 is continuous in the vertical, horizontal, and diagonal directions are grouped (that is, merged) into one line segment. The image after grouping is defined as a white line intensity image. Further, the cumulative value of the white line strength G included in the merged line segment is used as the white line strength G of the merged line segment, and the line segment having the strongest merged white line strength G is used as the reference line.

A line segment is searched near the assumed lane width position on each of the left and right sides from this reference line. The line segment is searched up to the second left and right based on the position of the own vehicle. Also, the length of the line segment distinguishes between the solid line and the broken line.

Then, from the detected line segment pattern, the traveling lane is recognized according to the relationship shown in FIG. The information to be recognized is how many lanes the road on which the vehicle is traveling is, and which traveling lane the vehicle 1 is traveling on.

In S13, the collation region is extracted from the white line intensity image generated in S12. The area to be extracted is fixed in the vertical direction, that is, the area in the front-rear direction of the own vehicle, and is variable in the horizontal direction. The vertical range of the collation area is a certain range from the edge of the white line intensity image on the own vehicle side.

The horizontal range of the collation area corresponds to the traveling lane recognized in S12. For example, when the number of traveling lanes recognized in S12 is three lanes, the entire white line intensity image is set as the width of the collation area. When the number of traveling lanes recognized in S12 is two lanes, the center of the line segments at both ends is set as the center of the extraction area, and the assumed lane width + α of the two lanes is set as the lateral range of the extraction area. The image extracted in S13 is a rectangular image. As a result, a bird's-eye view image of an appropriate size according to the number of road marking lines that can be detected from the bird's-eye view image can be used as a collation area. It should be noted that the portion removed by the obstacle removal in S7, such as the portion in which the vehicle in front and the parallel running vehicle are shown, may not be used as the collation area.

In S14, the matching target area is specified from the road surface map of the high-precision map 151. The matching target area is a fixed area determined with reference to the current position. The vertical and horizontal sizes of the matching target area are set to be larger than the matching area of the white line intensity image so that the matching judgment can be performed while shifting the matching area of the white line intensity image with respect to the matching target area. ing.

[Processing of collation processing unit 164]
The collation processing unit 164 detects a change point by collating the white line intensity image of the collation area extracted in S13 with the road surface map of the high-precision map 151 of the matching target area specified in S14. The detailed processing of the collation processing unit 164 will be described with reference to FIG. The collation processing unit 164 executes the process shown in FIG. 18 when the collation area and the matching target area are determined.

In FIG. 18, in S21, distance conversion is performed on the high-precision map 151 of the matching target area specified in S14. The distance conversion here is the same as the distance conversion widely used in image processing, and the distance from 0 pixel to the nearest non-zero pixel is calculated for all 0 pixels, and the pixel of 0 pixel is calculated. A transformation that replaces a value with that distance.

If the high-precision map 151 and the white line intensity image are matched as they are, the correlation value will be low even if the lane widths are slightly different. Therefore, the distance conversion is performed on the high-precision map 151 of the matching target area.

Equation 9 shows the equation for distance conversion. In Equation 9, the left side replaces the distance, x is the horizontal distance to non-zero pixels, y is the vertical distance to non-zero pixels, and N is a preset coefficient.

19 and 20 show maps before and after distance conversion. FIG. 19 is a map before distance conversion, that is, a high-precision map 151. When the distance conversion is performed, as shown in FIG. 20, the white line becomes blurred.

In S22, while changing the position of the collation area of the white line intensity image by a predetermined value, the correlation value between the matching target area of the high-precision map 151 after the distance conversion in S21 and the collation area of the white line intensity image is calculated. .. A correlation value map can be created by calculating the correlation value while changing the position of the collation area of the white line intensity image by a predetermined value. The correlation value map shows the relationship between the relative position of the collation area with respect to the matching target area and the correlation value.

In calculating the correlation value, if the spacing between the road marking lines drawn in the white line strength image and the spacing between the road marking lines drawn on the high-precision map 151 after distance conversion do not match, the white line strength is calculated. Enlarge / reduce the image and project the white line intensity image. FIG. 21 shows a diagram for enlarging / reducing a white line intensity image, and FIG. 22 shows a diagram for projecting and transforming a white line intensity image. By these scaling and projective transformation, the overall size of the white line intensity image is adjusted so that the division line of the high-precision map 151 and the division line of the white line intensity image match. S22 corresponds to the correlation value map creation unit.

In S23, scoring is performed. Scoring is to quantify the certainty of the position of the collation area of the white line intensity image. Specifically, in S23, first, a scoring map is created. The scoring map is a map in which the coefficient is set to 1 assuming that the current position of the vehicle 1 detected by the vehicle position detection unit 161 is the most probable, and the coefficient decreases as the distance from the current position increases.

As shown in FIG. 23, scoring is performed by multiplying this scoring map with the correlation value map created in S22. Then, the position with the highest score is set as the matching position. S23 corresponds to the matching position determining unit.

In S24, the white line intensity image is converted into grayscale, and the absolute value of the difference between the high-precision map 151 corresponding to the position of the white line intensity image determined in S23 and the white line intensity image is obtained. If this absolute value is equal to or greater than a predetermined value, the change point is determined.

Further, the absolute value of the difference between the high-precision map 151 and the white line intensity image is replaced with the HSV color space and output. Specifically, when replacing with HSV space, S (saturation) and V (brightness) are fixed, and the value of H (hue) is converted. Further, the absolute value of the above difference is normalized to 0 to 255. Let this normalized value be the value of H. Then, the red and blue of H are exchanged, and blue is assigned to be 0 and red is assigned to be 255. As a result, change points such as increase / decrease of paint are represented by different colors as road information.

The change points detected in this way are accumulated in the accumulated data storage unit 170. Further, the position of the change point is provided to the update information utilization system 2 which is an external system. In the update information utilization system 2, the change points are plotted on a map, a list is generated, and the change points, that is, the road update information indicating that the road has been updated is managed. In addition, the road update information may be compared by period so that it can be confirmed in which period the new road was opened.

[Effect of Embodiment]
As described above, the road map information updating system 100 converts the front image 180 captured by the camera 110 into a bird's-eye view image. By converting to a bird's-eye view image, the road width becomes an image having a constant width like an actual road regardless of the distance from the vehicle 1. Further, by converting to a bird's-eye view image, it is possible to remove various objects existing on a place other than the road surface, such as a building and a tree.

Since the white line intensity image generated from this bird's-eye view image is used as an image to be collated with the road surface map which is a part of the high-precision map 151, the change point can be detected accurately even if the resolution of the camera 110 is low.

Further, in the present embodiment, the obstacle is removed from the bird's-eye view image by executing the process of S7. As a result, the change point can be detected more accurately.

Further, in the present embodiment, the position of the white line intensity image having the highest correlation value in the matching process is not used as the matching position as it is, but is determined based on the current position measured by the vehicle position detection unit 161. The correlation value is corrected with. As a result, the detection accuracy of the change point is further improved.

<Second Embodiment>
Next, the second embodiment will be described. In the following description of the second embodiment, the elements having the same number as the codes used so far are the same as the elements having the same code in the previous embodiments, unless otherwise specified. Further, when only a part of the configuration is described, the embodiment described above can be applied to the other parts of the configuration.

FIG. 24 shows the configuration of the road map information updating system 200 of the second embodiment. The road map information update system 200 of the second embodiment is different from the road map information update system 100 of the first embodiment in that the calculation unit 260 includes the map generation unit 265. Further, the accumulated data storage unit 170 stores the generated map image 171. The generated map image 171 is a map in which a bird's-eye view image is combined.

When the collation processing unit 164 detects a change point, the map generation unit 265 generates a map that reflects the information of the change point. The detailed processing of the map generation unit 265 will be described with reference to FIG.

In FIG. 25, in S31, a bird's-eye view image for superposition is extracted. Specifically, a region corresponding to the collation region of the white line intensity image extracted in S13 is extracted from the bird's-eye view image for collation processing generated by the image processing unit 162. FIG. 26 shows a bird's-eye view image for collation processing generated by the image processing unit 162, and FIG. 27 shows a bird's-eye view image for superposition extracted in S31. In addition, the part where the extracted bird's-eye view image shows an object other than the road surface information is excluded from the used area. The image extracted in S31 is a map image.

In S32, the position to add the bird's-eye view image extracted in S31 is determined with respect to the generated map image 171. This position is the matching position determined in S23.

In S33, an averaged image is generated for the portion of the additional position determined in S32 where the image already exists in the generated map image 171. The averaged image is an image obtained by averaging an existing image and an image to be added this time. By averaging, it is possible to reduce the influence of temporary changes caused by objects other than the road surface such as the shadow of the vehicle 1.

Further, at the time of averaging, as shown in FIG. 27, the weight is increased toward the side closer to the camera 110, and the averaging process is performed with the image that already exists. This is because, due to the characteristics of the bird's-eye view image, the closer to the camera 110, the higher the resolution of the image.

In S34, the images are superimposed and the generated map image 171 is updated. At the time of superimposition, the averaged image generated in S33 is superposed on the generated map image 171 for the portion where the image already exists in the generated map image 171. For the portion where the image does not yet exist in the generated map image 171, the bird's-eye view image extracted in S31 is superimposed on the generated map image 171 as it is.

Since the generated map image 171 that is sequentially updated in this way includes an image of the change point, the change at the change point can be recognized by the image. Therefore, since the change point can be recognized without running the measurement trolley, the update cost of the high-precision map 151 can be reduced. In addition, the updated contents can be used as temporary information for vehicle control such as automatic driving.

<Third Embodiment>
In the third embodiment, the image processing unit 162 executes the process shown in FIG. 28. In the process shown in FIG. 28, S4A is added between S4 and S5 with respect to the process shown in FIG. Also, S5A is executed instead of S5.

In S4A, the amount of pitch fluctuation of the corrected front image 181 is calculated. The detailed processing of S4A is shown in FIG. In FIG. 29, in S41, the vanishing point region image 386 is determined. As illustrated in FIGS. 30 and 31, the vanishing point region image 386 is an image of a region determined with reference to the vanishing point in the corrected front image 181. The vanishing point region image 386 shown in FIGS. 30 and 31 is a horizontally long rectangular region centered on the vanishing point 183 (see FIG. 11), in which a pair of short sides is vertical and a pair of long sides are horizontal. ..

In S42, the vanishing point region image 386 obtained in the previous frame, that is, the vanishing point region image 386 determined in the previous execution of FIG. 28, and the vanishing point region image 386 obtained in S41 executed immediately before this S42 are displayed. Performs matching processing to match. FIG. 30 is a vanishing point region image 386 obtained in the previous frame, and FIG. 31 is a vanishing point region image 386 obtained in the current frame. In the matching process, the vanishing point area image 386 obtained in the current frame is moved based on the vanishing point area image 386 obtained in the previous frame so that the two vanishing point area images 386 match most this time. The vanishing point region image 386 obtained in the frame is moved.

In S43, the amount of movement of the vanishing point region image 386 obtained in this frame in S42 is calculated. This movement amount is used as a value meaning the pitch fluctuation generated in the vehicle 1 during one frame. S41 to S43 up to this point correspond to the movement amount calculation unit.

In S44, the movement amount calculated in S43 in the processing of FIGS. 28 and 29, which is repeatedly executed each time the front image 180 is input, is integrated. Hereinafter, the integrated value will be referred to as a positional deviation integrated value. This position deviation integrated value includes the position shift of the vanishing point region image 386 due to the pitch fluctuation and the position change of the vanishing point region image 386 due to a static reason other than the pitch fluctuation. The static reason other than the pitch fluctuation is, for example, a change in the camera optical axis due to the vehicle 1 traveling uphill. This S44 corresponds to the integral value calculation unit.

In S45, the moving average value of the position deviation integrated value calculated in S44 is calculated. The section for moving average is set in advance. The moving average value is used to indicate the position of the vanishing point region image 386 in the stationary state of the vehicle. The misalignment integral value also includes changes in the value due to static reasons other than pitch fluctuation. The moving average value calculates the value due to this static reason, and the moving average interval is set to the interval in which the value due to this static reason can be calculated.

In S46, the difference between the position deviation integrated value calculated in S44 and the moving average value calculated in S45 is calculated. This difference is defined as the pitch fluctuation amount δy. Whether the pitch fluctuation amount δy is the position deviation integrated value-moving average value or the moving average value-position deviation integrated value may be set as long as it is predetermined. S45 and S46 correspond to the pitch fluctuation amount calculation unit. FIG. 32 shows the position deviation integrated value, the moving average value, and the pitch fluctuation amount δy.

After executing S46, the process proceeds to S5A in FIG. 28. In S5A, the orientation of the camera 110 is detected. Specifically, the orientation of the camera 110 is a yaw angle rx and a tilt angle ry, as described in the first embodiment. The yaw angle rx is obtained from the formula 4 described in the first embodiment. The tilt angle ry is obtained from Equation 10. As shown in Equation 10, the yaw angle rx is calculated from the distance Δy between the image center 184 and the vanishing point 183 in the y direction and the pitch fluctuation amount δy.

The process after executing S5A is the same as that of the first embodiment. S6 to S9 are executed, and the bird's-eye view image for collation processing and the current position are output to the collation area specifying unit 163.

[Effect of pitch correction]
FIG. 33 shows a bird's-eye view image without pitch correction, and FIG. 34 shows a bird's-eye view image with pitch correction. The pitch-corrected bird's-eye view image is the bird's-eye view image obtained by executing FIG. 28. In the bird's-eye view image without pitch correction, the white line spacing increases from the bottom to the top. On the other hand, in the pitch-corrected bird's-eye view image, the intervals between the white lines are almost parallel.

By performing pitch correction in this way, the intervals between the white lines in the bird's-eye view image can be made close to parallel. Therefore, the accuracy of the subsequent processing performed based on this bird's-eye view image is improved.

<Fourth Embodiment>
Next, a fourth embodiment will be described. In the first embodiment, the collation processing unit 164 separately determines the matching position for each frame. On the other hand, in the fourth embodiment, the matching position is finally determined for each frame, but as a preprocessing thereof, the matching position is determined for the image in which a plurality of frames are combined. After that, the images for which the matching position is determined are sequentially made smaller. The details will be described below.

In the fourth embodiment, as described above, the matching position is determined with respect to the image in which a plurality of frames are combined. Therefore, the image processing unit 162 executes the process when the front image 180 for a plurality of frames is input. The content of the processing executed by the image processing unit 162 in the fourth embodiment is the same as that of the previous embodiments except that the front image 180 to be processed is for a plurality of frames.

The number of multiple frames can be set as appropriate. For example, it can be a total frame recorded in one event in a drive recorder.

The collation area specifying unit 163 specifies a collation processing area for all of the bird's-eye view images for collation processing generated by the image processing unit 162. The content of the processing executed by the collation processing unit 164 in the fourth embodiment is the same as that of the previous embodiments except that the bird's-eye view image to be processed is a plurality of frames.

In the fourth embodiment, the collation processing unit 164 executes the process shown in FIG. 35. The collation processing unit 164 executes the process shown in FIG. 35 when the collation area identification unit 163 specifies the collation area.

In S51, the first collation area 401 (see FIG. 36) is determined. The first collation area 401 is an area obtained by combining the collation areas of white line intensity images for a plurality of frames. The first collation area 401 shown in FIG. 36 is an area obtained by combining the collation areas of the white line intensity images determined from all the frames included in the image data for one event of the drive recorder.

In S52, the same processing as in S21 of FIG. 18 is performed on the matching target area of the high-precision map determined based on the first collation area 401 determined in S51. As described in S14 of FIG. 14, the collation area specifying unit 163 determines the matching target area from the road surface map of the high-precision map 151 based on the collation area. That is, the matching area specifying unit 163 determines the matching target area of the road surface map with respect to the matching area for one frame. Therefore, in this S52, the matching target area of the road surface map for one frame of the matching area determined by the matching area specifying unit 163 is added to the total of all the frames included in the first matching area 401 determined in S51. On the other hand, distance conversion is performed.

In S53, the collation areas of all the white line intensity images included in the first collation area 401 determined in S51 are overlapped. The overlay process is the same process as the map generation unit 265 described in the second embodiment.

In S54, a matching process is performed on the collation area to create a correlation value map. The collation area is the first collation area 401 determined in S51 if the first execution of S54 is performed after the executions of S51 to S53. Subsequent collation areas are determined by executing S59, which will be described later. The processing of S54 is the same as the processing of S22 of FIG. 18 except that the regions are different. Therefore, S54 corresponds to the correlation value map creation unit.

In S55, scoring and matching position determination are performed. The processing of S55 is the same as that of S23 in FIG. Therefore, S55 corresponds to the matching position determining unit.

In S56, it is determined whether or not the determination of the matching position of all the collation areas is completed. Although the first collation area 401 is one, a plurality of partial areas are determined from one collation area by executing S59 described later. Then, each subregion becomes the next collation region. Therefore, there may be a plurality of collation areas. When a plurality of collation areas exist, the matching position is determined in sequence. Therefore, in this S56, it is determined whether or not the determination of the matching position for all the collation areas is completed.

If the determination in S56 is NO, in S57, the collation area for determining the matching position is changed to a collation area for which the matching position has not yet been determined. After that, S54 and S55 are executed to determine the matching position.

If the determination in S56 is YES, the process proceeds to S58. In S58, it is determined whether or not the collation area for which the matching position is determined is for one frame. One frame corresponds to the final number of frames. In the present embodiment, the final number of frames is one frame, but a larger number of frames (for example, two frames) can be set as the final number of frames. If the determination in S58 is NO, the process proceeds to S59.

In S59, each collation area is divided to create a partial area. FIG. 36 shows the first collation region 401 and the partial regions 402a, 402b, 402c, 402d, 402e. In addition, FIG. 36 also shows a road surface map of the high-precision map 151. When the partial regions 402a, 402b, 402c, 402d, and 402e are not distinguished, it is described as the partial region 402. The partial area 402 is a part that becomes the second collation area.

The partial area 402 is an area included in the first collation area 401 and is a part of the first collation area 401. Further, the partial area 402 is a range including a portion that may be a lane marking from the first collation area 401. The possibility of a lane marking is determined from, for example, a change in edge strength.

The size of the partial region 402 with respect to the initial collation region 401 is, for example, half or more. This is because if the partial region 402 is too small with respect to the initial collation region 401, it becomes difficult to obtain the effect of suppressing the determination of an erroneous matching position by adjusting the alignment little by little. The number of subregions 402 is one or more, and can be set arbitrarily. However, it is preferable to set a plurality. In the example of FIG. 36, the number of partial regions 402 is 5.

Further, when the number of the partial regions 402 is set to a plurality, it is preferable that the partial regions 402 partially overlap each other. This is because if a part of a certain partial region 402 is also a part of another partial region 402, it is possible to prevent the matching position with respect to the partial region 402 from becoming an erroneous position.

The created partial area becomes the next collation area. After determining the next collation area, S54 and below are executed. In the second and subsequent executions of S55, the matching target area is the matching area for which the partial area is created in the immediately preceding S59. By executing S54 to S59, the matching position of the collation area is determined while sequentially reducing the collation area until the collation area becomes one frame.

FIG. 37 illustrates the second partial regions 403a, 403b, 403c, 403d, 403e created from the first partial region 402a. When the partial regions 403a, 403b, 403c, 403d, and 403e are not distinguished, they are described as the partial region 403. The partial area 403 is a part that becomes the third collation area.

The partial regions 403a, 403b, 403c, 403d, and 403e are partial regions of the partial region 402a. Further, each partial region 403a, 403b, 403c, 403d, 403e is a range including a portion that may be a lane marking. When the portion that may be a lane marking is near the edge of the collation area for which the partial area 403 is to be determined, the partial area is such that a part of the partial area 403 is located outside the collation area. 403 may be determined. That is, a part of the partial area determined from the collation area may be located outside the collation area. This is because the accuracy of the matching position is improved by setting the lane marking line closer to the center than the edge portion as compared with the case where the lane marking line is located at the edge portion of the partial region 403. Taking FIG. 37 as an example, the matching target region for the partial regions 403a, 403b, 403c, 403d, and 403e is the partial region 402a.

Like the first partial region 402, the second partial region 403 is created in five, and is created so as to partially overlap each other. The third and subsequent partial regions (that is, the fourth and subsequent collation regions) are also created in the same manner as the first and second partial regions 402 and 403.

When a plurality of collation areas are created in S59 and some of the collation areas overlap with each other, matching positions are obtained from each of the plurality of collation areas having the overlap areas. That is, a plurality of matching positions can be obtained for the overlapping area. The position of the overlapping region is determined by considering both of the plurality of matching positions, such as averaging a plurality of matching positions. The positions of the collation areas other than the overlapping area are also determined according to the positions of the overlapping areas. By determining the position of the overlapping area in consideration of the matching positions of the plurality of matching areas in this way, the matching position of the matching area is erroneous as compared with the case where the matching area is determined so that there is no overlapping area. It is possible to suppress the determination of the position.

When the collation area becomes one frame, the determination in S58 becomes YES and the process proceeds to S60. In S60, the change point is extracted. The processing of S60 is the same as that of S24 of FIG.

As described above, the collation processing unit 164 of the fourth embodiment determines the matching position in a large collation area, and sequentially determines the matching position while reducing the collation area. As a result, in a small area, even if an image similar to the image in a certain collation area exists at a position different from the original position of the collation area, the matching position may be determined at an erroneous position. Can be suppressed.

Although the present embodiment has been described above, the present disclosure is not limited to the above-described embodiment, the following modifications are also included in the scope of disclosure, and various other than the following, as long as the gist is not deviated. Can be changed and implemented.

<Modification example 1>
A center outside the vehicle 1 may perform a part of the processing performed by the map generation unit 265 of the second embodiment. In this case, the image extracted in S31 is transmitted to the center. Then, at the center, the processing after S32 is performed.

<Modification 2>
In the generated map image 171 of the second embodiment, the bird's-eye view image for collation processing is used as an image for generating the generated map image 171. However, a white line intensity image may be used as the image for generating the generated map image 171.

<Modification example 3>
The center has the functions of the image processing unit 162, the collation area identification unit 163, and the collation processing unit 164, and the vehicle 1 sequentially transmits the front image 180 and the current position of the vehicle 1 detected by the vehicle position detection unit 161 to the center. You may try to do it. Further, the processing of the image processing unit 162 may be performed by the vehicle 1, and the processing of the collation area specifying unit 163 and the collation processing unit 164 may be performed by the center. Further, the vehicle 1 may perform the processing up to the collation area specifying unit 163, and the collation processing unit 164 may perform the processing at the center. In these cases, the vehicle 1 includes a communication unit for communicating with the center.

When the processing of the collation area specifying unit 163 is performed by the vehicle 1, the center can also specify the area to be processed to the vehicle 1. Then, only the high-precision map 151 for the designated area may be transmitted to the vehicle 1. As a result, the vehicle 1 can reduce the capacity and the amount of communication for storing the high-precision map 151, and the center can also reduce the processing load.

<Modification example 4>
As the camera 110, a camera that images the rear of the vehicle may be used.

1: Vehicle 2: Update information utilization system 100: Road map information update system 110: Camera 120: Parameter storage unit 130: GPS receiver 140: Behavior detection sensor 150: Map storage unit 151: High-precision map 160: Calculation unit 161: Vehicle position detection unit 162: Image processing unit 163: Matching area identification unit 164: Matching processing unit 170: Accumulated data storage unit 171: Generated map image 180: Front image 181: Corrected front image 182: Compartment line 183: Vanishing point 184 : Image center 185: Distortion correction image 200: Road map information update system 260: Calculation unit 265: Map generation unit 386: Disappearance point area image 401: First collation area 402: Partial area S2: Distortion correction processing unit S4: Disappearance inspection Output part S5: Camera orientation detection part S6 Bird's-eye view conversion part, S7 Obstacle removal processing part S22, S54: Correlation value map creation part S23, S55: Matching position determination part S41, S42, S43: Movement amount calculation part S44: Integrated value Calculation unit S45, S46: Pitch fluctuation amount calculation unit δy: Pitch fluctuation amount

Claims (8)

A camera (110) provided in the vehicle that captures the surroundings of the vehicle,
A bird's-eye view conversion unit (S6) that converts an image captured by the camera into a bird's-eye view image, and
A map storage unit (150) that stores a road map (151) including a road surface map, and
A collation processing unit (164) that collates the bird's-eye view image with the road surface map to determine whether or not there is a change point at a position where the actual road surface has changed with respect to the road surface map.
A map change point detection device including a collation area specifying unit (163) that determines a region in the vehicle width direction used for collation from the bird's-eye view image based on the number of road marking lines that can be detected from the bird's-eye view image. Obstacle removal that removes obstacles included in the bird's-eye view image by designating the color of the object included in the road surface map for the bird's-eye view image and converting the colors other than the specified color to black. Equipped with a processing unit (S7)
The map change point detection device according to claim 1, wherein the collation processing unit collates the bird's-eye view image after the obstacle is removed with the road surface map. A camera (110) provided in the vehicle that captures the surroundings of the vehicle,
A bird's-eye view conversion unit (S6) that converts an image captured by the camera into a bird's-eye view image, and
A map storage unit (150) that stores a road map (151) including a road surface map, and
A collation processing unit (164) that collates the bird's-eye view image with the road surface map to determine whether or not there is a change point at a position where the actual road surface has changed with respect to the road surface map.
Obstacle removal that removes obstacles included in the bird's-eye view image by designating the color of the object included in the road surface map for the bird's-eye view image and converting the colors other than the specified color to black. Equipped with a processing unit (S7)
The collation processing unit is a map change point detection device that collates the bird's-eye view image after the obstacle is removed with the road surface map. The map change point detection device according to claim 1 or 2, wherein the collation processing unit converts the distance of the road surface map and collates the road surface map after the distance conversion with the bird's-eye view image. In claim 1,
The collation area specifying unit extracts a collation area, which is an area used for collation, from the bird's-eye view image used for collation based on the number of road marking lines that can be detected from the bird's-eye view image, and also extracts a collation area that is an area used for collation from the road surface map. Specify a certain area determined based on the above as the matching target area,
The collation processing unit
By calculating the correlation value of the matching area of the bird's-eye view image used for matching with respect to the matching target area of the road surface map while changing the position of the matching area, the relative position of the matching area with respect to the matching target area is calculated. And the correlation value map creation unit (S22, S54) that creates a correlation value map that is the relationship between the above and the correlation value.
A score obtained by multiplying the scoring map, which has the highest probability at the current position and the coefficient decreases as the distance from the current position increases, and the correlation value map is calculated for each position of the collation area with respect to the matching target area. A matching position determining unit (S23, S55) for determining the position of the matching region having the highest score as the matching position is provided.
A map change point detection device that collates the bird's-eye view image for collation with the road surface map of the road surface map, using the position of the collation region as the matching position. In claim 5 ,
The collation processing unit
A plurality of frames captured by the camera are combined to generate the first collation area (401).
The creation of the correlation value map by the correlation value map creation unit (S54) and the determination of the matching position by the matching position determination unit (S55) are started from the first collation area.
After the matching position is determined by the matching position determining unit, if the number of frames included in one matching area is larger than the final number of frames, a partial area (402) is determined from the matching area.
With the determined partial area as the collation area, the correlation value map creation unit creates the correlation value map and the matching position determination unit determines the matching position, and the number of frames included in the collation area is the final frame. A map change point detection device that repeats until the number is reached. A distortion correction processing unit (S2) that performs distortion correction and volume distortion correction on the image captured by the camera, and
A vanishing point detection unit (S4) that detects a vanishing point from the image after correction by the distortion correction processing unit, and
A camera orientation detection unit (S5, S5A) that detects the direction of the optical axis of the camera based on the difference in position between the vanishing point and the image center detected by the vanishing point detection unit is provided.
The bird's-eye view conversion unit converts the image captured by the camera into the bird's-eye view image by converting the direction of the optical axis of the camera detected by the camera orientation detection unit from upward to downward. The map change point detection device according to any one of 5 to 5 . In claim 7 ,
A vanishing point region image, which is a part of the image corrected by the distortion correction processing unit and is an image of a region determined based on the vanishing point detected by the vanishing point detection unit, is sequentially determined and time is determined. When matching two consecutive vanishing point region images, the vanishing point region image of the later time is imaged above and below the image of the vanishing point region image of the previous time as a reference. A movement amount calculation unit (S41, S42, S43) that calculates a movement amount to be moved in a direction, and
An integral value calculation unit (S44) that calculates a position shift integral value that is an integral value of the movement amount in the vertical direction of the image calculated by the movement amount calculation unit, and
The moving average value of the positional deviation integrated value calculated by the integrated value calculation unit is sequentially calculated, and the pitch fluctuation amount (δy) which is the difference between the moving average value and the moving amount calculated by the moving amount calculating unit. With a pitch fluctuation amount calculation unit (S45, S46) for calculating
The camera orientation detection unit (S5A) detects the orientation of the optical axis of the camera based on the difference in position between the vanishing point and the image center detected by the vanishing point detection unit and the pitch fluctuation amount. Map vanishing point detector .