JP2002120675A - Vehicle surroundings monitoring support system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problems that a conventional vehicle surroundings monitoring support system requires the installation of many monitoring cameras in all-around surroundings monitoring and poses poor cost-effectiveness and adverse installation positions. SOLUTION: A direct-vision image shot at every given interval by an electronic camera 2 arranged to shoot a route ahead of one's own vehicle is converted into a bird's-eye planar image, which is stored in a planar image memory 15. The planar movement of the vehicle is measured between times of shooting by the electronic camera 2. On the basis of the measured planar movements, the bird's-eye planar images shot at respective times are composited to form a composite image, which is displayed on a display 8 for monitoring of the sides of the vehicle.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle periphery monitoring support system for monitoring surroundings of an automobile using a monitoring camera, and more particularly to a vehicle capable of performing side monitoring using a front or rear monitoring camera. The present invention relates to a peripheral monitoring support device.

[0002]

2. Description of the Related Art As a preventive safety measure for automobiles, improvement of peripheral monitoring support function by a vehicle monitoring camera is promoted. For example, a front monitoring camera detects a front obstacle and a white line for a traveling lane to prevent a rear-end collision. Attempts have been made to provide automatic warning / inter-vehicle distance control, lane departure warning / autopilot control, and the like. Similarly, the rear-facing surveillance camera performs backward driving support by detecting a rear obstruction, and a rear side approaching vehicle detection warning when changing lanes on an expressway.

For example, Japanese Patent Laid-Open Publication No. Hei 10-211849 discloses a technique for monitoring and controlling a vehicle, which converts a direct-view image of a rear-view camera into a bird's-eye view image and displays a white line or a host vehicle display. Techniques for performing are presented. Japanese Patent Application Laid-Open No. 10-11585 discloses an "object detection device", which discloses a technique relating to white line detection and front obstacle detection by two vehicle front monitoring cameras.
Japanese Patent Application Publication No. 107096 “Vehicle Forward Monitoring Method” also discloses an obstacle detection technique using one forward monitoring camera. Further, Japanese Patent Application Laid-Open No. 7-17328, entitled "Vehicle Peripheral Recognition Assistance Device,"
This document describes an example in which a surveillance camera and a laser range finder for distance measurement are used to monitor the periphery of the entire surrounding area.

[0004]

In the case of the prior art as described above, a large number of surveillance cameras need to be installed in order to monitor the surroundings of the entire surroundings, which increases the cost and increases the cost.
It has not yet reached the stage where it can be widely used due to the problem of installation locations. In particular, the fact that there is no sufficient value in terms of function / cost for side monitoring has actually been recognized.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and a function other than its original function is added to a front monitoring camera or a rear monitoring camera.
Even if a side monitoring camera is not installed, a side monitoring can be performed using a front monitoring camera or a rear monitoring camera, thereby increasing the overall function / cost and promoting the spread, and improving various safety. It is a first object to obtain a vehicle periphery monitoring support device having a function. In addition, even in special situations where monitoring images cannot be obtained, an auxiliary function has been added that allows the user to know the limits of subtle width adjustment and forward / backward movement, and as a whole, highly practical vehicle periphery monitoring The second purpose is to obtain a support device.

[0006]

According to the present invention, there is provided an apparatus for monitoring surroundings of a vehicle, which is arranged so as to photograph the front of the course of the own vehicle and performs photographing at predetermined time intervals. Of the direct-view image
Image conversion means for converting n number of direct-view images from the first direct-view image to the latest n-th direct-view image to a bird's-eye plane image, and converting the first bird's-eye plane image converted by the image conversion means to the n-th bird's-eye plane A plane image memory for storing n bird's-eye plane images leading to an image, a movement amount measuring means for measuring the movement amount of the vehicle at predetermined time intervals, and n stored in the plane image memory.
Combining means for combining the bird's-eye plane images so as to sequentially add together according to the moving amount of the vehicle measured by the moving amount measuring means to form a combined image; and combining the combined image formed by the combining means. It has a display device for displaying.

[0007] An electronic camera arranged to photograph the front of the course of the vehicle and photographing at predetermined time intervals, and an n-th latest one from the first direct-view image of the direct-view image taken by the electronic camera. Image conversion means for converting n direct-view images leading to a direct-view image into a bird's-eye plane image, and n bird's-eye plane images from the first bird's-eye plane image converted by the image conversion means to the n-th bird's-eye plane image , A moving amount measuring means for measuring the moving amount of the vehicle at every predetermined time, and an n-1 th bird's-eye plane image stored in the n-th direct-view image and the plane image memory. A combining means for combining the n-1 bird's-eye plane images leading to the bird's-eye plane image of the own vehicle so as to be successively added in accordance with the moving amount of the own vehicle measured by the moving amount measuring means; Formed by synthetic means Those having a display device for displaying the composite image. The electronic camera is configured to take a picture when the own vehicle speed is equal to or higher than a predetermined speed.

[0008] Further, an electronic camera arranged so as to photograph the front of the course of the vehicle and photographing at a predetermined moving distance of the vehicle, and a first direct-view image of a direct-view image photographed by the electronic camera. Image conversion means for converting the n number of direct-view images leading to the latest n-th direct-view image into a bird's-eye plane image, and n from the first bird's-eye plane image converted by the image conversion means to the n-th bird's-eye plane image A plane image memory for storing a plurality of bird's-eye plane images, and n bird's-eye plane images stored in the plane image memory are combined so as to be successively added in accordance with the moving amount of the own vehicle to form a combined image. Combining means;
A display device for displaying a composite image formed by the synthesizing means is provided.

Further, an electronic camera arranged to photograph the front of the course of the vehicle and photographing every predetermined amount of movement of the vehicle, and a first of direct-view images photographed by the electronic camera.
Image conversion means for converting n number of direct-view images from the first direct-view image to the latest n-th direct-view image to a bird's-eye plane image, and converting the first bird's-eye plane image converted by the image conversion means to the n-th bird's-eye plane A planar image memory for storing n bird's-eye plane images leading to an image, and n-1 from an n-th direct-view image and a first bird's-eye plane image stored in the plane image memory to an (n-1) -th bird's-eye plane image Combining means for combining a plurality of bird's-eye plane images so as to be successively added in accordance with the moving amount of the vehicle to form a combined image, and a display device for displaying the combined image formed by the combining means It is. The electronic camera is configured to take a picture when the vehicle speed is lower than a predetermined speed.

The image processing apparatus further includes a synthetic image memory for storing a synthetic image, wherein the synthesizing means synthesizes in advance n-1 bird's-eye plane images from the first bird's-eye plane image to the (n-1) -th bird's-eye plane image. And a composite image with the n-th image is formed using the composite image stored in the composite image memory. Further, the synthesizing means includes:
The m-th bird's-eye plane image is deleted from the composite image. The composite image is represented so that a portion corresponding to the n-th image is visually classified.

Further, the composite image is formed so as to include the side of the own vehicle. Further, the position of the own vehicle is displayed in the composite image. In the composite image, the predicted course direction of the own vehicle is displayed.

[0012] Further, the image conversion means provides a conversion limit line for converting the direct-view image into a bird's-eye plane image in front of the course, and converts the direct-view image of the own vehicle side from the conversion limit line into a bird's-eye plane image. The conversion limit line is changed according to the own vehicle speed. Further, the composite image is configured to be partially enlarged.

A plurality of electronic cameras are installed, and images synthesized by the plurality of electronic cameras are synthesized to form a synthesized image. Furthermore, the composite image displays one or both of the latest moving time and the moving distance of the own vehicle in the straight traveling direction and the lateral direction between the latest photographing time and the immediately preceding photographing time, and the cumulative moving distance obtained by accumulating the moving distance. Things.

The electronic cameras are arranged in front of and behind the own vehicle, and form a composite image using the electronic cameras arranged in front of and behind the own vehicle.

[0015]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiment 1 FIG. 1 is a block diagram showing a configuration of a vehicle periphery monitoring support device according to Embodiment 1 of the present invention. In FIG. 1, reference numeral 1 denotes a control unit having a built-in microprocessor and various memories to be described later, 2 denotes an electronic camera such as a CCD camera installed in a vehicle for monitoring the front of the course, and 3 denotes a rotating unit provided on an axle, for example. The movement amount is measured by the angle sensor. Reference numeral 4 denotes a steering angle sensor provided on a steering shaft, for example, and the rotation angle sensor 3 and the steering angle sensor 4 constitute a movement amount measuring means. Reference numeral 5 denotes a zoom-up switch such as an eight-way switch for instructing an enlarged display portion. Reference numeral 6 denotes a distance display switch for resetting the current distance display and displaying the travel distance of the vehicle after the reset in two axes (straight direction and lateral direction). , 7 are shift lever switches that respond to the position of the course direction selection lever, and 8 is a display device using, for example, liquid crystal. Electronic camera 2 and various sensors
The switch and the display device 8 are connected to the control unit 1.

The configuration inside the control unit 1 is as follows. 10 is a microprocessor, 1
1 is a serial transmission circuit provided between the microprocessor 10 and the electronic camera 2, 12 is an input interface provided between the microprocessor 10 and various sensors and switches, and 13 is a direct-view image memory connected to the microprocessor 10. It is. The electronic camera 2 uses NTSC (N
national Television System
An image signal according to the (Committee) standard is supplied to the microprocessor 10 via the transmission circuit 11, and the microprocessor 10 sends the image signal to the direct-view image memory 13 corresponding to each pixel on the virtual screen of, for example, 256 dots × 240 dots. Thus, chromaticity information and luminance information are written as, for example, 8-bit digital information.

Reference numeral 14 denotes a conversion parameter memory in which actual dimensions corresponding to the dimensions between pixels in the direct-view image are stored.
Reference numeral 5 denotes a plane image memory for storing a bird's-eye plane image obtained by subjecting each pixel position of the direct-view image memory 13 to magnification conversion based on the contents of the conversion parameter memory 14, and 16 a latest bird's-eye plane image and a past bird's-eye view. A synthesized image memory for storing a synthesized image obtained by synthesizing a two-dimensional image by the synthesizing means, 17 is a serial transmission circuit for displaying the contents of the synthesized image memory 16 on the display device 8, and 18 is the current time since the previous photographing by the electronic camera 2. The plane movement measurement memory 18 stores the movement distance of the vehicle up to the photographing. The plane movement measurement memory 18 stores two signals in the straight traveling direction and the lateral direction of the vehicle based on the signals of the rotation angle sensor 3 and the steering angle sensor 4. It measures the amount of dimensional movement.

FIG. 2 is a flowchart illustrating the operation of the vehicle periphery monitoring support device according to the first embodiment of the present invention. FIG. 3 is a view showing a bird's-eye view plane image of the vehicle periphery monitoring support device according to the first embodiment of the present invention.
FIG. 3A shows an example of a direct-view image obtained by the electronic camera 2, and FIG. 3B is a bird's-eye view plane image corresponding to the direct-view image of FIG. In FIG. 3, 100a is a virtual whole screen, 101 is a horizon, 102a and 103a are white lines for displaying left and right lanes, and 104 is a conversion limit line in front of the vehicle when converting to a bird's-eye view plane image. The position of the horizon 101
It is determined by the mounting angle and mounting height of the electronic camera. 100b is a bird's-eye plane image, and 102b and 103b are white lines for displaying left and right lanes. The actual distance between the pixels in the direct-view image 100a increases as the distance increases, and the information is stored in the conversion parameter memory 14 as an approximate expression or table information. The actual distance between the pixels in the bird's-eye plane image 100b is uniform, and the left white line 102b and the right white line 103b are displayed as parallel lines. However, the bird's-eye plane image is displayed by changing the scale factor in the horizontal direction and the vertical direction of the screen, and the field of view in the vertical direction is widened compared to the horizontal direction.
The triangular black portion drawn in the lower corner of FIG.
2 shows a portion that is not included in the field of view of the electronic camera 2.

FIG. 4 is a diagram for explaining creation of a composite image by the vehicle periphery monitoring support device according to the first embodiment of the present invention. FIG. 4A shows a direct view image of the electronic camera 2 according to time. FIG. 4 is a diagram showing a converted bird's-eye plane image, and FIG.
FIG. 4B is a diagram for explaining the concept of image movement, and FIG.
(C) is a combined image obtained by combining the latest bird's-eye plane image with the past bird's-eye plane image. In FIG. 4, reference numeral 105 denotes the own vehicle, reference numeral 106 denotes a virtual fixed point on the road, and reference numerals 110 to 114 denote bird's-eye view plane images obtained by converting direct-view images captured at the old time T110 to the new time T114. Reference numeral 115 denotes a composite image.

FIG. 5 is an enlarged image of the vehicle periphery monitoring support device according to the first embodiment of the present invention. In FIG. 5, reference numeral 107 denotes a wheel of the vehicle 105, reference numeral 108 denotes a distance display in which a total plane movement distance after the distance display switch 6 is operated is displayed two-dimensionally, and reference numeral 109 denotes a course direction display. It shows the expected course based on. Reference numeral 116 denotes an enlarged display screen.

Next, the operation will be described with reference to FIGS. 3 and 4 based on the flowchart of FIG. In FIG. 2, step 200 is an operation start step, and step 201 is
This is a vehicle speed determination process following the start process.
If it is less than Km / H, the process proceeds to step 202 and the vehicle speed becomes 4
If 0 km / H or more, the process proceeds to step 204. In steps 202 and 204, the plane movement amount of the vehicle after the reset operation is performed in step 214 described later is measured, and the measurement result every moment is stored in the plane movement amount measurement memory 18 in FIG. It has become. The measurement of the plane movement amount is performed by counting the pulses generated by the rotation angle sensor 3 and storing the counted value as a vector movement distance L converted into the actual movement distance of the vehicle. Based on the angle information, it is decomposed into a moving distance Lz in the Z-axis direction, which is the straight traveling direction of the vehicle (a monitoring center direction of the electronic camera 2), and a moving distance Lx in the X-axis direction orthogonal to the Z-axis direction. is there.

In step 203 following step 202, the measurement of the plane movement amount in step 202 is continued again until the vector movement distance L updated and stored in the plane movement amount measurement memory 18 reaches, for example, 33.3 cm. , 33.3c
This is a moving distance determining step configured to move to step 206 when m is exceeded. Step 205 following step 204
Until the measurement time of the movement distance by the plane movement amount measurement memory 18 reaches, for example, about 30 msec, the measurement of the plane movement amount in the step 204 is continued again, and
This is a measurement time determination step configured to move to step 206 when ec is exceeded. The vehicle speed is V = 40km
In / H, the measured vector movement distance L (cm) of the vehicle between T = 30 msec is calculated as follows. ^{L = (V × 10 5/} 3600) × T × 10 -3 = V × T /
36 = 33.3cm Therefore, if the vehicle speed is V = 180Km / H, 30ms
The vector movement distance L between ec is 150 cm. Also, the time T (msec) required for the vehicle speed to reach V = 39.9 Km / H and the vector movement distance L = 33.3 cm is reached.
c) is calculated as follows: T = (L / (V × 10 5/3600)) × 10 3 = 36
L / V = 30 msec Therefore, if the vehicle speed is V = 4 Km / H, the time required to reach the vector movement distance L = 33.3 cm is 3
00 msec.

Step 206 is a direct-view image capturing step of capturing the direct-view image from the electronic camera 2 into the direct-view image memory 13.
The step 207 following the step 206 is a bird's-eye plane image conversion step of converting the direct-view image into a bird's-eye plane image and writing it into the plane image memory 15.
This is as shown in FIGS. 3 (a) and 3 (b).

Next, the image moving step 20 following the step 207
8. The image combining step 209 will be described in detail with reference to FIG. In FIG. 4A, the old time T110 to the new time T1
In the bird's-eye view images 110 to 114 obtained by converting the direct-view images captured in 14 respectively, the fixed point 106 on the road is approaching as the vehicle 105 advances. FIG. 4B shows the concept of image movement, for example, at time T111.
The bird's-eye plane image 111 is moved upward by the moving distance of the vehicle during this time from the bird's-eye plane image 110 at time T110, and as a result, the road fixed point 106 is drawn at the same horizontal position. Time T11
The same applies to the bird's-eye plane images 112 to 114 at 2 to T114. The bird's-eye plane images 110 to 114 are moved to positions sequentially moved by the moving distance of the vehicle between the respective times. They are aligned.

In FIG. 4C, a composite image 115 obtained by successively adding past bird's-eye plane images 110 to 113 to the latest bird's-eye plane image 114 and extending is extended to several meters behind the vehicle, and is further extended. Older images located rearward are sequentially updated and deleted. With the combined image 115, side monitoring can be performed.

In the above description, the case where the vehicle is moving straight has been described. However, when the vehicle is moving in a curved manner, based on the biaxial movement components of the plane movement amount measurement memory 18. Screen movement is performed, and the left white line 102b and the right white line 103b are displayed as curves.

In the above description, the composite image 115 is generated by using a large number of bird's-eye plane images 110 to 114. However, the previously generated composite image and the next bird's-eye plane image are synthesized, If the latest composite image is obtained and the old bird's-eye plane image or the previous composite image is deleted, the capacity of the image memory is reduced.

Further, when a preceding vehicle is present at a position close to the front of the course, if the bird's-eye view plane image at each time is moved by the plane movement distance between the times, the preceding vehicle is located at a different road surface position. Will be displayed. Therefore, images shown at positions that do not overlap with each other can be regarded as being caused by a moving object and removed from the composite image.

Returning to FIG. 2, step 210 following step 209
FIG. 1 shows a process of determining whether or not to perform enlarged display.
When the zoom-up switch 5 is not operated, the process proceeds to the whole display step 211. When the zoom-up switch 5 is operated, the enlarged portion instruction reading step 21 is performed.
2 and the zoom-up switch 5, which is an 8-way switch, switches the front two wheels, left front one wheel, left front and rear two wheels, left rear one wheel, rear two wheels, right rear one wheel, right front and rear two wheels, right It is determined which one of the front wheels is instructed, and in a subsequent step 213, an enlarged display of the instructed portion is performed. FIG. 5 shows an enlarged display screen 116 of the left two wheels. The distance display 108 and the steering angle sensor 4 which display the total plane movement distance after the distance display switch 6 in FIG. The course direction display 109 indicating the predicted course based on the route is performed.

The plane movement amount measurement memory 18 in FIG. 1 is reset every time the electronic camera 2 takes a picture. However, a cumulative plane movement amount measurement memory (not shown) is provided to collect the total value of the plane movement amount. Memorize and store this value in distance display 1
08 may be displayed two-dimensionally. This accumulated value can be prevented from being reset even when the distance display switch 6 is operated. When the distance display switch 6 is operated, for example, if the user gets off the vehicle and measures and measures the movable distance to a groove or an obstacle in advance, the moving distance of the vehicle thereafter is displayed by the distance display 108. Therefore, it is possible to delicately adjust the width or move forward / backward with peace of mind. This display function can be used even when the electronic camera 2 is not taking a picture or in a driving operation for an area that cannot be monitored by the electronic camera 2. It is valid.

Returning to FIG. 2, step 211 or step 213
In step 214 following step 214, the plane movement amount measurement memory 18 in FIG. 1 is reset, and the process shifts to the start step 200 again through the end step 215.

The processing capacity of the microprocessor 10 assumed here is calculated from the start step 200 to the end step 215.
The processing time up to about 30 msec, and 40 Km / H was used as the threshold value for the vehicle speed determination in step 201 is because the photographing method at a fixed distance interval in step 202 in low-speed operation and step 204 in high-speed operation This is because the same measurement interval time and the same movement amount are used when the threshold speed is 40 Km / H in any case of the photographing method with the fixed time interval according to the above. If the switching decision threshold speed is further reduced,
In addition to shortening the measurement distance interval during low-speed driving, the use of such a fixed-distance interval shooting method and a fixed-time interval shooting method makes it unnecessary to perform meaningless shooting processing (for example, when the vehicle speed is 0, the fixed time interval shooting method is not used). There is no point in taking a photo).

The conversion limit line 10 in FIG.
Assuming that the position of 4 corresponds to the normal braking distance of the vehicle, it is set farther forward when driving at high speed, and is set closer to display at the low speed driving to display a truly effective part on the screen. It can also be adjusted automatically.

In the above description, each of the front monitoring camera and the rear monitoring camera has been described as a single camera. However, two cameras are provided at each of the front and rear left and right sides, and the images of the two monitoring cameras are synthesized. Thus, a monitoring image for one of the traveling directions can be obtained. In addition, it has been described that one of the front monitoring camera and the rear monitoring camera of the vehicle is used to perform the side monitoring while monitoring the front of the course for the forward or backward movement. However, the monitoring cameras are installed in front and rear. Also in this case, side monitoring can be performed by similar screen synthesis. In such a case, the bird's-eye plane image obtained by the forward monitoring and the bird's-eye plane image obtained by the rear monitoring are effectively used.
A synthesized image including the side monitoring image is quickly generated, and the real-time monitoring image area is enlarged to improve safety.

Further, by changing the color tone and shading degree of the past bird's-eye plane image portion combined with the latest bird's-eye plane image portion in the composite image, safer usage can be achieved. For example, if an infant suddenly jumps out,
If this is the latest image area, it is displayed, but there is a danger that it will not be displayed in the past image area, so it is effective as a reminder.

In the above description, the image displayed on the display device 8 is a composite bird's-eye view plane image stored in the composite image memory 16. In the composite image memory 16, the latest image portion is a direct-view image. Can also be used. In this case as well, the color tone may be changed as described above, but a more effective display of the front of the turn can be displayed. That is, when the track is converted into a plane image, the front of the track overflows to the left or right of the display device 8 and cannot be displayed.
If a direct-view image is used, it is possible to display more forward.

According to the first embodiment, since the current side monitoring image of the vehicle can be displayed based on the past monitoring image relating to the front of the course, the front monitoring camera or the rear monitoring camera of the vehicle can be effectively used. As a result, the side monitoring can be performed, and the function / cost of the monitoring camera can be improved. Further, by converting the direct-view image into a bird's-eye plane image and handling it, there is an effect that the image can be easily synthesized and the side monitoring display can be easily understood.

[0038]

Since the present invention is configured as described above, it has the following effects. An electronic camera that is arranged to photograph the front of the course of the vehicle and performs photographing at predetermined time intervals; and n ranging from the first direct-view image of the direct-view image captured by the electronic camera to the latest n-th direct-view image Image conversion means for converting the number of direct-view images into a bird's-eye plane image, and a plane image storing n bird's-eye plane images from the first bird's-eye plane image to the n-th bird's-eye plane image converted by the image conversion means A memory, a moving amount measuring means for measuring a moving amount of the own vehicle every predetermined time, and n bird's-eye view plane images stored in the plane image memory, the moving amount of the own vehicle measured by the moving amount measuring means. And a display device for displaying the combined image formed by the combining means, so that the side of the own vehicle can be monitored using the combined image. Can do .

Further, an electronic camera arranged to photograph the front of the course of the own vehicle and photographing at predetermined time intervals, and an n-th latest one from the first direct-view image of the direct-view image taken by the electronic camera. Image conversion means for converting n direct-view images leading to a direct-view image into a bird's-eye plane image, and n bird's-eye plane images from the first bird's-eye plane image converted by the image conversion means to the n-th bird's-eye plane image , A moving amount measuring means for measuring the moving amount of the vehicle at every predetermined time, and an n-1 th bird's-eye plane image stored in the n-th direct-view image and the plane image memory. A combining means for combining the n-1 bird's-eye plane images leading to the bird's-eye plane image of the own vehicle so as to be successively added in accordance with the moving amount of the own vehicle measured by the moving amount measuring means; Formed by synthetic means Since provided with a display device for displaying the composite image, the composite image can monitor the vehicle side with. In addition, the electronic camera is configured to shoot when the own vehicle speed is equal to or higher than a predetermined speed, so that an appropriate composite image can be formed according to the vehicle speed.

Further, an electronic camera arranged so as to photograph the front of the course of the vehicle and photographing the vehicle at every predetermined movement amount, and a first direct-view image of the direct-view image photographed by the electronic camera. Image conversion means for converting the n number of direct-view images leading to the latest n-th direct-view image into a bird's-eye plane image, and n from the first bird's-eye plane image converted by the image conversion means to the n-th bird's-eye plane image A plane image memory for storing a plurality of bird's-eye plane images, and n bird's-eye plane images stored in the plane image memory are combined so as to be successively added in accordance with the moving amount of the own vehicle to form a combined image. Combining means;
Since the display device for displaying the combined image formed by the combining means is provided, it is possible to monitor the side of the own vehicle using the combined image.

Further, an electronic camera arranged to photograph the front of the course of the vehicle and photographing at a predetermined moving distance of the vehicle, and a first of a direct-view image photographed by the electronic camera.
Image conversion means for converting n number of direct-view images from the first direct-view image to the latest n-th direct-view image to a bird's-eye plane image, and converting the first bird's-eye plane image converted by the image conversion means to the n-th bird's-eye plane A planar image memory for storing n bird's-eye plane images leading to an image, and n-1 from an n-th direct-view image and a first bird's-eye plane image stored in the plane image memory to an (n-1) -th bird's-eye plane image Since a bird's-eye plane image is composed so as to sequentially combine the bird's-eye plane images according to the moving amount of the vehicle to form a composite image, and a display device that displays the composite image formed by the composite unit, It is possible to monitor the side of the own vehicle using the composite image.

Further, the electronic camera is configured to take a picture when the own vehicle speed is lower than a predetermined speed, so that an appropriate composite image can be formed according to the vehicle speed. Furthermore, the image processing apparatus further includes a composite image memory for storing the composite image, wherein the composition unit preliminarily composites n-1 bird's-eye planar images from the first bird's-eye planar image to the (n-1) th bird's-eye planar image, and composes the composite image. Since the composite image stored in the memory and the composite image stored in the composite image memory is used to form a composite image with the n-th image, the composite image can be formed more easily.

Further, the synthesizing means deletes the m-th bird's-eye view plane image from the synthesized image, so that an appropriate synthesized image can always be obtained. In addition, the composite image is represented so that a portion corresponding to the n-th image is visually classified, so that the latest image can be easily understood.

Further, since the combined image is formed so as to include the side of the own vehicle, the side of the own vehicle can be monitored by the combined image. Also, since the position of the own vehicle is displayed in the composite image, monitoring can be performed while confirming the own vehicle position.

Since the predicted course direction of the own vehicle is displayed in the composite image, the course of the own vehicle can be confirmed. Further, the image conversion means provides a conversion limit line for converting the direct-view image into the bird's-eye plane image in front of the course, converts the direct-view image on the host vehicle side from the conversion limit line into a bird's-eye plane image, and converts the conversion limit line. Is changed according to the own vehicle speed, so that it is possible to form a composite image according to the own vehicle speed.

Further, since the composite image is configured to be partially enlarged, it is possible to confirm the partially enlarged image when necessary. In addition, a plurality of electronic cameras are installed,
Since a composite image is formed by combining images captured by a plurality of electronic cameras, a more accurate composite image can be formed.

Further, the synthesized image includes one or both of the moving distance of the own vehicle in the straight-ahead direction and the lateral direction between the latest photographing time point and the immediately preceding photographing time point and the accumulated moving distance. Is displayed, so that the moving distance can be confirmed. In addition, the electronic camera is arranged in front of and behind the own vehicle, and a composite image is formed using the electronic cameras arranged in front of and behind the own vehicle.
It can respond to retreat.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a vehicle periphery monitoring support device according to a first embodiment of the present invention.

FIG. 2 is a flowchart illustrating an operation of the vehicle periphery monitoring support device according to the first embodiment of the present invention.

FIG. 3 is a diagram showing a bird's-eye view planar image of the vehicle periphery monitoring support device according to the first embodiment of the present invention;

FIG. 4 is a diagram illustrating creation of a composite image by the vehicle periphery monitoring support device according to the first embodiment of the present invention.

FIG. 5 is a diagram showing an enlarged image of the vehicle periphery monitoring support device according to the first embodiment of the present invention.

[Explanation of symbols]

1 control unit, 2 electronic camera, 3 rotation angle sensor, 4 steering angle sensor, 5 zoom up switch, 6 distance display switch, 7 shift lever switch, 8 display device, 10 microprocessor, 11,
Reference Signs List 17 transmission circuit, 12 input interface, 13 direct-view image memory, 14 conversion parameter memory, 15 plane image memory, 16 composite image memory, 18 plane movement amount measurement memory, 100a direct-view image, 100b bird's-eye plane image, 101 horizon, 102a left white line (Direct view image), 102b left white line (plane image), 103a right white line (direct view image), 103b right white line (plane image), 10b
4 Conversion limit line, 105 Own vehicle, 106 Fixed point on road, 1
07 wheel, 108 distance display, 109 course direction display, 110 to 114 bird's-eye view plane image at time T110 to T114, 115 composite image, 116 enlarged display screen.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int. Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) B60R 21/00 621 B60R 21/00 621C 624 624C 626 626F 626G 1/00 1/00 A G06T 1/00 330 G06T 1/00 330A H04N 7/18 H04N 7/18 J UV

Claims (17)

[Claims] 1. An electronic camera arranged to photograph the front of the course of a vehicle and taking images at predetermined time intervals, and an n-th (n) image based on a first direct-view image of a direct-view image taken by the electronic camera. Is a positive integer greater than 1) image conversion means for converting n number of direct-view images leading to a direct-view image into a bird's-eye plane image, from the first bird's-eye plane image converted by the image conversion means to the n-th bird's-eye plane image A plane image memory for storing n bird's-eye plane images, a movement amount measuring means for measuring the movement amount of the vehicle at each predetermined time, and the n bird's-eye plane images stored in the plane image memory. Combining means for forming a combined image by combining them sequentially according to the movement amount of the vehicle measured by the movement amount measuring means, and a display device for displaying the combined image formed by the combining means Features Vehicle periphery monitoring support device. 2. An electronic camera arranged so as to photograph the front of the course of the own vehicle and performing photographing at predetermined time intervals, and the latest n-th (n) from the first direct-view image of the direct-view image taken by the electronic camera. Is a positive integer greater than 1) image conversion means for converting n number of direct-view images leading to a direct-view image into a bird's-eye plane image, from the first bird's-eye plane image converted by the image conversion means to the n-th bird's-eye plane image A plane image memory for storing n bird's-eye plane images, a movement amount measuring means for measuring the movement amount of the vehicle at each predetermined time, the n-th direct-view image, and the first image stored in the plane image memory. N-1 bird's-eye plane images ranging from the bird's-eye plane image to the (n-1) th bird's-eye plane image are combined so as to be sequentially added in accordance with the moving amount of the own vehicle measured by the moving amount measuring means. Compositing means for forming a composite image And a display device for displaying a combined image formed by the combining means. 3. The vehicle periphery monitoring support device according to claim 1, wherein the electronic camera is configured to capture an image when the own vehicle speed is equal to or higher than a predetermined speed. 4. An electronic camera that is arranged to photograph the front of the course of the own vehicle and performs photographing every predetermined moving distance of the own vehicle, and an electronic camera that is the latest one from the first direct-view image of the direct-view image captured by the electronic camera. Image conversion means for converting n number of direct-view images leading to an n-th (n is a positive integer greater than 1) direct-view image into a bird's-eye plane image, and converting the first bird's-eye plane image converted by this image conversion means A plane image memory for storing n bird's-eye plane images leading to n bird's-eye plane images, and n bird's-eye plane images stored in the plane image memory are sequentially added according to the moving amount of the vehicle. A vehicle periphery monitoring support device, comprising: combining means for forming a combined image by combining, and a display device for displaying the combined image formed by the combining means. 5. An electronic camera which is arranged so as to photograph the front of the course of the own vehicle and performs photographing at every predetermined moving distance of the own vehicle, and an electronic camera which is the latest from the first direct-view image of the direct-view image taken by the electronic camera. Image conversion means for converting n number of direct-view images leading to an n-th (n is a positive integer greater than 1) direct-view image into a bird's-eye plane image, and converting the first bird's-eye plane image converted by this image conversion means a plane image memory for storing n bird's-eye plane images leading to n bird's-eye plane images; an n-th bird's-eye plane image from the n-th direct-view image and the first bird's-eye plane image stored in the plane image memory Combining means for combining the (n-1) bird's-eye plane images sequentially to form the combined image in accordance with the movement amount of the vehicle, and displaying the combined image formed by the combining means. Features equipped with a device Vehicle periphery monitoring support device. 6. The vehicle periphery monitoring support device according to claim 4, wherein the electronic camera is configured to capture an image when the own vehicle speed is lower than a predetermined speed. 7. A combined image memory for storing a combined image, wherein the combining means combines in advance n-1 bird's-eye plane images from the first bird's-eye plane image to the (n-1) -th bird's-eye plane image. The composite image stored with the composite image memory and a composite image with the n-th image is formed using the composite image stored in the composite image memory. The vehicle periphery monitoring support device according to the above item. 8. The image processing apparatus according to claim 1, wherein the synthesizing unit deletes an m-th (m is a positive integer of m <n) bird's-eye view plane image from the synthesized image. The vehicle periphery monitoring support device according to any one of the preceding claims. 9. The vehicle periphery monitoring according to claim 1, wherein the composite image is represented so that a portion corresponding to the n-th image is visually classified. Support equipment. 10. The vehicle periphery monitoring support device according to claim 1, wherein the composite image is formed so as to include the side of the own vehicle. 11. The vehicle periphery monitoring support device according to claim 1, wherein the position of the own vehicle is displayed in the composite image. 12. The synthesized image displays the predicted course direction of the own vehicle.
The vehicle periphery monitoring support device according to any one of claims 1 to 4. 13. An image conversion means, wherein a conversion limit line for converting a direct-view image into a bird's-eye plane image is provided in front of the course,
While converting the direct-view image of the own vehicle side from the conversion limit line into a bird's-eye plane image, the conversion limit line is, according to the own vehicle speed,
The vehicle periphery monitoring support device according to claim 1, wherein the device is changed. 14. The vehicle periphery monitoring support device according to claim 1, wherein the composite image is configured to be partially enlarged. 15. The electronic camera according to claim 1, wherein a plurality of electronic cameras are installed, and a composite image is formed by combining images captured by the plurality of electronic cameras. A vehicle periphery monitoring support device according to any one of the preceding claims. 16. The combined image includes one or both of a moving distance of the own vehicle in the straight traveling direction and the lateral direction between the latest photographing time point and the immediately preceding photographing time point, and an accumulated moving distance obtained by accumulating the moving distance. The vehicle periphery monitoring support device according to any one of claims 1 to 15, wherein is displayed. 17. The electronic camera according to claim 1, wherein the electronic camera is arranged in front of and behind the own vehicle, and a composite image is formed using the electronic cameras arranged in front of and behind the own vehicle. The vehicle periphery monitoring support device according to claim 16.