JP5305750B2 - Vehicle periphery display device and display method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide "a vehicle surroundings display apparatus and a display method" for correcting distortion of captured data, and displaying an image having less difference from a real object. <P>SOLUTION: The vehicle surroundings display apparatus has an imaging means for imaging surroundings of a vehicle using an imaging lens and producing imaged data of two dimensional coordinate which is captured by the imaging lens, an image processing means for correcting the captured data, and a displaying means for displaying the captured data which is corrected by the image processing means on a display. The image processing means corrects the imaged data so that a change rate of image height of an object existing in the periphery (a range of H2) is increased, while the change rate of image height of the object existing in the center (a range of H1) decreases. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to a vehicle periphery display device having a function of capturing an image of the periphery of a vehicle with an imaging camera and displaying the captured image data on a display or the like, and particularly relates to a technique for correcting distortion of the image data.

  The vehicle periphery display device is equipped with an imaging camera having a wide-angle lens at a corner or a license plate of the vehicle, and displays imaging data around the vehicle imaged by the imaging camera on a display. As a result, the user can be aware of the dangers of automobiles, passers-by, and other obstacles around the vehicle. In particular, it is possible to provide effective assistance to the driver when parked in a narrow parking lot or garage, or when traveling on a narrow road.

  Patent Document 1 includes an optical device that can be switched between a first mode in which light incident from the front is directed to the image pickup means and a second mode in which light incident from the left-right direction is directed to the image pickup means. When traveling or entering the main road, the mode is switched to the second mode, and left and right imaging data is displayed on the display.

JP-A-6-278531

  At intersections with poor visibility where there are walls on both the left and right sides of the vehicle, it is necessary for the driver to enter the intersection until the driver can see the situation on the approach road. If the discovery was delayed, there was a risk of collision with the approaching vehicle.

  Therefore, there is a wide view system that displays using a special wide-angle lens with a wide viewing angle in order to capture the situation in the left-right direction in front of the vehicle at an intersection with poor visibility as described above. As a result, the left and right visual fields can be imaged with a wide viewing angle of 180 degrees or more.

  However, since such a special wide-angle lens is configured by combining a plurality of lenses, there is a problem that the cost is higher than that of a general-purpose fish-eye lens. Furthermore, when a special wide-angle lens is used, there is a problem that the calculation processing for converting the viewpoint of the imaging data and displaying an image looking down from directly above the own vehicle (hereinafter referred to as a top view image) becomes complicated. is there.

  On the other hand, when an object is imaged with a fisheye lens, distortion occurs such that the image height of the object far from the optical axis is small and the image height of the object near the optical axis is large. It is difficult to predict the distance to the object because it is displayed smaller than when viewed with. Furthermore, since a relatively large distortion appears in the shape of the object shown in the center of the imaged data, the user is given an impression different from the actual object, and a sense of incongruity occurs.

  FIGS. 1A and 1B schematically show display examples around the vehicle when a fisheye lens is used. FIG. 1A is a display example when the vehicle 1 and the lane 2 that are relatively far from the imaging camera are imaged. The size of the vehicle 1 at a position away from the optical axis, that is, at the periphery of the imaging data, is displayed much smaller than when viewed with the naked eye, giving the illusion that it exists at a position far from the actual position. End up. For example, when the user reverses the vehicle, such a display may cause a collision.

  Moreover, as shown in FIG.1 (b), it is a display example when the vehicle 1 and the lane 2 which are a short distance to the imaging camera are imaged. The vehicle 1 in the vicinity of the optical axis, that is, in the center of the imaging data, is displayed with a distorted shape because the image height decreases toward the periphery, and the vehicle 1 cannot be accurately recognized. For example, even when a small car such as a minivan is actually adjacent, it may be misunderstood as a large car such as a wagon because the image is distorted. is there.

The present invention solves the above-described problems, and an object of the present invention is to provide a vehicle periphery display device capable of correcting the distortion of imaging data and performing display while suppressing a sense of incongruity with an actual object. To do.
Another object of the present invention is to provide a vehicle periphery display device that can perform display with less distortion using a relatively inexpensive fisheye lens.

  The vehicle periphery display device according to the present invention displays the vehicle periphery, images the vehicle periphery using an imaging lens, and generates imaging data of two-dimensional coordinates captured by the imaging lens; Image processing means for correcting the imaging data, and display means for displaying the imaging data corrected by the image processing means on a display, wherein the image processing means is at a certain distance or more from the optical axis of the imaging lens. The imaging data is corrected so that the change rate of the image height of an object increases. Preferably, the image processing unit further corrects the imaging data so that a change rate of an image height of an object that is less than the predetermined distance is reduced.

  The vehicle periphery display device according to the present invention displays the vehicle periphery, images the vehicle periphery using an imaging lens, and generates imaging data of two-dimensional coordinates captured by the imaging lens; Image processing means for correcting the imaging data, and display means for displaying the imaging data corrected by the image processing means on a display, wherein the image processing means is incident on an imaging lens of the imaging means. The imaging data is corrected so that the rate of change of the image height of an object whose angle is equal to or greater than a certain angle increases. Preferably, the image processing unit further corrects the imaging data so that a change rate of an image height of an object that is less than the certain angle is reduced.

  Preferably, the corrected imaging data includes image data in which an image height is reduced in a first region corresponding to less than a certain distance from the optical axis of the imaging lens, and a second corresponding to the certain distance or more. Image data in which the image height is increased in the region.

Preferably, the image processing unit corrects the values of k1, k3, k5, k7, and k9 when the imaging unit generates imaging data according to the following equation (where θ is an angle between the optical axis and incident light, f Is the focal length of the imaging lens, r (θ) is the distortion at the distance between the image point and the origin, (Cx, Cy) is the image principal point of the imaging data, and (x, y) is the input image (image principal point) M _u and m _v are the number of pixels in the horizontal and vertical directions, and k1, k3, k5, k7, and k9 are coefficients).

  Furthermore, the vehicle periphery display method according to the present invention images a vehicle periphery using an imaging lens, generates imaging data of two-dimensional coordinates imaged by the imaging lens, and displays the vehicle periphery using the generated imaging data. Imaging data such that the rate of change of the image height of an object that is a certain distance or more from the optical axis of the imaging lens increases and the rate of change of the image height of an object that is less than the certain distance decreases. And a step of displaying the corrected imaging data on a display.

  According to the present invention, the imaging data is such that the rate of change of the image height of an object that is less than a certain distance from the optical axis of the imaging lens decreases and the rate of change of the image height of an object that is greater than a certain distance increases. Therefore, even a wide-angle lens such as a fish-eye lens in which the image height of an object away from the optical axis is small and the image height near the optical axis is large, distortion due to optical and distance effects is suppressed. Thus, it is possible to display a vehicle peripheral image with little discomfort with an actual object. Furthermore, since a general-purpose wide-angle lens can be used for the vehicle periphery display device, the cost can be reduced. Furthermore, by using a general-purpose wide-angle lens, image conversion processing of captured data is facilitated, and a combination with a top view image is also facilitated.

  The best mode for carrying out the present invention will be described in detail with reference to the drawings.

  FIG. 2 is a block diagram showing the configuration of the vehicle periphery display device according to the embodiment of the present invention. The vehicle periphery display device 10 includes an imaging unit 20 including an imaging camera that captures the periphery of the vehicle, and an image processing unit 30 that receives imaging data captured by the imaging unit 20, corrects the imaging data, and generates image data. And a display control unit 40 that displays the generated image data on a display.

  The imaging unit 20 includes at least one imaging camera, and the imaging camera can be mounted, for example, at the rear, side, or front of the vehicle. The imaging camera preferably includes a fisheye lens having a large viewing angle, and the imaging camera outputs imaging data around the vehicle imaged by the fisheye lens.

  In a wide-angle lens with a large viewing angle such as a fisheye lens, the degree of distortion differs greatly depending on aberrations or the like in the vicinity of the optical axis of the lens and its peripheral portion. The image processing unit 30 corrects such a distortion, thereby correcting the distortion of the shape of the object and its image, the distortion generated depending on the position in the plane of the imaging data, and the like. The image processing unit 30 is executed, for example, by a general-purpose central processing unit of a computer device or by a dedicated image processing processor. The image processing unit 30 may include a program for controlling hardware as well as processing by hardware.

  FIG. 3 is a diagram for explaining the relationship between an object imaged by a fisheye lens and its image. When the optical axis of the fisheye lens L is Z and the viewing angle is about 180 degrees, the three-dimensional space represented by the radius s by the fisheye lens L is an imageable range. M indicates a measurement plane where the object p imaged by the fisheye lens L exists, and I indicates an image plane where the image P exists when the object p is imaged by the fisheye lens L. (X, y) is the coordinates of the image on the image plane I, (Cx, Cy) is the image principal point, θ is the incident angle of the object p, and φ is the angle formed with the x axis of the image P. The relationship between the object p and the image P shown in FIG. 3 can be expressed by the following equation.

Here, r (θ) is the distance between the image P on the image plane I and the origin 0, and means distortion. Also, m _u and m _v are the number of unit pixels in the horizontal and vertical directions, f is the focal length of the fisheye lens, and (U, V) are image coordinates after distortion correction.

  When the object p on the measurement plane M is imaged by the fisheye lens L, it is converted into an image P on the image plane I. The image processing unit 30 performs correction from the correspondence between the object p on the measurement plane M, the distance d1 to the intersection P ′ where the optical axis Z intersects the measurement plane M, and the distance d2 to the image P and the origin 0.

  As described above, in the image data captured by the fisheye lens L, the image height of the peripheral portion is smaller than the image height near the optical axis. In other words, the difference Δd between the distance d1 and the distance d2 increases in the peripheral portion of the imaging data, and the difference Δd between the distances d1 and d2 decreases in the vicinity of the optical axis. In this embodiment, the difference Δd in the vicinity of the optical axis is increased so as to reduce the difference Δd in the peripheral portion, thereby reducing the overall distortion of the imaging data.

  In the above equation, r (θ) is determined by the optical design of the lens, and can be represented by, for example, Euler's approximate equation. K1, k3, k5, k7, and k9 in the approximate expression are coefficients. By changing these coefficients, the image (x, y) of the image plane I is corrected, and the distortion-corrected image (U, V) ) Can be obtained. The distortion-corrected image (U, V) has a small difference Δd in the peripheral portion and a large difference Δd in the vicinity of the optical axis, thereby correcting the degree of distortion depending on the position in the image plane.

  Next, a preferred correction example of the vehicle periphery display device of this embodiment will be described. FIG. 4 shows a display area on the display of the imaging data imaged by the fisheye lens. Each pixel of the display area 100 corresponds to an image P (that is, a distortion-corrected image) on the image plane I shown in FIG. Here, the coordinates of the central region 110 in the vicinity of the optical axis are A1, A2, A3, and A4, and the coordinates of the left and right peripheral regions 120 and 130 adjacent thereto are B1 to B4 and B5 to B8.

  FIG. 5 shows a specific correction example of the central region and the peripheral region. The vertical axis is the image height, and the horizontal axis is the angle of view (corresponding to the distance d1 or d2 of the object from the optical axis). A curve K1 shows the relationship between the angle of view and the image height when following the optical design value of the fisheye lens L, and a curve K2 shows the relationship between the angle of view and the image height when distortion correction is performed according to this embodiment. The field angle range H1 corresponds to the central region 110 shown in FIG. 4, and the field angle range H2 corresponds to the peripheral region 120 or 130. In the angle of view range H1, the change rate of the image height after distortion correction is smaller than the change rate of the image height of the optical design value. In the angle of view range H2, the change rate of the image height after distortion correction is the optical design. It is larger than the rate of change of the image height of the value.

  FIG. 6 shows a display when distortion correction is performed according to the present embodiment. FIG. 6 (a) corresponds to FIG. 1 (a), and FIG. 6 (b) corresponds to FIG. 2 (b). . Compared with FIG. 6A and FIG. 1A, in the image data subjected to the distortion correction, the image height change in the peripheral portion is increased, so that the vehicle 1 is displayed larger than the conventional one. As a result, the user can easily grasp the sense of distance to the vehicle 1. Further, when FIG. 6B is compared with FIG. 1B, in the image data subjected to the distortion correction, the change in the image height at the center is reduced, so that the distortion of the shape of the vehicle 1 is displayed smaller than the conventional one. The

  In the above example, one central region 110 and two peripheral regions 120 and 130 are set, but the range and number to be set can be changed as appropriate. For example, four peripheral areas may be set at four corners of the imaging data. Further, the distortion correction may be varied at a plurality of levels for each region. For example, as shown in FIG. 7, the display area 100 is set to a plurality of elliptical areas 140, 150, 160 as the distance from the optical axis increases, and the change rate of the image height of each elliptical area 140, 150, 160 is set. You may make it adjust.

  The vehicle periphery display device 10 according to the present embodiment can also apply the imaging data captured by the imaging unit 20 to a top view image whose viewpoint is converted from directly above the vehicle. FIG. 8 is a display example of a top view image. The top view image 200 includes a front image F, left and right images SR and SL, and a rear image R of the vehicle. Each of the images F, SR, SL, and R is obtained by performing viewpoint conversion on imaging data captured by an imaging camera having a wide-angle lens. In order to image the periphery of the vehicle with a small number of imaging cameras, it is necessary to use a wide-angle lens with a large viewing angle, and in this case as well, distortion occurs in the imaging data. Therefore, a top view image with less distortion can be obtained by applying the distortion correction of the present embodiment to each of the images F, SR, SL, and F.

  The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to specific embodiments according to the present invention, and various modifications can be made within the scope of the gist of the present invention described in the claims. Deformation / change is possible.

  The vehicle periphery display device according to the present embodiment may function as a part of a system incorporated in a navigation device or a video device mounted on a vehicle, for example, or may function as a single unit. . Furthermore, the vehicle periphery display device may be a device that issues a warning to an object in combination with a measurement technique that measures the distance to the imaged object.

In a conventional vehicle periphery display device, FIG. 1A shows a display example when an object at a long distance is imaged, and FIG. 1B shows an example display when an object at a short distance is imaged. It is a block diagram which shows the structure of the vehicle periphery display apparatus which concerns on the Example of this invention. It is a figure explaining the relationship between the object imaged with the fisheye lens, and its image. It is a figure which shows the display area on the display of the imaging data imaged with the fisheye lens. It is a figure explaining the correction example of a center area | region and a periphery area | region. In the vehicle periphery display device of the present embodiment, FIG. 6A shows a display example when an image of a long-distance object is imaged, and FIG. 6B shows a display example when an image of a short-distance object is imaged. It is a figure explaining the other distortion correction method in a present Example. It is a figure which shows the top view image to which a present Example is applied.

Explanation of symbols

1: vehicle 2: lane 10: vehicle periphery display device 20: imaging unit 30: image processing unit 40: display control unit 100: display region 110: central region 120, 130: peripheral region 200: top view image

Claims (7)

A vehicle periphery display device for displaying a vehicle periphery,
An imaging means for imaging the surroundings of the vehicle by using a fisheye lens and image pickup data of the two-dimensional coordinates that have been captured by the fisheye lens,
Image processing means for correcting the imaging data;
Display means for displaying the imaging data corrected by the image processing means on a display;
Wherein the image processing means compares the relationship between the angle and the image height in accordance with the optical design values of the fisheye lens, the image height of the rate of change of the object from the optical axis of the fisheye lens beyond a certain distance of the optical design values The imaging data is corrected so that the change rate of the image height of the object that is larger than the change rate of the image height and less than the certain distance is smaller than the change rate of the image height of the optical design value
Vehicle periphery display device. The corrected imaging data includes image data in which an image height is reduced in a first region corresponding to less than a certain distance from the optical axis of the fisheye lens, and in a second region corresponding to the certain distance or more. The vehicle periphery display device according to claim 1, comprising image data with an increased image height. A vehicle periphery display device for displaying a vehicle periphery,
An imaging means for imaging the surroundings of the vehicle by using a fisheye lens and image pickup data of the two-dimensional coordinates that have been captured by the fisheye lens,
Image processing means for correcting the imaging data;
Display means for displaying the imaging data corrected by the image processing means on a display;
Wherein the image processing means compares the angle and an image height relationship in accordance with the optical design values of the fisheye lens, the object changes the incident angle of the object image height in more than a certain angle to the fisheye lens of the image pickup means The imaging data is adjusted such that the rate of change of the image height of the optical design value is larger than the rate of change of the image height of the optical design value, and the rate of change of the image height of the object that is less than the certain angle is smaller than the rate of change of the image height of the optical design value. The vehicle periphery display device to be corrected. The corrected imaging data was includes image data the image height in the first region is decreased corresponding to less than a certain angle from the optical axis of the fisheye lens, the second corresponding to the above said predetermined angle The vehicle periphery display device according to claim 3 , comprising image data in which an image height is increased in a region. The image processing means corrects the values of k1, k3, k5, k7, and k9 when the imaging means generates imaging data according to the following equation (where θ is the angle between the optical axis and the incident light, and f is the fish The focal length of the eye lens, r (θ) is the distortion at the distance between the image point and the origin, (Cx, Cy) is the image principal point of the imaging data, (x, y) is the input image (image principal point is the origin) The vehicle periphery display device according to any one of claims 1 to 4, wherein m _u and m _v are horizontal and vertical pixel numbers, and k1, k3, k5, k7, and k9 are coefficients).

The vehicle periphery display device according to claim 1, wherein the image processing means includes means for converting the corrected imaging data into a viewpoint and converting the image data into image data such that the own vehicle is looked down from directly above. Imaging the surroundings of the vehicle by using a fisheye lens, and image pickup data of the two-dimensional coordinates that have been captured by the fisheye lens, a vehicle peripheral display method for displaying a peripheral vehicle using the generated imaged data,
Compared to the relationship between the angle of view and the image height according to the optical design value of the fisheye lens, the change rate of the image height of an object that is more than a certain distance from the optical axis of the fisheye lens is higher than the change rate of the image height of the optical design value. Correcting the imaging data so that the rate of change of the image height of an object that is large and less than the certain distance is smaller than the rate of change of the image height of the optical design value ;
Displaying the corrected imaging data on a display;
A vehicle periphery display method comprising:

Images