JP2014181957A - In-vehicle camera calibration device, in-vehicle camera calibration method, program, and medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an in-vehicle camera calibration device configured to enable more accurate calibration without increasing man-hour and space required, and an in-vehicle camera calibration method.SOLUTION: An in-vehicle camera calibration device 1 includes: an in-vehicle camera 2 which images an irradiation range S irradiated by a head lamp 5 on a plane D located forward of a vehicle; and calculation means 3a which calculates a vanishing point FOE of the imaging range of the in-vehicle camera 2 on the basis of a distance between feature points A, B of the irradiation range S, the plane D, and the in-vehicle camera 2, and a relative position between the in-vehicle camera 2 and the plane.

Description

  The present invention relates to an in-vehicle camera calibration device, an in-vehicle camera calibration method, a program, and a medium that are used for calibration of an in-vehicle camera suitable for use in vehicles such as passenger cars, trucks, and buses.

  In recent vehicles, IPA (Intelligent Parking Assist) for assisting the driver in parking operation and LKA (Lane Keep for assisting steering to maintain the center of the lane on the expressway or exclusive road for driving) A driver assistance system such as Assist may be installed in the vehicle. In this system, the white lines located on both sides of the lane on the road surface are binarized using image data around the vehicle imaged by an in-vehicle camera typified by a CCD camera or a CMOS camera mounted on the vehicle. Is detected.

  When the vehicle-mounted camera is attached to the vehicle at the vehicle manufacturing stage, it is necessary to perform calibration in image processing so that the vehicle-mounted camera can capture a desired direction and range. For this calibration, for example, a plurality of grid-like targets are installed in the imaging range of the in-vehicle camera in front of the vehicle, and the optical axis direction of the in-vehicle camera is calibrated based on the recognized target position. . Regarding the optical axis adjustment of the radar, for example, there is a technique described in Patent Document 1 below.

Japanese Patent No. 0410933

  However, Patent Document 1 is an optical axis adjustment of a radar and cannot be applied to the above-described method. In the above-described method, it is necessary to install a dedicated target, and a space for installing the target and a target for recognizing the target are required. It is necessary to select an appropriate amount of light and background. That is, in the prior art, there arises a problem that man-hours and required space increase.

  In view of the above problems, an object of the present invention is to provide an in-vehicle camera calibration device, an in-vehicle camera calibration method, a program, and a medium that can perform calibration with higher accuracy without increasing man-hours and necessary space. .

  In order to solve the above problem, an in-vehicle camera calibration apparatus according to the present invention includes an in-vehicle camera that captures an irradiation range irradiated by a headlamp on a plane located in front of the vehicle, a feature point of the irradiation range, and the Computation means for computing a vanishing point of the imaging range of the in-vehicle camera from a distance between the plane and the in-vehicle camera and a relative position of the in-vehicle camera with respect to the plane is included. Here, it includes a determination unit that determines whether or not calibration based on the headlamp is possible. When the determination unit determines affirmative, the in-vehicle camera images the irradiation range and the calculation unit The vanishing point may be calculated, and the determination unit includes an acquisition unit that acquires, from an equipment control device, adjustment information including whether or not the headlamp has undergone an adjustment process and position information including the relative position. The determination unit may determine whether the calibration is possible based on the adjustment information and the position information. The irradiation range of the headlamp corresponds to a low beam, and the feature point corresponds to the low beam. It is good also as being a point included in the cut-off line in the irradiation range.

  The vehicle-mounted camera calibration method according to the present invention includes an imaging step of capturing an irradiation range irradiated by a headlamp on a plane located in front of the vehicle with the vehicle-mounted camera, a feature point of the irradiation range, and the vehicle-mounted camera And a calculation step of calculating a vanishing point of the imaging range of the in-vehicle camera from a relative position with respect to the plane. The program of the present invention is a program for executing the method, and the medium of the present invention is a medium storing the program.

  ADVANTAGE OF THE INVENTION According to this invention, the vehicle-mounted camera calibration apparatus and vehicle-mounted camera calibration method which can perform highly accurate calibration, without causing the increase in a man-hour and a required space can be provided.

1 is a schematic diagram illustrating an embodiment of an in-vehicle camera calibration device 1 according to an embodiment of the present invention. It is a schematic diagram which shows the relative positional relationship of the headlamp 5, the vehicle-mounted camera 2, the vehicle V, and the wall surface D in the vehicle-mounted camera calibration apparatus 1 of an Example. It is a schematic diagram which shows the aspect of the irradiation range S and the cut-off line CL in the wall surface D of the headlamp 5 in the vehicle-mounted camera calibration apparatus 1 of an Example. It is a schematic diagram which shows the calculation aspect of the vanishing point based on the parameter which the positional information used in the vehicle-mounted camera calibration apparatus 1 of an Example contains, and a parameter. It is a sequence diagram which shows the processing content of the vehicle-mounted camera calibration apparatus 1 of an Example with the processing content of the tool 4 by the side of an installation. It is a schematic diagram which shows the aspect of the target used for the conventional calibration. It is a schematic diagram which shows the installation position of the target used for the conventional calibration. It is a schematic diagram which shows the irradiation range S in the wall surface D at the time of making the headlamp 5 into a high beam in the vehicle-mounted camera calibration apparatus 1 of an Example.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the accompanying drawings.

  As shown in FIG. 1, the in-vehicle camera calibration device 1 of the present embodiment includes an in-vehicle camera 2 mounted on a vehicle V and an in-vehicle camera ECU 3 (Electronic Control Unit). The in-vehicle camera ECU 3 and the tool 4 provided on the facility side of the factory are connected so as to be communicable with a communication standard such as CAN (Controller Area Network). In addition, the vehicle V includes a pair of left and right headlamps 5 (headlights) at the front end as an existing configuration, and includes a body ECU 6 that controls lighting or extinction of the headlamp 5, and the body ECU 6 and the in-vehicle camera ECU 3 are also configured by CAN. It is connected.

  The in-vehicle camera 2 is, for example, a CCD camera or a CMOS camera, and as shown in FIG. 2, the floor is offset by the camera lateral position dimension W from the vehicle center VC, for example, on the right side of the lower surface of the roof of the vehicle so as to face the front of the vehicle. It is set apart from the surface by a height H. The in-vehicle camera 2 has an irradiation range S as shown in FIG. 3 that is irradiated from the in-vehicle camera 2 to the wall surface D (plane) located in front of the vehicle by the target distance L by the low beam of the headlamp 5 as shown in FIG. The captured image data is output to the in-vehicle camera ECU 3.

  The in-vehicle camera ECU 3 includes, for example, a CPU, a ROM, a RAM, an EEPROM, a data bus that connects them, and an input / output interface. The in-vehicle camera ECU 3 extracts the cut-off line CL and its feature points A and B by recognizing the irradiation range S by an appropriate method such as binarization processing based mainly on luminance in accordance with a program stored in the ROM. And recognize. The in-vehicle camera ECU 3 functions as a calculation unit 3a, a determination unit 3b, and an acquisition unit 3c that perform the following controls.

  The tool 4 includes, for example, a CPU, a ROM, a RAM, an EEPROM, a data bus for connecting them, an input / output interface, and a screen display device. The tool 4 constitutes a storage means 4a for performing each control described below and a transmission means 4b in accordance with a program stored in the ROM, and constitutes an equipment control device for controlling all equipment in the factory. .

  The tool 4 is connected to a CAN mounted on the vehicle V, and also functions as a diagnostic tool for performing self-diagnosis of various ECUs mounted on the vehicle V. The tool 4 also performs overall adjustment of equipment in the factory and controls inspection equipment. .

  As shown in FIG. 4, the cut-off line CL included in the irradiation range S irradiated to the wall surface D has an inclined line that inclines in a direction away from the horizontal direction as the left half goes to the left, and the right half in the horizontal direction. It is in the form of a “U” with a horizontal line that points to it.

  The cut-off line CL corresponding to the left headlamp 5 has a characteristic point A that is an inflection point between the inclined line and the horizontal line, and the cut-off line CL corresponding to the right headlamp 5 is an inflection point between the inclination line and the horizontal line. It has a certain feature point B. The feature points A and B are separated in the horizontal direction by a distance corresponding to the separation distance in the width direction of the left and right headlamps 5 and have the same height.

  The aiming process (adjustment process) for assembling the headlamp 5 to the vehicle V is a process performed before the calibration process of the in-vehicle camera 2 shown in the present embodiment. In this aiming process, the cut-off line height CLH, which is the vertical distance between the horizontal line of the cut-off line CL shown in FIG. 3 and the ground line GL (floor surface in the factory), is adjusted so as to satisfy the regulations of each country. . The regulations of each country are defined according to, for example, the head lamp mounting height, and the cut-off line position is required to be within a predetermined angle from the lamp mounting height.

  This cut-off line height CLH is stored in the storage means 4 a of the tool 4. When the aiming process is completed, adjustment information indicating that the headlamp 5 has undergone the adjustment process is stored and held in the storage unit 4 a of the tool 4.

  The storage means 4a of the tool 4 is the above-described relative positional relationship shown in FIGS. 2 and 3, that is, the wall surface D of the in-vehicle camera 2 having the camera height H, the camera lateral position dimension W, the target distance L, and the cut-off line height CLH. For example, position information including a relative position with respect to the vehicle V is stored in association with the vehicle type and serial number of the vehicle V.

  When the vehicle V is arranged in the calibration process line and the tool 4 is activated by an operator in charge of the calibration process and the adjustment of the in-vehicle camera 2 is started, the transmission means 4b of the tool 4 adjusts corresponding to the vehicle V. A data frame including information and position information is transmitted to the in-vehicle camera ECU 3, and the acquisition unit 3c of the in-vehicle camera ECU 3 receives the data frame, reads out the adjustment information and the position information, and stores these information.

  When the storage is completed, the acquisition unit 3c of the in-vehicle camera ECU 3 transmits a storage completion notification to the tool 4, and after the tool 4 receives the storage completion notification information, the transmission unit 4b of the tool 4 transmits a calibration start notification. It transmits to camera ECU3. The determination unit 3b of the in-vehicle camera ECU 3 determines that the calibration is possible when the calibration start notification is received, and determines that the calibration is impossible when it cannot be received.

  When the determination means 3b determines that calibration is possible, the in-vehicle camera ECU 3 outputs an irradiation command to the body ECU 6 so as to irradiate the low beam of the headlamp 5 toward the wall B. Thereafter, the in-vehicle camera ECU 3 images the front wall surface D including the irradiation range S and sets the cut-off line CL of the irradiation range S. The calculating means 3a uses the following equation to calculate the vanishing point from the coordinates and position information of the extracted feature points A and B, the focal length FL stored in itself, the horizontal pixel size HPS and the vertical pixel size VPS in the image. Calculate the FOE coordinates.

  Here, the X coordinate of the vanishing point FOE is FOEX, the Y coordinate is FOEY, the X coordinate of the midpoint C of the feature point A and the feature point B is CX, the Y coordinate is CY, the X of the feature point A and the feature point B, If the Y coordinates are AX, AY, BX, and BY, respectively, the arithmetic expression is as follows.

That is,
CX = (AX + BX) / 2,
CY = (AY + BY) / 2,
FOEX = CX + (FL × W / L / HPS)
FOEY = CY− (FL × (H−CLH) / L / VPS)
It is.

  The in-vehicle camera ECU 3 calculates the vanishing point FOE from the above-described arithmetic expression, and then sets a recognition frame in the image from the vanishing point FOE, for example. Setting a recognition frame reduces processing load and increases recognition accuracy.

  The control content of the on-vehicle camera calibration device 1 of the present embodiment described above will be described with reference to the sequence diagram shown in FIG. In step S1, the tool 4 is activated by an operator in charge of the calibration process, and it is determined whether or not the adjustment of the in-vehicle camera 2 has been started. If yes, the process proceeds to step S2, and if no, the process returns to step S1. .

  In step S2, the transmission means 4b of the tool 4 transmits a data frame including adjustment information and position information to the in-vehicle camera ECU 3. Correspondingly, the acquisition unit 3c of the vehicle-side camera ECU 3 on the vehicle side receives the data frame in step S3, and reads and stores the adjustment information and the position information from the data frame.

  When the storage in step S3 is completed, the acquisition unit 3c of the in-vehicle camera ECU 3 transmits a storage completion notification to the tool 4 in step S4, and the tool 4 receives this storage completion notification information in step S5.

  Subsequently, in step S6, the transmission means 4b of the tool 4 transmits a calibration start notification to the in-vehicle camera ECU 3. In step S7, the determination unit 3b of the in-vehicle camera ECU 3 determines whether or not the calibration start notification is received. If the determination is affirmative, it is determined that the calibration is possible, and if the determination is negative, the calibration is not possible. It is determined that there is, and the calibration process from step S8 to step S11 is not executed, and the control is terminated.

  In step S7, when it is determined that the calibration is possible by the determination unit 3b, the body ECU 6 irradiates the low beam of the headlamp 5 toward the wall surface D based on the irradiation command of the in-vehicle camera ECU 3 in step S8. In step S9, the in-vehicle camera 2 images the wall surface D including the irradiation range S, and in step S10, the in-vehicle camera ECU 3 extracts the cut-off line CL and the coordinates of the feature points A and B based on image processing.

  In step S11, the calculation means 3a calculates the vanishing point from the coordinates and position information of the feature points A and B, the focal length FL stored by itself, the horizontal pixel size HPS in the image, and the vertical pixel size VPS by the above-described calculation formula. Calculate the FOE coordinates. In the sequence diagram shown in FIG. 4 described above, the on-vehicle camera calibration method according to the present invention is executed.

  According to the in-vehicle camera calibration apparatus 1 of the present embodiment realized by these control contents and the in-vehicle camera calibration method executed thereby, the following operational effects can be obtained. That is, the parameters related to the cut-off line CL based on the regulations of each country can be extracted, and the in-vehicle camera 2 can be calibrated by obtaining the vanishing point FOE based on the position information and a predetermined arithmetic expression.

  Conventionally, a target having a lattice pattern of the form shown in FIG. 7 is installed at, for example, three points P1, P2, and P3 arranged in parallel in the width direction shown in FIG. 6, and the in-vehicle camera ECU 3 recognizes the position of the in-vehicle camera 2 There was a need. In this embodiment, such a target can be eliminated, and the man-hours in the calibration process and the space for calibration can be reduced.

  Further, in the conventional calibration, it was necessary to avoid the presence of a pattern similar to a lattice pattern in the front of the target, that is, in front of the vehicle. The wall surface can be used, and the selection of the pattern can be made unnecessary. Further, it is possible to eliminate the need to separately irradiate the target with a fluorescent lamp or the like even at night. In addition, since the cut-off line CL has a large luminance difference and can be easily extracted at night, the calibration accuracy can be improved.

  In addition, in the present embodiment, the calibration process is performed only when the cut-off line CL of the irradiation range S based on the irradiation on the wall surface D of the headlamp 5 is reliably adjusted based on the regulations of each country in the aiming process which is the previous process. Therefore, calibration with higher accuracy can be performed.

  Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the above-described embodiments, and various modifications and substitutions are made to the above-described embodiments without departing from the scope of the present invention. be able to.

  For example, in the embodiment described above, the inflection point of the low beam cut-off line is used as the feature point. This is based on the fact that the regulations of each country define the low beam cut-off line at present. For this reason, when there is a definition for the high beam, the feature points at the corners of the rectangular irradiation range S of the high beam shown in FIG. 8 can be used.

  Further, the wall surface D may be a flat surface and a screen or the like may be used. In addition to the tools used at the factory, the tools may be used at the dealer.

  According to the in-vehicle camera calibration device, the in-vehicle camera calibration method, the program, and the medium of the present invention, it is possible to calibrate the in-vehicle camera with higher accuracy without increasing man-hours and space. Therefore, the present invention is useful when applied to various vehicles such as passenger cars, trucks, and buses.

1 In-vehicle camera calibration device 2 In-vehicle camera 3 In-vehicle camera ECU
3a Calculation means 3b Determination means 3c Acquisition means 4 Tool (equipment control device)
4a Storage means 4b Transmission means 5 Headlamp (headlight)
6 Body ECU
D Wall surface (plane)
CL Cut-off line CLH Cut-off line height A Feature point (left)
B Feature point (right)
S irradiation range FL focal length H camera height W camera horizontal position dimension L target distance HPS horizontal pixel size VPS vertical pixel size FOE vanishing point

Claims (7)

  A vehicle-mounted camera that captures an irradiation range irradiated by a headlamp on a plane located in front of the vehicle; a feature point of the irradiation range; a distance between the plane and the vehicle-mounted camera; and a relative position of the vehicle-mounted camera with respect to the plane. The vehicle-mounted camera calibration apparatus characterized by including the calculating means which calculates the vanishing point of the imaging range of the said vehicle-mounted camera from these.   A determination unit configured to determine whether or not calibration based on the headlamp is possible, and when the determination unit determines affirmative, the in-vehicle camera images the irradiation range, and the calculation unit includes the vanishing point. The vehicle-mounted camera calibration device according to claim 1, wherein   The determination unit includes an acquisition unit that acquires, from an equipment control device, adjustment information including whether or not the headlamp has undergone an adjustment step and position information including the relative position, and the determination unit includes the adjustment information and the position. The in-vehicle camera calibration apparatus according to claim 2, wherein the calibration possibility determination is performed based on information.   The irradiation range of the headlamp corresponds to a low beam, and the feature point is a point included in a cut-off line in the irradiation range corresponding to the low beam. The on-vehicle camera calibration device according to one item.   An imaging step of imaging an irradiation range irradiated by a headlamp on a plane located in front of the vehicle with an in-vehicle camera, imaging of the in-vehicle camera from a feature point of the irradiation range and a relative position of the in-vehicle camera with respect to the plane An on-vehicle camera calibration method comprising a calculation step of calculating a vanishing point of a range.   The program which performs the vehicle-mounted camera calibration method of Claim 5.   A medium storing the program according to claim 6.

Images