JP2018136739A - Calibration device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To highly accurately achieve calibration processing of a parameter of a rotational angle around one shaft in an imaging surface in a short time.SOLUTION: A calibration device 116 which performs calibration of a camera that is mounted on a vehicle and photographs the periphery of the vehicle includes: a feature point extraction section 117 which extracts a feature point of the photographed image from a plurality of frames obtained by photographing at time intervals; a steering angle acquisition section 119 which acquires a steering angle of the vehicle; a feature point sequence calculation section 120 which calculates a length of a feature point sequence that is a straight line obtained by displacement of the feature point between a set of frames of the time intervals, when it is determined that the vehicle is travelling straight based on the steering angle acquired by the steering angle acquisition section 119; and a rotational angle calibration section 122 which calibrates a parameter of a rotational angle around one shaft in an imaging surface so that ratios of travelling distances in which the vehicle has traveled between one set of the frames to the length of the feature point sequence calculated by the feature point sequence calculation section 120 coincide each other between different sets of frames.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a calibration apparatus.

  In recent years, many systems have been proposed for assisting a driver using images captured by a camera installed in a vehicle. As an example, there are a bird's-eye view image display system for confirming the surroundings of the vehicle, an expected route line display system when the vehicle moves backward, an automatic parking system, and the like.

  The camera is attached to the vehicle at a position / angle according to a design value, but at that time, an error may occur in the position / angle. In order to suppress such an error, a calibration process called calibration is performed on camera parameters such as the position and angle of the camera.

  In general, calibration at the time of shipment of a vehicle is performed in an empty state where no one is on board. Therefore, when the state of the vehicle after calibration is in an empty state where calibration has actually been performed, no deviation occurs in the image of the camera.

  However, when the user actually uses the vehicle, the number of passengers, the seating location, the loading state of the luggage, and the like change variously. When the loading state of the vehicle changes, the posture of the vehicle also changes, and accordingly, the camera installation state with respect to the ground also changes. Also, the camera posture may change due to aging. That is, the camera parameter changes and an error occurs. There has been a problem that the image captured by the camera is shifted due to the error of the camera parameter.

  In order to solve such a problem, Patent Documents 1 to 3 below disclose a method of calibrating camera parameters while a vehicle is traveling. In this method, feature points are extracted and tracked from an image captured by a camera installed in a vehicle. The basic principle of this method is to correct camera parameters using a linear feature point series (calibration index) resulting from two-dimensional movement of feature points between frames.

JP 2011-217233 JP Japanese Patent Laid-Open No. 2014-48803 International Publication No.2012 / 139636

  By the way, consider a case where the rotation angle parameter about one axis in the imaging plane perpendicular to the optical axis of the camera is calibrated. When the rotation angle parameter about one axis is calibrated by the method disclosed in Patent Documents 1 to 3 above, one of the frames obtained by the imaging of the camera while the vehicle is traveling is one that is in chronological order. It is necessary to obtain the length of two feature point series using a set of frames. In other words, it is necessary to simultaneously extract two feature points between a set of frames and simultaneously track the two extracted feature points.

  For this reason, there is a problem that the time required for the calibration processing of the parameter of the rotation angle about one axis becomes longer as a result of the longer time required to extract and track the feature points. In addition, the success rate until two feature points are extracted and tracked at the same time becomes extremely low, resulting in a problem that the accuracy of calibration processing of a parameter of a rotation angle around one axis is lowered.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a calibration device that can realize a calibration process of a rotation angle parameter about one axis in an imaging surface in a short time and with high accuracy. To do.

  The calibration device according to the present invention uses a ratio of the length of a feature point series serving as a calibration index of a rotation angle parameter about one axis in the imaging surface and the travel distance of the vehicle, This is to calibrate the rotation angle parameter.

  That is, the calibration device according to the present invention is a calibration device that performs calibration of a camera that is mounted on a vehicle and images the periphery of the vehicle, and from a plurality of frames obtained in time series by the imaging, The vehicle travels straight on the basis of the feature point extraction unit that extracts the feature point of the captured image, the steering angle acquisition unit that acquires the steering angle of the vehicle, and the steering angle acquired by the steering angle acquisition unit. A feature point sequence calculating unit that calculates a length of a feature point sequence that is a straight line in which the feature points are displaced between a set of frames that are in series with each other when determined to be, and the set Around one axis in the imaging plane so that the ratio between the distance traveled by the vehicle between the frames and the length of the feature point series calculated by the feature point series calculation unit matches between different sets of frames. Rotation angle parameter It characterized by having a a rotation angle correction unit for calibrating.

  According to the calibration device according to the present invention configured as described above, the calibration processing of the rotation angle around one axis in the imaging surface can be realized in a short time and with high accuracy.

1 is a block diagram showing a schematic configuration of a calibration system equipped with a calibration apparatus according to an embodiment of the present invention. 1 is a block diagram illustrating a schematic configuration of a calibration apparatus according to an embodiment of the present invention. It is explanatory drawing which shows an example of a camera coordinate system. It is a flowchart which shows the flow of the calibration process performed with a calibration apparatus. It is a figure which shows an example of the bird's-eye view image in the process of the calibration process performed with a calibration apparatus. It is a figure which shows an example of the bird's-eye view image in the process of the calibration process performed with a calibration apparatus. It is explanatory drawing for demonstrating the calibration process of a yaw angle.

  Hereinafter, specific embodiments of a calibration device according to the present invention will be described with reference to the drawings. In the following, a case where four cameras 111 to 114 are used will be described.

<Schematic configuration of calibration system>
FIG. 1 shows a schematic configuration of a calibration system equipped with a calibration apparatus according to an embodiment of the present invention. The calibration system 100 is installed in a vehicle 1 whose front wheels are steering wheels.

  The calibration system 100 mainly includes four cameras 111 to 114, a calculation device 101, a RAM (Random Access Memory) 102, a ROM (Read Only Memory) 103, a display device 104, a vehicle speed sensor 105, and the like. A steering angle sensor 106, a yaw rate sensor 107, an input device 108, and a communication device 109.

  Each camera 111-114 is mounted in the vehicle 1, for example, is installed in the front, back, left, and right of the vehicle 1. For example, the front and rear cameras are attached to the vehicle body near the license plate, and the left and right cameras are attached to the lower part of the side mirror. Here, the camera 111 is attached to the front of the vehicle 1, the camera 112 is attached to the rear of the vehicle 1, the camera 113 is attached to the left of the vehicle 1, and the camera 114 is attached to the right of the vehicle 1.

  The front camera 111 and the rear camera 112 are attached such that the optical axis direction coincides with the front-rear direction of the vehicle 1. The left camera 113 and the right camera 114 are attached so that the optical axis direction coincides with the vertical direction of the vehicle 1. Each of the cameras 111 to 114 is attached according to known design information decided in advance, but an attachment error has actually occurred, and this error is unknown. Moreover, the camera provided with the wide-angle fish-eye lens is employ | adopted for each camera 111-114 so that the image of the perimeter of the vehicle 1 can be acquired. In order to acquire a wide-angle image, the fish-eye camera distorts the image based on a known distortion function. Four images captured by the cameras 111 to 114 are output to the arithmetic device 101.

  The vehicle speed sensor 105, the rudder angle sensor 106, and the yaw rate sensor 107 are sensors for detecting the vehicle speed, the rudder angle, and the yaw rate, respectively. Sensor information detected by each sensor is output to the arithmetic unit 101, and the arithmetic unit 101 calculates Used in processing. The steering angle sensor 106 detects the tire angle of the front wheels as the steering angle.

  The input device 108 is a device that accepts user operations such as switches and buttons. The input device 108 is used for turning on / off a calibration function, initializing a calibration result, changing a calibration method, and the like. Various types of information input to the input device 108 through user operations are output to the arithmetic device 101.

  The communication device 109 is a device used for communication with an external device (not shown). The arithmetic device 101 receives various information from an external device via the communication device 109 and outputs various information calculated by the arithmetic device 101 to the external device.

  In the RAM 102, numerical data required in the arithmetic processing process in the arithmetic device 101, program variables for processing results during the arithmetic processing, and the like are written. The written data is read out as necessary in the arithmetic processing process of the arithmetic unit 101 and used for arithmetic processing. The RAM 102 also stores image data captured by the cameras 111 to 114.

  In the ROM 103, for example, a program for executing calibration and information used without rewriting among information necessary for the program are stored in advance. For example, design values (external parameters) of installation positions and angles (pitch angle, yaw angle, roll angle, camera height) of the cameras 111 to 114, focal lengths, pixel sizes, optical axis centers of the cameras 111 to 114, Stores camera parameters such as distortion functions (internal parameters).

  The arithmetic device 101 receives various information transmitted from the cameras 111 to 114, the vehicle speed sensor 105, the rudder angle sensor 106, the yaw rate sensor 107, the input device 108, the communication device 109, and the like. Using the received information, the arithmetic unit 101 performs a calculation process based on a program or the like.

  For example, the arithmetic device 101 converts the viewpoints of the images input from the cameras 111 to 114 and generates an overhead image as if looking down at the ground from above. More specifically, an image obtained by removing image distortion is generated using a known distortion function stored in advance in the ROM 103 or the like for images captured by the respective cameras 111 to 114 including fisheye cameras.

  Each image from which the distortion has been removed is converted into an image viewed from an overhead viewpoint based on a known design value relating to camera attachment stored in advance in the ROM 103 or the like (image generation device 115). In such viewpoint conversion processing, a specific pixel of the overhead image and specific pixels of each camera 111 to 114 corresponding thereto are calculated using a well-known camera geometric conversion formula, and the luminance value of the pixel is calculated as the overhead image. This can be realized by assigning to the pixels.

  When the corresponding pixel includes a decimal point and there is no corresponding pixel, a process of assigning an intermediate luminance of peripheral pixels is performed by a known luminance interpolation process. The arithmetic device 101 executes arithmetic processing using the output results of the vehicle speed sensor 105, the steering angle sensor 106, the yaw rate sensor 107, and the communication device 109, and executes operation program switching processing according to the input results of the input device 108. To do.

  Further, the arithmetic device 101 is equipped with a calibration device 116 that calibrates each of the cameras 111 to 114 so that the bird's-eye view image that has undergone viewpoint conversion becomes an image that looks down on the vehicle 1 from directly above. The configuration of the calibration device 116 will be described later.

  The display device 104 receives the processing result of the arithmetic device 101 and presents the processing result to the user using a display or the like. For example, a bird's-eye view image obtained by converting viewpoints of four images of the cameras 111 to 114 is displayed to the user. The display device 104 can also switch display contents according to the output of the arithmetic device 101, such as displaying only the image of the camera 112 that captures the rear of the vehicle 1.

<Configuration of calibration device>
FIG. 2 shows a schematic configuration of a calibration apparatus according to an embodiment of the present invention. The calibration device 116 is mounted on the vehicle 1 and calibrates each of the cameras 111 to 114 that images the periphery of the vehicle. The calibration device 116 includes a feature point extraction unit 117, a speed acquisition unit 118, a steering angle acquisition unit 119, a feature point series calculation unit 120, a feature point series accumulation unit 121, and a camera parameter calibration unit 122 (rotation angle). Calibration unit).

  The feature point extraction unit 117 extracts feature points of the captured image from a plurality of frames obtained in time series by the imaging of the cameras 111 to 114. The speed acquisition unit 118 acquires the detection result of the traveling speed detected by the vehicle speed sensor 105. The steering angle acquisition unit 119 acquires the detection result of the steering angle detected by the steering angle sensor 106. Details of the feature point series calculation unit 120, the feature point series accumulation unit 121, and the camera parameter calibration unit 122 will be described later.

<Camera coordinate system>
FIG. 3 is an explanatory diagram illustrating an example of a camera coordinate system. Hereinafter, an example of the camera coordinate system (X, Y, Z) will be described with reference to FIG. In this camera coordinate system (X, Y, Z), the optical axis direction of the camera is defined as the Z-axis direction. A surface IS indicated by shading in the drawing indicates an imaging surface IS that is perpendicular to the optical axis (Z-axis) of the camera. In this camera coordinate system (X, Y, Z), the X axis and the Y axis are taken in parallel with the imaging surface IS.

  Pitching in the figure refers to rotation around the X axis, yawing refers to rotation around the Y axis, and rolling refers to rotation around the Z axis. Parameters of rotation angles around the X, Y, and Z axes are referred to as a pitch angle, a yaw angle, and a roll angle, respectively.

  Like the front camera 111 and the rear camera 112, when the optical axis direction coincides with the front-rear direction of the vehicle 1, the front-rear direction is defined as the Z-axis direction, and the vehicle 1 is perpendicular to the front-rear direction (Z-axis direction). The axis extending in the vertical direction is defined as the Y axis.

  On the other hand, when the optical axis direction coincides with the vertical direction of the vehicle 1 like the left camera 113 and the right camera 114, the vertical direction is defined as the Z-axis direction, and the vertical direction (Z-axis direction) An axis extending in the left-right direction of the vertical vehicle 1 is defined as an X axis.

<Flow of a series of calibration processes performed by the calibration device>
Hereinafter, the flow of a series of calibration processes performed by the calibration apparatus will be described with reference to the flowchart shown in FIG. 4 and FIGS. The following description is based on the premise that the camera parameters (pitch angle, yaw angle, roll angle, camera height) of the front camera 111 are calibrated.

  First, in step S101, the front camera 111 captures an image of the periphery of the vehicle 1 traveling straight ahead in order to obtain the length of the feature point series serving as a camera parameter calibration index. The frame F obtained by the imaging is output to the feature point extraction unit 117. The feature point series refers to a straight line formed by displacing feature points between a pair of frames that follow each other in time series when the vehicle 1 is traveling straight ahead. The road on which the vehicle 1 travels preferably has white lines and road markings with many characteristic points.

  In step S102, the vehicle speed sensor 105 and the steering angle sensor 106 detect the vehicle speed and the steering angle, respectively. The vehicle speed information and the steering angle information obtained by the detection are output to the speed acquisition unit 118 and the steering angle acquisition unit 119 via an in-vehicle network such as CAN (Controller Area Network) as a communication protocol.

  In step S103, the feature point series calculation unit 120 determines whether or not the running state of the vehicle 1 is an appropriate running state for obtaining the length of the feature point series. When the traveling speed of the vehicle 1 is not stable, it may be difficult to track feature points. For example, when the vehicle 1 suddenly accelerates or decelerates, the attitude of the vehicle 1 varies greatly. Under the influence of the fluctuation, an image obtained by imaging with the front camera 111 becomes an image inappropriate for obtaining the length of the feature point series. Therefore, the camera parameter calibration unit 122 determines whether or not the vehicle 1 is traveling straight on the basis of the steering angle information acquired by the steering angle acquisition unit 119.

  When it is determined that the vehicle 1 is traveling straight (YES in step S103), the traveling state of the vehicle 1 is in an appropriate traveling state for obtaining the length of the feature point series, and the process proceeds to step S104. On the other hand, when it is determined that the vehicle 1 is not traveling straight (NO in step S103), since the traveling state of the vehicle 1 is in an unsuitable traveling state for obtaining the length of the feature point series, the process is step. Return to S101.

  In step S104, the feature point series calculation unit 120 extracts a point with a high edge strength from the first acquired frame F1 as a feature point. Such a feature point extraction technique has already been well-known, for example, proposed by a technique using a Harris operator, and will not be described in detail.

  In step S105, the feature point series calculation unit 120 calculates the length of the feature point series. The length of the feature point series is calculated by tracking from which point the feature point has moved to which point on the imaging surface IS between the pair of frames F1 and F2. Such a feature point tracking technique is already well known, such as proposed by the LK (Lucas-Kanade) method. Therefore, although detailed description is omitted, in this embodiment, tracking of feature points is continued as much as possible.

  For example, even after the process of tracking where the feature point extracted from the first frame F1 has moved to the point in the second frame F2, the frame FX (X = 3 to N), a coordinate value indicating the same point is calculated. Of course, in order to increase the success rate of feature point tracking, it is desirable that the number of feature point series is larger.

  Therefore, in step S106, the feature point series accumulation unit 121 accumulates the feature point series. The calibration of the front camera 111 can be realized if two feature point series can be acquired. Hereinafter, an example of acquiring two feature point series will be described with reference to FIGS. 5 and 6.

  As shown in FIGS. 5A and 5B, a feature point P1 (white line drawn on the road) between the 300th frame F300 and the 301st frame F301, which are in chronological order. The feature point series length D1 (Euclidean distance D1 in the world coordinate system), which is a straight line displaced at the corner of WL1, is acquired as the length of the first feature point series. Subsequently, as shown in FIGS. 6A and 6B, a feature point P2 (drawn on the road) between the 600th frame F600 and the 601st frame F601, which are in chronological order. The feature point series length D2 (Euclidean distance D2 in the world coordinate system) that is a straight line in which the corner of the white line WL1 is displaced is acquired as the length of the second feature point series.

  However, there is a possibility that an error may occur in the information acquired from the feature point extraction result, the feature point tracking process, the vehicle speed sensor or the steering angle sensor. For this reason, it is desirable to accumulate a large amount of information indicating feature points, travel speeds, and steering angles in order to reduce errors.

  For example, when the feature point extraction process is performed while the vehicle 1 is traveling on a road that is not sufficiently paved and is not horizontal with respect to the vehicle 1, it is not sufficient as a calibration index for pitch angle, yaw angle, and roll angle. It is. Therefore, there is a possibility that a large amount of error may occur in the finally calculated camera parameters (pitch angle, yaw angle, roll angle, camera height). Even if the length of a feature point series that is insufficient as a calibration index is calculated, if a large amount of correct feature point series is accumulated as a calibration index, it is caused by a feature point series that is insufficient as a calibration index. It is possible to mitigate errors that occur in camera parameters (pitch angle, yaw angle, roll angle, camera height).

  In step S107, the camera parameter calibration unit 122 determines whether or not the number of feature point series has reached a predetermined number. If it is determined that the number of feature point series has reached the required number (YES in step S107), the accumulation of feature point series for the required number is completed, and the process proceeds to step S108. On the other hand, when it is determined that the number of feature point series has not reached the required number (NO in step S107), accumulation of the necessary number of feature point series has not been completed, and the process returns to step S101.

  In step S108, the camera parameter calibration unit 122 calibrates the pitch angle (the rotation angle parameter about the X axis shown in FIG. 3) using the design values of the camera mounting position and angle stored in the ROM 103. . Various methods for correcting the pitch angle have been introduced, and any of these methods may be used. In this embodiment, the feature point series accumulated in step S107 is used.

  The pitch angle calibration method using the feature point series is already known, for example, as proposed in International Publication No. 2012/139636. Therefore, although detailed description is omitted, in the present embodiment, the pitch angle is calibrated so that two or more feature point series subjected to overhead conversion are parallel.

  In step S109, the camera parameter calibration unit 122 calibrates the yaw angle (the parameter of the rotation angle around the Y axis shown in FIG. 3).

First, in the camera parameter calibration unit 122, a feature point series indicating the same point of the Nth frame FN and the (N + 1) th frame FN + 1 is converted from the image coordinate system to the world coordinate system. This conversion method is already well known. Therefore, although detailed description is omitted, in this embodiment, for example, the feature coordinates in the world coordinates of the frame FN are (P _X1 , P _Y1 ), and the feature coordinates in the world coordinates of the frame FN + 1 are (P _X2 , P _Y2). ), The Euclidean distance D in the world coordinate system between the frame FN and the frame FN + 1 is calculated as the length of the feature point series using the following equation (1).
... (1)

Next, the camera parameter calibration unit 122 calculates a travel distance d [km] traveled by the vehicle 1 between the frame FN and the frame FN + 1 based on information indicating the travel speed of the vehicle 1. The travel distance d [km] between the frame FN and the frame FN + 1 uses the following formula (2), where the travel speed between the frames is vk [km /] and the number of frames per second is f. Is calculated. The traveling speed vk is output from the speed acquisition unit 118 to the camera parameter calibration unit 122.
... (2)

  As shown in the above equation (2), as the traveling speed of the vehicle 1 when moving from the frame FN to the frame FN + 1, an arithmetic average value of the traveling speeds acquired in the respective frames is employed.

  For example, as shown in FIG. 5, the traveling speed of the vehicle 1 when moving from the frame F300 (FIG. 5A) to the frame F301 (FIG. 5B) was acquired in each of the frames F300 and F301. An arithmetic average value of travel speed is adopted.

Next, the camera parameter calibration unit 122 normalizes the accumulated feature point series by dividing the Euclidean distance D by the travel distance d as shown in the following equation (3). R is calculated. As shown in Expression (3), the normalized length R is a value that depends on the travel distance of the vehicle 1.
... (3)

  Next, the camera parameter calibration unit 122 calibrates the yaw angle so that the normalized length R matches between different sets of frames. Specifically, when the travel distance between the pair of frames F300 and F301 shown in FIG. 5 is d1 [km] and the travel distance between the frames F600 and F601 shown in FIG. 6 is d2 [km], The yaw angle is calibrated so that the normalized length R1 (= D1 / d1) matches the normalized length R2 (= D2 / d2).

Such a yaw angle calibration technique is already well known, such as proposed by the LM method (Levenberg-Marquardt Method). Therefore, although detailed description is omitted, in the present embodiment, first, an arithmetic average X _CENTER of the X coordinate of the feature point series is calculated between the frame FN and the frame FN + 1. Next, the arithmetic mean X _CENTER and the normalized length R are voted into a two-dimensional voting space as shown in FIG. Next, the voted principal components are analyzed, and an approximate straight line as shown by a broken line in FIG. 7 is calculated. Next, the yaw angle at which the calculated slope of the approximate curve is closest to 0 ° is estimated as the corrected yaw angle.

  In step S110, the camera parameter calibration unit 122 calibrates the roll angle (the rotation angle parameter about the Z axis shown in FIG. 3) using the design values of the camera mounting position and angle stored in the ROM 103. . Various methods for calibrating the roll angle have been introduced, and any of these methods may be used. In this embodiment, the roll angle is calibrated by the method proposed in International Publication No. 2012/139636. Done. Specifically, the roll angle when the angle of the feature point series accumulated in step S107 is parallel to the traveling direction of the vehicle 1 is calibrated as the roll angle of the front camera 111.

  In step S <b> 111, the camera parameter calibration unit 122 calibrates the camera height of the front camera 111 using the design values of the camera attachment position and angle stored in the ROM 103. Various methods for calibrating the camera height have been introduced, and any of these methods may be used. In this embodiment, the camera height can be adjusted by the method proposed in International Publication No. 2012/139636. Calibration is performed. Specifically, the camera height when the Euclidean distance D calculated in step S109 is equal to the travel distance d is calculated as the height of the front camera 111.

  As described above, according to the calibration device of the present embodiment, one feature point (feature point P1 or feature point P2) between a pair of frames (between frames F300 and F301 or between frames F600 and F601). Extraction and tracking are performed. Therefore, two feature points P1 and P2 are extracted simultaneously between a set of frames (between frames F300 and F301 or between frames F600 and F601), or the two extracted feature points P1 and P2 are simultaneously extracted. There is no need to track.

  That is, only one feature point is extracted and tracked between a set of frames. As a result, it is possible to shorten the time required for extracting and tracking feature points compared to the case where two feature points are extracted and tracked simultaneously between a set of frames. Therefore, the time required for the yaw angle calibration process can be shortened.

  In addition, as described above, the number of feature points to be extracted and tracked between a set of frames is one, so that compared to the case where two feature points are simultaneously extracted and tracked between a set of frames. , Improve the success rate of extraction and tracking. For this reason, the yaw angle calibration process can be realized with high accuracy.

  Therefore, the yaw angle calibration process can be realized in a short time with high accuracy.

  In addition, according to the calibration apparatus of the present embodiment, calibration using a specified marker that is performed at the time of factory shipment becomes unnecessary.

  In the above embodiment, it is assumed that the camera parameters (pitch angle, yaw angle, roll angle, camera height) of the front camera 111 are calibrated. However, it is not limited to this. Of course, the camera parameters (pitch angle, yaw angle, roll angle, camera height) of the rear camera 112, the left camera 113, and the right camera 114 may be calibrated.

  Similar to the front camera 111, the rear camera 112 has an optical axis direction (Z-axis direction) that coincides with the front-rear direction (traveling direction) of the vehicle 1. The calibration of the parameter (yaw angle) of the rotation angle about the vertical direction (Y axis) perpendicular to the front-rear direction (travel direction) is the normalized length depending on the travel distance d of the vehicle 1, as with the front camera 111. This can be realized using the length R.

  On the other hand, the left camera 113 and the right camera 114 differ from the front camera 111 and the rear camera 112 in that the optical axis direction (Z-axis direction) coincides with the vertical direction of the vehicle 1. Therefore, among the pitch angle, yaw angle, and roll angle, the rotation angle that depends on the travel distance d of the vehicle 1 is a pitch angle around the left-right direction (X axis) perpendicular to the up-down direction of the vehicle 1. Therefore, the calibration of the pitch angle can be realized by using the ratio between the length D of the feature point series and the travel distance d of the vehicle 1, as in the calibration of the yaw angle in the front camera 111.

  In the above embodiment, an example is shown in which the yaw angle is calibrated using the normalized length R obtained by dividing the length d of the feature point series by the travel distance D. However, it is not limited to this. For example, the yaw angle may be calibrated using a normalized speed obtained by dividing the length d of the feature point series by the traveling speed vk of the vehicle 1.

  In the above embodiment, an example in which the calibration system 100 is installed in the vehicle 1 whose front wheels are steering wheels has been described. However, it is not limited to this. For example, the calibration system 100 may be installed in a vehicle whose rear wheels are steering wheels.

  In the above embodiment, an example in which four cameras 111 to 114 are used has been described. However, it is not limited to this. The number of cameras can be appropriately changed according to a request from a user or the like.

  In the above embodiment, the vehicle speed information and the steering angle information are output to the speed acquisition unit 118 and the steering angle acquisition unit 119 via CAN as a communication protocol, respectively. However, it is not limited to this. For example, the vehicle speed information and the rudder angle information are respectively obtained through a speed acquisition unit 118 via a CAN FD (Controller Area Network With Flexible Data Rate) in which the physical layer of the OSI (Open Systems Interconnection) reference model is compatible with CAN. And the steering angle acquisition unit 119.

  As mentioned above, although the Example of this invention was explained in full detail with drawing, since an Example is only an illustration of this invention, this invention is not limited only to the structure of an Example. Of course, changes in design and the like within a range not departing from the gist are included in the present invention.

DESCRIPTION OF SYMBOLS 1 ... Vehicle 111, 112, 113, 114 ... Camera 116 ... Calibration apparatus 117 ... Feature point extraction part 118 ... Speed acquisition part 119 ... Steering angle acquisition part 120 ... Feature point series calculation unit 121... Feature point series accumulation unit 122... Camera parameter calibration unit (rotation angle calibration unit)
D, D1, D2 ... Euclidean distance (length of feature point series)
d, d1, d2 ... travel distances F, F1, F2, F300, F301, F600, F601 ... frames P1, P2 ... feature points R, R1, R2 ... normalized length vk ...・ Running speed

Claims (5)

A calibration device for carrying out calibration of a camera mounted on a vehicle and imaging the periphery of the vehicle,
A feature point extraction unit that extracts feature points of the captured image from a plurality of frames obtained in time series by the imaging;
A rudder angle obtaining unit for obtaining a rudder angle of the vehicle;
Based on the rudder angle acquired by the rudder angle acquisition unit, when the vehicle is determined to travel straight, the feature point is a straight line displaced between a pair of frames that are in tandem with each other. A feature point series calculation unit for calculating the length of the feature point series;
In the imaging plane, the ratio of the travel distance traveled by the vehicle between the set of frames and the length of the feature point series calculated by the feature point series calculation unit is the same between different sets of frames. A rotation angle calibration unit that calibrates the parameters of the rotation angle around one axis,
A calibration apparatus comprising: The calibration device according to claim 1,
The rotation angle calibration unit sets a parameter of a rotation angle about an axis extending in the vertical direction of the vehicle perpendicular to the front-rear direction of the vehicle when the optical axis direction of the camera coincides with the front-rear direction of the vehicle. A calibration device characterized by calibrating as a parameter of a rotation angle of rotation. The calibration device according to claim 1,
The rotation angle calibration unit sets a parameter of a rotation angle around an axis extending in the left-right direction of the vehicle perpendicular to the vertical direction of the vehicle when the optical axis direction of the camera coincides with the vertical direction of the vehicle. A calibration device characterized by calibrating as a parameter of a rotation angle of rotation. In the calibration apparatus according to any one of claims 1 to 3,
A speed acquisition unit for acquiring the traveling speed of the vehicle;
The rotation angle calibration unit calculates the travel distance based on the travel speed acquired by the speed acquisition unit. The calibration device according to any one of claims 1 to 4,
A feature point series accumulation unit for accumulating the number of feature point series calculated by the feature point series calculation unit;
The rotation angle calibration unit calibrates the parameter of the rotation angle when the number accumulated by the feature point series accumulation unit reaches a predetermined required number.