CN113450415B

CN113450415B - Imaging equipment calibration method and device

Info

Publication number: CN113450415B
Application number: CN202010222903.XA
Authority: CN
Inventors: 丁银章; 陈康平; 赖百胜; 黄建强; 华先胜
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2020-03-26
Filing date: 2020-03-26
Publication date: 2024-07-30
Anticipated expiration: 2040-03-26
Also published as: CN113450415A

Abstract

The application discloses a calibration method and a calibration device for imaging equipment, wherein the method comprises the following steps: acquiring two plane lines corresponding to two parallel space lines on a target image shot by imaging equipment to be calibrated; acquiring known distances between a plurality of spatial points on the spatial straight lines, and acquiring known distances between the two spatial straight lines; and calibrating the imaging equipment to be calibrated by taking the space straight line as a calibration reference object and combining the plane straight line and the known distance. According to the method, the parallel space straight lines are used as calibration reference objects, and the parallel space straight lines and space points on the parallel space straight lines are easy to obtain in a traffic scene, so that the problem that the existing camera calibration process for the traffic scene is limited due to the limitation of selecting the calibration reference objects can be avoided.

Description

Imaging equipment calibration method and device

Technical Field

The application relates to the field of imaging equipment calibration, in particular to an imaging equipment calibration method. The application also relates to an imaging device calibration device and electronic equipment.

Background

In applications such as computer vision, image measurement, and three-dimensional scene reconstruction, in order to correct lens distortion of a camera, determine a conversion relationship between a physical size in a three-dimensional space and a pixel size of an image, and determine a correlation between a three-dimensional geometric position of a space object (or a certain point on a surface of the space object) and coordinates of a corresponding pixel point in the image of the space object (or a certain point on a surface of the space object), a geometric model of camera imaging needs to be established, parameters of the geometric model are camera parameters, a process of solving the camera parameters is camera calibration or camera calibration, and accuracy of a calibration result and stability of an algorithm directly affect accuracy of camera work.

The camera calibration process mainly comprises the following steps: and obtaining the internal and external parameters of the camera model by a preset algorithm through establishing a corresponding relation between the points with known coordinates on the calibration reference object and the pixel points on the image. The process can be divided into camera calibration based on a three-dimensional target, camera calibration based on a two-dimensional plane target, camera calibration based on radial constraint and the like according to different calibration references.

The existing camera calibration method for traffic scenes mainly comprises the following two steps:

method 1: and (3) using a rectangle on a parallel lane line as a calibration reference object to calibrate the focal length and azimuth information of the camera.

Method 2: and searching a group of three parallel lines with known mutual distance and a straight line with known slope and intersecting the parallel lines on the road as known calibration reference objects to calibrate the focal length and azimuth information of the camera.

The above method has the following disadvantages:

In the method 1, a rectangle is selected as a calibration reference object in a road, and in most traffic scenes, the rectangle calibration reference object is not easy to obtain, and the selection of the calibration reference object has limitation, so that the camera calibration process has limitation;

In the method 2, three parallel lines and a straight line with a known slope and intersecting the parallel lines are selected as calibration reference objects, and the straight line intersecting the parallel lines is also difficult to obtain in a traffic scene, and the selection of the calibration reference objects is limited, so that the process of calibrating the camera is limited.

Disclosure of Invention

The embodiment of the application provides an imaging equipment calibration method, an imaging equipment calibration device and electronic equipment, which are used for solving the problem that in the existing camera calibration process for traffic scenes, the camera calibration process is limited due to the limitation of a selected calibration reference object.

The embodiment of the application provides an imaging equipment calibration method, which comprises the following steps:

acquiring two plane lines corresponding to two parallel space lines on a target image shot by imaging equipment to be calibrated;

Acquiring known distances between a plurality of spatial points on the spatial straight lines, and acquiring known distances between the two spatial straight lines;

And calibrating the imaging equipment to be calibrated by taking the space straight line as a calibration reference object and combining the plane straight line and the known distance.

Optionally, the acquiring two plane lines corresponding to the two parallel spatial lines on the target image captured by the imaging device to be calibrated includes: selecting a first plane straight line and a second plane straight line from a target image shot by imaging equipment to be calibrated, wherein the first plane straight line corresponds to a first space straight line in the real world, the second plane straight line corresponds to a second space straight line in the real world, and the first space straight line and the second space straight line are parallel to each other;

the obtaining a known distance between a plurality of spatial points on the spatial straight line and obtaining a known distance between the two spatial straight lines includes: taking the known distance between a plurality of preset space points on the first space straight line as a target space distance, and taking the distance between the first space straight line and the second space straight line as a parallel line distance;

The calibrating the imaging device to be calibrated by taking the space straight line as a calibration reference object and combining the plane straight line and the known distance comprises the following steps: and obtaining a mapping relation between a pixel coordinate system corresponding to the target image and a world coordinate system corresponding to the real world according to the target space distance, the parallel line distance, the second plane straight line and pixel coordinate data of a plurality of plane points corresponding to the plurality of space points on the first plane straight line.

Optionally, the predetermined plurality of spatial points on the first spatial line include: the first space point, the second space point and the third space point are sequentially distributed on the first space straight line;

The plurality of planar points on the first planar line corresponding to the plurality of spatial points include: a first planar point corresponding to the first spatial point, a second planar point corresponding to the second spatial point, a third planar point corresponding to the third spatial point;

Correspondingly, the target space distance comprises: a first spatial distance between the first spatial point and the second spatial point, and a second spatial distance between the first spatial point and the third spatial point.

Optionally, the obtaining a mapping relationship between a pixel coordinate system corresponding to the target image and a world coordinate system corresponding to the real world according to the target space distance, the parallel line pitch, the second plane straight line, and pixel coordinate data of a plurality of plane points corresponding to the plurality of space points on the first plane straight line includes:

Calculating to obtain a camera focal length, a first included angle between imaging equipment to be calibrated and an X _wOY_w plane of a world coordinate system, and a second included angle between the first space straight line or the second space straight line and an X _w axis of the X _wOY_w plane according to the first space distance, the second space distance, the parallel line spacing, the second plane straight line and pixel coordinate data of a plurality of plane points corresponding to the plurality of space points on the first plane straight line;

deriving an internal reference matrix from the pixel coordinate system to the world coordinate system according to the camera focal length;

And deriving an extrinsic matrix from the pixel coordinate system to the world coordinate system according to the first included angle and the second included angle.

Optionally, the calculating, according to the first spatial distance, the second spatial distance, the parallel line distance, the second plane straight line, and pixel coordinate data of a plurality of plane points corresponding to the plurality of spatial points on the first plane straight line, obtains a camera focal length, a first included angle between an imaging device to be calibrated and an X _wOY_w plane of a world coordinate system, and a second included angle between the first space straight line or the second space straight line and an X _w axis of the X _wOY_w plane, includes:

Performing straight line fitting based on pixel coordinates of a plurality of points on the second plane straight line to obtain the slope and intercept of the second plane straight line;

Determining the planes of the first space straight line and the second space straight line as an X _wOY_w plane of the world coordinate system, and obtaining intersection point pixel coordinate data corresponding to a space intersection point of one of the first space point, the second space point and the third space point and the second space straight line along the X _w axis direction of the X _wOY_w plane according to the slope and the intercept of the second plane straight line;

And solving by using a pinhole imaging model and a similar triangle principle by taking the first space distance, the second space distance, the parallel line distance, the intersection point pixel coordinate data and pixel coordinate data of a plurality of plane points corresponding to the plurality of space points on the first plane straight line as known data to obtain the camera focal length, the first included angle and the second included angle.

Optionally, the method further comprises:

Taking the known distance of the imaging equipment to be calibrated in the real world relative to the plane where the first space straight line and the second space straight line are located as the height of the imaging equipment to be calibrated;

according to the height of the imaging equipment to be calibrated, carrying out position transformation on the world coordinate system to obtain a building information model coordinate system;

obtaining space coordinate data of a plurality of preset space points on the first space straight line according to pixel coordinate data of a plurality of plane points corresponding to the plurality of space points on the first plane straight line and the mapping relation between the pixel coordinate system corresponding to the target image and the world coordinate system corresponding to the real world;

Obtaining model coordinate data of a plurality of space points corresponding to the building information model coordinate system according to space coordinate data of the plurality of space points preset on the first space straight line;

Obtaining model coordinate data of vertical projection points of the plurality of space points corresponding to the second space straight line;

obtaining pixel coordinate data of the vertical projection point corresponding to the target image according to a mapping relation between the pixel coordinate system corresponding to the target image and the world coordinate system corresponding to the real world;

and obtaining a mapping relation between a pixel coordinate system corresponding to the target image and the building information model coordinate system according to the model coordinate data of the plurality of space points corresponding to the building information model coordinate system, the model coordinate data of the vertical projection points corresponding to the building information model coordinate system, the pixel coordinate data of the vertical projection points corresponding to the target image and the pixel coordinate data of the plurality of plane points corresponding to the plurality of space points.

Optionally, the performing position transformation on the world coordinate system according to the height of the imaging device to be calibrated to obtain a building information model coordinate system includes:

and according to the height of the imaging equipment to be calibrated, transforming the origin of the world coordinate system to the position of the vertical intersection point of the imaging equipment to be calibrated and the X _wOY_w plane of the world coordinate system, and obtaining the target origin of the building information model coordinate system.

Optionally, the obtaining model coordinate data of the plurality of spatial points corresponding to the building information model coordinate system according to the spatial coordinate data of the plurality of spatial points preset on the first spatial line includes:

Acquiring relative position information between a plurality of preset space points on the first space straight line and the target origin;

And obtaining model coordinate data of the plurality of space points corresponding to the building information model coordinate system according to the space coordinate data of the plurality of space points preset on the first space straight line and the relative position information.

Optionally, the obtaining model coordinate data of the vertical projection points of the plurality of spatial points corresponding to the second spatial line includes: obtaining relative position information between the vertical projection point and the target origin according to the relative position information between the plurality of space points and the target origin; and obtaining model coordinate data of the vertical projection point according to the relative position information between the vertical projection point and the target origin.

Optionally, the method further comprises: and carrying out edge extraction on the image shot by the imaging equipment to be calibrated to obtain the target image.

Optionally, the first spatial straight line and the second spatial straight line are lane lines in a traffic scene.

The embodiment of the application also provides an imaging device calibration device, which comprises:

The linear acquisition unit is used for acquiring two plane lines corresponding to two parallel space lines on a target image shot by imaging equipment to be calibrated;

a distance data acquisition unit configured to acquire known distances between a plurality of spatial points on the spatial straight lines, and acquire known distances between the two spatial straight lines;

and the calibration unit is used for calibrating the imaging equipment to be calibrated by taking the space straight line as a calibration reference object and combining the plane straight line and the known distance.

The embodiment of the application also provides electronic equipment, which comprises a processor and a memory; wherein,

The memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to perform operations comprising:

Compared with the prior art, the embodiment of the application has the following advantages:

According to the imaging equipment calibration method provided by the embodiment of the application, two plane lines corresponding to two parallel space lines on a target image shot by imaging equipment to be calibrated are obtained; acquiring known distances between a plurality of spatial points on the spatial straight lines, and acquiring known distances between the two spatial straight lines; and calibrating the imaging equipment to be calibrated by taking the space straight line as a calibration reference object and combining the plane straight line and the known distance. According to the method, the parallel space straight lines are used as calibration reference objects, and the parallel space straight lines and space points on the parallel space straight lines are easy to obtain in a traffic scene, so that the problem that the existing camera calibration process for the traffic scene is limited due to the limitation of the selected calibration reference objects can be avoided.

Drawings

FIG. 1 is a flowchart of an imaging device calibration method according to a first embodiment of the present application;

FIG. 1-A is a diagram showing the relationship between a pixel coordinate system and a world coordinate system according to a first embodiment of the present application;

FIG. 1-B is a diagram showing the transformation of the world coordinate system and BIM coordinate system provided by the first embodiment of the present application;

FIG. 1-C is a schematic diagram showing the relative positions of a calibration point and an imaging device to be calibrated in a BIM coordinate system according to a first embodiment of the present application;

FIG. 2 is a block diagram of a unit of an imaging device calibration apparatus provided in a second embodiment of the present application;

fig. 3 is a schematic logic structure of an electronic device according to a third embodiment of the present application.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. The present application may be embodied in many other forms than those herein described, and those skilled in the art will readily appreciate that the present application may be similarly embodied without departing from the spirit or essential characteristics thereof, and therefore the present application is not limited to the specific embodiments disclosed below.

Aiming at an imaging equipment calibration scene, the application provides an imaging equipment calibration method, an imaging equipment calibration device corresponding to the method and electronic equipment in order to reduce the complexity of a calibration process. The following provides examples to describe the method, apparatus and electronic device in detail.

A first embodiment of the present application provides an imaging device calibration method, where an application body of the method may be applied to a computing device for calibrating a camera/video camera, fig. 1 is a flowchart of the imaging device calibration method provided in the first embodiment of the present application, and the method provided in the present embodiment is described in detail below with reference to fig. 1. The embodiments referred to in the following description are intended to illustrate the method principles and not to limit the practical use.

As shown in fig. 1, the calibration method of the imaging device provided in this embodiment includes the following steps:

s101, obtaining two plane straight lines corresponding to two parallel space straight lines on a target image shot by imaging equipment to be calibrated.

In applications such as computer vision, image measurement, and three-dimensional scene reconstruction, in order to correct lens distortion of a camera, determine a conversion relationship between a physical size in a three-dimensional space and a pixel size of an image, and determine a correlation between a three-dimensional geometric position of a space object (or a certain point on a surface of the space object) and coordinates of a corresponding pixel point in the image of the space object (or a certain point on a surface of the space object), a geometric model of camera imaging needs to be established, parameters of the geometric model are camera parameters, and a process of solving the parameters is camera calibration or camera calibration.

In this embodiment, two plane lines corresponding to two parallel spatial lines on a target image captured by an imaging device to be calibrated are obtained, which specifically means: selecting a first plane straight line and a second plane straight line from target images shot by imaging equipment to be calibrated, wherein the first plane straight line corresponds to a first space straight line in the real world, the second plane straight line corresponds to a second space straight line in the real world, and the first space straight line and the second space straight line are parallel to each other. The first space straight line and the second space straight line are calibration reference objects preset in real space, the calibration reference objects refer to movable objects which are preset and are in the shooting range of the imaging equipment to be calibrated, the movable objects are used for obtaining matched space coordinate information and image coordinate information in the calibration process of the imaging equipment, for example, the imaging equipment to be calibrated is a camera, the camera is arranged at an airport, and then the calibration reference objects can be airplanes with determined flight information in the take-off or landing process; the camera is arranged on a highway, and the calibration reference can be a running vehicle; the camera is placed in a mall and the calibration reference may be a mobile robot fitted with a positioning sensor or a person wearing a positioning sensor. In this embodiment, the first spatial line and the second spatial line may be parallel lane lines on a highway, a line where a street lamp is located, or reflective strips at the edge of the road, and so on. In this embodiment, the first spatial line and the second spatial line are preferably lane lines in the traffic scene.

It should be noted that, before the first plane line and the second plane line are obtained, edge extraction is further performed on the image captured by the imaging device to be calibrated to obtain the target image. For example, a bilateral filter is adopted to reduce noise of an original image shot by a camera, and edge detection methods such as a Canny operator, a Sobel operator, a Prewitt operator and the like are applied to perform edge detection on the original image subjected to noise reduction treatment so as to extract the edge position of the original image, and the target image is obtained.

S102, acquiring known distances among a plurality of space points on the space straight lines, and acquiring known distances between the two space straight lines.

In this embodiment, the above-mentioned obtaining the known distances between the plurality of spatial points on the spatial straight line and the obtaining the known distances between the two spatial straight lines may specifically refer to: the known distance between a plurality of preset space points on the first space straight line is taken as a target space distance, and the distance between the first space straight line and the second space straight line is taken as a parallel line distance. The preset plurality of spatial points on the first spatial line may be a first spatial point, a second spatial point, and a third spatial point sequentially distributed on the first spatial line, such as a first spatial point a ', a second spatial point B ', and a third spatial point C ' shown in fig. 1-a, and in the traffic scene, the first spatial point, the second spatial point, and the third spatial point may be points preset on the lane line and the road-side reflection band. The known distances between the plurality of spatial points preset on the first spatial line may be a first spatial distance (distance d1 of a 'and B') between the first spatial point and the second spatial point, and a second spatial distance (distance d2 of a 'and C') between the first spatial point and the third spatial point. In this embodiment, the first spatial distance and the second spatial distance are known data, that is, the above spatial points are spatial points whose pitch is known. The parallel line distance dw may be a vertical distance between two parallel lane lines acquired in advance.

And S103, calibrating the imaging equipment to be calibrated by taking the space straight line as a calibration reference object and combining the plane straight line and the known distance.

In this embodiment, the calibrating the imaging device to be calibrated by using the spatial line as a calibration reference and combining the plane line and the known distance may specifically refer to: and obtaining a mapping relation between a pixel coordinate system corresponding to the target image and a world coordinate system corresponding to the real world according to the target space distance, the parallel line distance, the second plane straight line and pixel coordinate data of a plurality of plane points corresponding to the plurality of space points on the first plane straight line.

The plurality of planar points corresponding to the plurality of spatial points on the first planar line may refer to: a first planar point corresponding to the first spatial point, a second planar point corresponding to the second spatial point, and a third planar point corresponding to the third spatial point. As shown in fig. I-a, the first plane point is a, the second plane point is B, the third plane point is C, and in the pixel coordinate system, the pixel coordinate data of the first plane point, the pixel coordinate data of the second plane point, and the pixel coordinate data of the third plane point are all known data.

In the present embodiment, the mapping relationship between the pixel coordinate system corresponding to the target image and the world coordinate system corresponding to the real world can be obtained by:

First, according to the first space distance, the second space distance, the parallel line distance, the second plane straight line and pixel coordinate data of a plurality of plane points corresponding to the plurality of space points on the first plane straight line, a first included angle between imaging equipment to be calibrated and an X _wOY_w plane of a world coordinate system and a second included angle between the first space straight line or the second space straight line and an X _w axis of the X _wOY_w plane are calculated and obtained. The process specifically comprises the following steps:

And performing straight line fitting based on pixel coordinates of a plurality of points on the second plane straight line to obtain the slope and intercept of the second plane straight line. For example, according to the pixel coordinates of 10 or more points on the second plane line, the linear equation y=kx+b corresponding to the second plane line, and the slope and intercept of the second plane line corresponding to k and b are fitted by using the least square method.

And determining planes of the first space straight line and the second space straight line as XwOYw planes of the world coordinate system, and obtaining pixel coordinate data of a space intersection point of one of the first space point, the second space point and the third space point and the second space straight line along the Xw axis direction of the XwOYw plane in a pixel coordinate system corresponding to the target image according to the slope and the intercept of the second plane straight line and the pixel coordinates of a plurality of points on the second plane straight line. As shown in fig. 1-a, A, B, C is a plurality of planar points on the second planar line, the spatial intersection point D 'of a' in the Xw axis direction corresponds to the planar intersection point of the target image, the pixel coordinates of the above planar points are a (h 1, v 1), B (h 2, v 2), C (h 3, v 3), D (h 4, v 4), the image center coordinates are (h 0, v 0), and v4=v1, h4= (v 1-B)/k because the point a 'and the point D' have the same horizontal coordinates.

And solving by using the first space distance D1, the second space distance D2, the parallel line distance dw, pixel coordinate data D ((v 1-B)/k, v 1) of the space intersection point in a pixel coordinate system corresponding to the target image and pixel coordinate data A (h 1, v 1), B (h 2, v 2) and C (h 3, v 3) of a plurality of plane points corresponding to the plurality of space points on the first plane straight line as known data and using a pinhole imaging model and a similar triangle principle to obtain the camera focal length f, a first included angle theta and a second included angle alpha. As shown in fig. 1-a, based on the above spatial points, the above planar points, the above spatial intersection, the above planar intersection D ', the focal point corresponding to the imaging device to be calibrated (point P ' in fig. 1-a), and the positional relationship between the optical center of the imaging device to be calibrated (point L in fig. 1-a), a pinhole imaging model and a similar triangle principle are used to obtain a similar proportional relationship and a trigonometric function relationship of a similar triangle, and the above D1, D2, dw, D ((v 1-B)/k, v 1) and a (h 1, v 1), B (h 2, v 2), C (h 3, v 3), and the like are known to be substituted into the similar proportional relationship and the trigonometric function relationship to solve for the camera focal length f (P ' L), the first included angle θ, and the second included angle α. For example, the similar proportional relationship is: p ' L/pl=cl/C ' l=bl/B ' l=al/a ' l=dl/D ' L, the trigonometric relation being: dw=a 'D' ×sin α, substituting the known data can obtain the following derivation formula:

In the above formula, t1, λ, h12, h13, d13 are intermediate variables obtained by the simultaneous similar proportional relation, and the focal length f, the first included angle θ, and the second included angle α of the camera can be obtained by calculation through the above derivation formula.

And secondly, deriving an internal reference matrix from the pixel coordinate system to the world coordinate system according to the focal length of the camera, and deriving an external reference matrix from the pixel coordinate system to the world coordinate system according to the first included angle and the second included angle.

For example, the mapping matrix P may be expressed as:

Taking the center point coordinate of the target image as the principal point coordinate, the internal reference matrix K may be:

the extrinsic matrix may be:

The external parameter matrix comprises a rotation component R and a translation component T, the preset camera rolling angle is 0, and the first included angle theta and the second included angle alpha obtained through calculation are combined to indicate that the angles of the three directions of the camera are determined, so that R can be obtained. The translation component may be determined based on the position of the camera in the BIM coordinate system, assuming that the coordinate values of the camera in the BIM coordinate system are (x, y, z), then t= [ x, y, z ].

In the present embodiment, after the above-described process of obtaining the mapping relationship between the pixel coordinate system corresponding to the target image and the world coordinate system corresponding to the real world, the mapping relationship between the pixel coordinate system and the Building Information Model (BIM) coordinate system may also be obtained, the process specifically includes the following:

A. Taking the known distance of the imaging equipment to be calibrated in the real world relative to the plane where the first space straight line and the second space straight line are located as the height H of the imaging equipment to be calibrated; in this embodiment, the plane where the first spatial line and the second spatial line are located may be a highway plane corresponding to a lane line, and the height of the imaging device to be calibrated may refer to the height of the camera relative to the highway plane.

B. and carrying out position transformation on the world coordinate system according to the height H of the imaging equipment to be calibrated to obtain a Building Information Model (BIM) coordinate system. In this embodiment, as shown in fig. 1-B, the transformation process transforms the origin of the world coordinate system to the position of the vertical intersection point of the imaging device to be calibrated and the X _wOY_w plane of the world coordinate system to obtain the target origin of the building information model coordinate system, and the transformation process needs to calculate the transformation translation vector of the world coordinate system and the building information model coordinate system according to the height H of the imaging device to be calibrated, and since the translation is only performed on the Y axis (Yw, yb), the translation distance is OO ", and OO" =h×tan (θ) can be calculated by the known height H of the imaging device to be calibrated.

C. According to the pixel coordinate data a (h 1, v 1), B (h 2, v 2), C (h 3, v 3) of the plurality of plane points corresponding to the plurality of space points on the first plane line and the mapping relationship between the pixel coordinate system and the world coordinate system obtained in the step S103, the space coordinate data of the plurality of space points a ', B ', C ' preset on the first plane line are obtained.

D. and obtaining model coordinate data of the plurality of space points corresponding to the building information model coordinate system according to the space coordinate data of the plurality of space points A ', B ', C ' preset on the first space straight line. As shown in fig. 1-C, the three-dimensional height of the imaging device to be calibrated in the BIM coordinate system can be measured by the BIM, the relative position information between the spatial points a ', B ', C ' and the target origin o″ of the building information model coordinate system is determined based on the relation between the three-dimensional height and the height H of the imaging device to be calibrated, and the model coordinate data of the spatial points a ', B ', C ' corresponding to the points a ", B", c″ of the building information model coordinate system is obtained according to the spatial coordinate data of the spatial points a ', B ', C ' and the relative position information. The three-dimensional height of the imaging device to be calibrated in the BIM coordinate system is measured through the BIM, and the relative position of the calibration reference object in the BIM coordinate system can be deduced by combining the real height H of the imaging device to be calibrated, so that a reliable basis is provided for labeling of the calibration reference object in the BIM coordinate system.

E. Model coordinate data of vertical projection points D ', E', F 'of the space points A', B ', C' corresponding to the second space straight line are obtained, and as shown in fig. 1-C, relative position information between the vertical projection points D ', E', F 'and a target origin O' is obtained according to relative position information between the space points A ', B', C 'and the target origin O' of the building information model coordinate system; and obtaining model coordinate data of points D ', E', F 'of the perpendicular projection points D', E ', F' corresponding to the building information model coordinate system according to the relative position information between the perpendicular projection points D ', E', F 'and the target origin O'.

F. And determining pixel coordinate data of the vertical projection points D ', E ', F ' corresponding to the target image according to the mapping relation between the pixel coordinate system and the world coordinate system obtained in the step S103.

G. Points A ', B ', C ', corresponding to the building information model coordinate system according to the spatial points A ', B ', C ': model coordinate data of a point C ', points D ', E ', F ' of the vertical projection points D ', which correspond to the building information model coordinate system, are obtained E ', F ', pixel coordinate data of the vertical projection points D ', E ', F ' corresponding to the target image, pixel coordinate data of the plurality of plane points A, B, C corresponding to the plurality of spatial points a ', B ', C ', and obtaining a mapping relation between a pixel coordinate system corresponding to the target image and the building information model coordinate system. In this embodiment, the corresponding homography matrix may be calculated based on the coplanar assumption, so as to obtain a mapping relationship between the pixel coordinate system and the BIM coordinate system, where the homography matrix expresses a linear transformation relationship between planes, so that in a road traffic scene, it may express a transformation relationship between a road plane and a corresponding road plane in an image. The formula is expressed as x=h 'X, where X is a pixel coordinate point in the target image, X is a coordinate point of a BIM coordinate system, H' is a matrix of 3*3, which contains 4 degrees of freedom, so that 4 equations are constructed for at least 4 pairs of points to solve the unknown number of H ', and since there is an error in taking the points, the embodiment selects the 6 pairs of points, and uses the least square method to calculate H', so as to improve the accuracy of calculation. It should be noted that, in this embodiment, the projection matrix may be used instead of the homography matrix to calculate the mapping relationship, where the projection matrix has a direct physical meaning compared with the homography matrix, that is, under the pinhole camera model, the intersection point of the light from the camera to the road plane is calculated. The form of the transformation calculated by the homography matrix is x=hx, the form of the transformation calculated by the projection matrix is x=p 'X, and P' is the projection matrix. When solving the homography matrix, the homography matrix is obtained by using 6 point pairs to solve equations, and the projection matrix can be obtained by two transformation relations: the conversion relation from the BIM coordinate system to the world coordinate system is calculated (the space coordinates of the space points A ', B', C ', D' and the BIM three-dimensional coordinate corresponding to the space points are used for solving and calculating), the projection conversion from the BIM coordinate system to the pixel coordinate system is obtained by combining the obtained mapping relation between the pixel coordinate system and the world coordinate system, and the real-world calibration reference object can be projected into the BIM through the projection conversion, so that the space state of the calibration reference object can be intuitively displayed in a computer, and the relation can be further excavated and analyzed.

Three-dimensional information can be generated from two-dimensional images through the mapping relation between the pixel coordinate system and a Building Information Model (BIM) coordinate system, for example, in face 3D recognition technology application, a face three-dimensional model can be directly generated from face images shot by a camera based on the mapping relation.

According to the imaging equipment calibration method provided by the embodiment, two parallel lines with known distances in real space and a space point with known distance on one parallel line are used as calibration reference objects, the calibration reference objects are easy to obtain in a traffic monitoring scene, and the obtained distance scale information is more accurate, so that the mapping relation between a pixel coordinate system and a world coordinate system is accurately and efficiently constructed. By using the method, the problem that the existing camera calibration process for traffic scenes has limitation due to the limitation of the selected calibration reference object can be avoided.

And the positioning method of the calibration reference object in the BIM is characterized in that the height information of the imaging equipment to be calibrated in the BIM coordinate system is measured through the BIM, and the relative position of the calibration reference object in the BIM coordinate system can be deduced by combining the real height of the imaging equipment to be calibrated, so that a reliable basis is provided for labeling of the calibration reference object in the BIM coordinate system, accurate coordinate data of the calibration reference object in the BIM coordinate system is obtained, the mapping relation of a monitoring image to the BIM is constructed, and the information perceived by a camera can be fused into a three-dimensional model to form a global dynamic monitoring system.

The first embodiment provides a calibration method for an imaging device, and the second embodiment of the present application also provides a calibration device for an imaging device, which is substantially similar to the method embodiment, so that the description is relatively simple, and details of relevant technical features should be referred to the corresponding description of the method embodiment provided above, and the following description of the device embodiment is merely illustrative.

Referring to fig. 2 for understanding the embodiment, fig. 2 is a block diagram of a unit of an apparatus provided in the embodiment, and as shown in fig. 2, the apparatus provided in the embodiment includes:

A straight line acquiring unit 201, configured to acquire two plane straight lines corresponding to two parallel spatial straight lines on a target image captured by an imaging device to be calibrated;

a distance data acquisition unit 202 for acquiring known distances between a plurality of spatial points on the spatial straight lines, and acquiring known distances between the two spatial straight lines;

And the calibration unit 203 is configured to calibrate the imaging device to be calibrated by using the spatial straight line as a calibration reference object and combining the planar straight line and the known distance.

Determining the planes of the first space straight line and the second space straight line as an X _wOY_w plane of the world coordinate system, and obtaining pixel coordinate data of a space intersection point of one space point of the first space point, the second space point and the third space point and the second space straight line along the X _w axis direction of the X _wOY_w plane in a pixel coordinate system corresponding to the target image according to the slope and the intercept of the second plane straight line and the pixel coordinates of a plurality of points on the second plane straight line;

And solving by using a pinhole imaging model and a similar triangle principle by taking the first space distance, the second space distance, the parallel line distance, pixel coordinate data of the space intersection point in a pixel coordinate system corresponding to the target image and pixel coordinate data of a plurality of plane points corresponding to the plurality of space points on the first plane straight line as known data to obtain the camera focal length, the first included angle and the second included angle.

Optionally, the method further comprises:

Optionally, the obtaining model coordinate data of the vertical projection points of the plurality of spatial points corresponding to the second spatial line includes:

Obtaining relative position information between the vertical projection point and the target origin according to the relative position information between the plurality of space points and the target origin;

And obtaining model coordinate data of the vertical projection point according to the relative position information between the vertical projection point and the target origin.

According to the imaging equipment calibration device provided by the embodiment, two parallel lines with known distances in a real space and a space point with known distances on one parallel line are used as calibration reference objects, the calibration reference objects are easy to obtain in a traffic monitoring scene, and the obtained distance scale information is more accurate, so that the mapping relation between a pixel coordinate system and a world coordinate system is accurately and efficiently constructed. By using the device, the problem that the existing camera calibration process for traffic scenes has limitation due to the limitation of the selected calibration reference object can be avoided.

In the foregoing embodiments, a method for calibrating an imaging device and an apparatus for calibrating an imaging device are provided, and in addition, a third embodiment of the present application further provides an electronic device, which is described relatively simply because the electronic device embodiment is substantially similar to the method embodiment, and details of related technical features should be referred to the corresponding descriptions of the method embodiment provided above, and the following description of the electronic device embodiment is merely illustrative. The electronic device embodiment is as follows:

fig. 3 is a schematic diagram of an electronic device according to the present embodiment.

As shown in fig. 3, the electronic device includes: a processor 301; a memory 302;

the memory 302 is configured to store an imaging device calibration program, which when read and executed by the processor performs the following operations:

Optionally, the method further comprises:

According to the electronic equipment provided by the embodiment, two parallel lines with known distances in a real space and a space point with known distances on one parallel line are used as calibration reference objects, the calibration reference objects are easy to obtain in a traffic monitoring scene, and the obtained distance scale information is more accurate, so that the mapping relation between a pixel coordinate system and a world coordinate system is accurately and efficiently constructed. By using the electronic equipment, the problem that the existing camera calibration process for traffic scenes has limitation due to the limitation of the selected calibration reference object can be avoided.

In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

1. Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer readable media, as defined herein, does not include non-transitory computer readable media (transmission media), such as modulated data signals and carrier waves.

2. It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

While the application has been described in terms of preferred embodiments, it is not intended to be limiting, but rather, it will be apparent to those skilled in the art that various changes and modifications can be made herein without departing from the spirit and scope of the application as defined by the appended claims.

Claims

1. An imaging device calibration method, comprising:

Obtaining two plane lines corresponding to two parallel space lines on a target image captured by imaging equipment to be calibrated comprises the following steps: selecting a first plane straight line and a second plane straight line from the target image shot by the imaging equipment to be calibrated, wherein the first plane straight line corresponds to a first space straight line in the real world, the second plane straight line corresponds to a second space straight line in the real world, and the first space straight line and the second space straight line are parallel to each other;

the space straight line is used as a calibration reference object, and the imaging equipment to be calibrated is calibrated by combining the plane straight line and the known distance;

According to the height of the imaging equipment to be calibrated, carrying out position transformation on a world coordinate system to obtain a building information model coordinate system;

2. The method for calibrating an imaging apparatus according to claim 1, wherein,

The obtaining a known distance between a plurality of spatial points on the spatial straight line and obtaining a known distance between the two spatial straight lines includes:

Taking the known distance between a plurality of preset space points on the first space straight line as a target space distance, and taking the distance between the first space straight line and the second space straight line as a parallel line distance;

the calibrating the imaging device to be calibrated by taking the space straight line as a calibration reference object and combining the plane straight line and the known distance comprises the following steps:

and obtaining a mapping relation between a pixel coordinate system corresponding to the target image and a world coordinate system corresponding to the real world according to the target space distance, the parallel line distance, the second plane straight line and pixel coordinate data of a plurality of plane points corresponding to the plurality of space points on the first plane straight line.

3. The imaging apparatus calibration method according to claim 2, wherein the predetermined plurality of spatial points on the first spatial line include: the first space point, the second space point and the third space point are sequentially distributed on the first space straight line;

4. The imaging apparatus calibration method according to claim 3, wherein the obtaining of the mapping relationship between the pixel coordinate system corresponding to the target image and the world coordinate system corresponding to the real world from the pixel coordinate data of the target spatial distance, the parallel line pitch, the second planar straight line, and the plurality of planar points corresponding to the plurality of spatial points on the first planar straight line includes:

Calculating to obtain a camera focal length, a first included angle between the imaging device to be calibrated and an X _wOY_w plane of a world coordinate system, and a second included angle between the first space straight line or the second space straight line and an X _w axis of the X _wOY_w plane according to the first space distance, the second space distance, the parallel line spacing, the second plane straight line and pixel coordinate data of a plurality of plane points corresponding to the plurality of space points on the first plane straight line;

5. The method according to claim 4, wherein calculating to obtain the camera focal length, a first angle between the imaging device to be calibrated and an X _wOY_w plane of a world coordinate system, and a second angle between the first spatial straight line or the second spatial straight line and an X _w axis of the X _wOY_w plane based on the first spatial distance, the second spatial distance, the parallel line pitch, the second planar straight line, and pixel coordinate data of a plurality of planar points corresponding to the plurality of spatial points on the first planar straight line includes:

6. The method for calibrating an imaging device according to claim 1, wherein the performing a position transformation on the world coordinate system according to the height of the imaging device to be calibrated to obtain a building information model coordinate system comprises:

7. The method according to claim 6, wherein the obtaining model coordinate data of the plurality of spatial points corresponding to the building information model coordinate system based on spatial coordinate data of the plurality of spatial points preset on the first spatial line includes:

8. The method of calibrating an imaging apparatus according to claim 7, wherein the obtaining model coordinate data of the plurality of spatial points corresponding to the perpendicular projection points of the second spatial line includes:

9. The imaging apparatus calibration method according to claim 1, further comprising:

and carrying out edge extraction on the image shot by the imaging equipment to be calibrated to obtain the target image.

10. The imaging apparatus calibration method according to claim 1, wherein the spatial straight line is a lane line in a traffic scene.

11. An imaging device calibration apparatus, comprising:

The linear acquisition unit is used for acquiring two plane lines corresponding to two parallel space lines on a target image shot by imaging equipment to be calibrated, and comprises the following components: selecting a first plane straight line and a second plane straight line from the target image shot by the imaging equipment to be calibrated, wherein the first plane straight line corresponds to a first space straight line in the real world, the second plane straight line corresponds to a second space straight line in the real world, and the first space straight line and the second space straight line are parallel to each other;

the calibration unit is used for calibrating the imaging equipment to be calibrated by taking the space straight line as a calibration reference object and combining the plane straight line and the known distance;

12. An electronic device comprising a processor and a memory; wherein,

The memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method of any of claims 1-10.