CN110378967B - Virtual target calibration method combining grating projection and stereoscopic vision - Google Patents
Virtual target calibration method combining grating projection and stereoscopic vision Download PDFInfo
- Publication number
- CN110378967B CN110378967B CN201910537131.6A CN201910537131A CN110378967B CN 110378967 B CN110378967 B CN 110378967B CN 201910537131 A CN201910537131 A CN 201910537131A CN 110378967 B CN110378967 B CN 110378967B
- Authority
- CN
- China
- Prior art keywords
- grating pattern
- projector
- coordinates
- grating
- binocular camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention provides a virtual target calibration method combining grating projection and stereoscopic vision, which comprises the following steps: s1, calibrating a binocular camera based on a checkerboard to obtain parameters of the binocular camera; s2, taking the checkerboard as a background, projecting a grating to the checkerboard through a projector, and obtaining a grating pattern on the checkerboard through a binocular camera; s3, acquiring angular point pixel coordinates of the grating pattern and discrete coordinates of the grating pattern, fitting a space plane equation of the binocular camera according to the angular point pixel coordinates of the grating pattern and parameters of the binocular camera, and fitting a space plane equation of the projector according to the discrete coordinates of the grating pattern; s4, obtaining world coordinates corresponding to the discrete coordinates of the grating pattern according to the space plane equation of the binocular camera and the space plane equation of the projector; s5, repeating the steps S2 to S4 to obtain corner pixel coordinate-world coordinate data sets with more than 10 groups, and obtaining projector parameters according to the data sets. The invention can reduce the calculation difficulty and improve the calibration precision.
Description
Technical Field
The invention relates to the technical field of vision calibration, in particular to a virtual target calibration method combining grating projection and stereoscopic vision.
Background
The binocular is one of three-dimensional measurement methods for hot research at present, and is a mode for acquiring depth information only by means of parallax of two cameras. However, the three-dimensional measurement method has a larger limitation in practical use, firstly, a camera is very dependent on image feature matching, on one hand, feature extraction and matching are difficult to carry out under the condition of darker illumination or overexposure, on the other hand, feature extraction and matching are difficult to carry out if the measured scene lacks textures, and secondly, the effect of binocular vision reconstruction is not good for the scene with fewer texture features. Thus, there is a need to solve the above problems with a solution of structured light and binocular combination and to add features to the reconstructed object by projecting structured light.
At present, most schemes adopt a grating projection stereoscopic vision system, but the calibration precision of the grating projection stereoscopic vision system finally influences the precision of a measurement result, so how to ensure the calibration precision of the system becomes a working key point and a working difficulty. The traditional calibration method is to consider a projector as a reverse camera for calibration, obtain projection pattern feature points by obtaining projection patterns, obtain world coordinates according to calibrated parameters of a binocular camera, and finally obtain accurate translation and rotation matrixes of the projector relative to the internal reference and the world coordinates and the relative position relation of the projector and the camera. The method is complex in calculation, high in calculation difficulty and large in calibration error.
Disclosure of Invention
The present invention aims to solve at least to some extent one of the technical problems in the above-described technology. Therefore, the invention aims to provide a virtual target calibration method combining grating projection and stereoscopic vision, which can reduce the calculated amount, reduce the calculation difficulty and improve the calibration precision.
In order to achieve the above objective, an embodiment of the present invention provides a virtual target calibration method combining raster projection and stereoscopic vision, including the following steps: s1, calibrating a binocular camera based on a checkerboard to obtain parameters of the binocular camera; s2, taking the checkerboard as a background, projecting a grating to the checkerboard through a projector, and acquiring a grating pattern on the checkerboard through the binocular camera; s3, acquiring angular point pixel coordinates of the grating pattern and discrete coordinates of the grating pattern, fitting a space plane equation of the binocular camera according to the angular point pixel coordinates of the grating pattern and parameters of the binocular camera, and fitting a space plane equation of the projector according to the discrete coordinates of the grating pattern; s4, obtaining world coordinates corresponding to the discrete coordinates of the grating pattern according to the space plane equation of the binocular camera and the space plane equation of the projector; s5, repeating the steps S2 to S4 to obtain corner pixel coordinate-world coordinate data sets with more than 10 groups, and obtaining parameters of the projector according to the data sets.
According to the virtual target calibration method combining grating projection and stereoscopic vision, parameters of the virtual target calibration method are firstly obtained through a calibration camera, then a grating pattern is projected to a checkerboard through a projector, the grating pattern and angular point pixel coordinates and discrete coordinates of the grating pattern are obtained, a space plane equation of a binocular camera is further fitted according to the parameters of the binocular camera and the angular point pixel coordinates of the grating pattern, a space plane equation of the projector is fitted according to the discrete coordinates of the grating pattern, finally world coordinates corresponding to the discrete coordinates of the grating pattern are calculated according to the space plane equation of the binocular camera and the space plane equation of the projector, a angular point pixel coordinate-world coordinate data set is obtained, more than 10 groups of angular point pixel coordinate-world coordinate data sets are obtained, and constraint targets are set to obtain parameters of the projector.
In addition, the virtual target calibration method combining grating projection and stereoscopic vision according to the embodiment of the invention may further have the following additional technical features:
according to one embodiment of the invention, the grating projected by the projector is a grating of a specific period.
Further, the step S3 includes the steps of: discretizing the grating pattern and the checkerboard pattern, and separating the grating pattern according to the discretization result to obtain the discrete coordinates of the grating pattern; according to the corner pixel coordinates of the grating pattern and the parameters of the binocular camera, world coordinates corresponding to the corner pixel coordinates of the grating pattern are obtained, and the world coordinates corresponding to the corner pixel coordinates of the grating pattern are fitted to a space plane equation of the binocular camera through a least square method; and fitting a space plane equation of the projector according to the discrete coordinates of the grating pattern.
Further, the fitted spatial plane equation of the binocular camera is:
A 1 x a +B 1 y a +C 1 z a +D 1 ＝0
A 2 x b +B 2 y b +C 2 z b +D 2 ＝0，
the fitted spatial plane equation of the projector is:
conversion to world coordinate system: />
Wherein, (X s ，Y s ，Z s ) Is the coordinate of the grating pattern surface, alpha is the included angle between the grating pattern surface and a plane in the projector coordinate system, R sw And T sw Rotation matrix and translation matrix from projector coordinate system to world coordinate system, respectively, (x) w ，y w ，z w ) Is the coordinates of the world coordinate system.
Further, the space plane equation of the binocular camera and the space plane equation of the projector are intersected, and the intersected plane equations are solved simultaneously to obtain world coordinates corresponding to the discrete coordinates of the grating pattern.
Further, the step S5 includes: setting constraint targets, wherein the constraint targets are the minimum world coordinate errors of each group of corner pixel coordinates-world coordinate data sets; and obtaining a conversion matrix of the projector internal parameters and the world coordinate system through a maximum likelihood method according to the constraint target, and carrying out SVD (Singular Value Decomposition ) decomposition to obtain the projector parameters.
Drawings
FIG. 1 is a flow chart of a virtual target calibration method combining raster projection and stereoscopic vision according to an embodiment of the present invention;
FIG. 2 is a grating pattern according to one embodiment of the present invention;
FIG. 3 is a mathematical model diagram of a raster projection stereovision in accordance with one embodiment of the present invention;
fig. 4 is a schematic diagram of the overall principle of an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Fig. 1 is a flowchart of a virtual target calibration method combining raster projection and stereoscopic vision according to an embodiment of the present invention.
As shown in fig. 1, the virtual target calibration method combining raster projection and stereoscopic vision according to the embodiment of the invention includes the following steps:
s1, calibrating the binocular camera based on the checkerboard to obtain parameters of the binocular camera.
In one embodiment of the invention, a Zhang Zhengyou calibration method is adopted to calibrate the binocular camera, the single camera is calibrated respectively, and then the two cameras are calibrated jointly to obtain the parameters of the binocular camera.
S2, taking the checkerboard as a background, projecting the grating to the checkerboard through a projector, and acquiring the grating pattern on the checkerboard through a binocular camera.
In one embodiment of the present invention, as shown in fig. 2, the grating projected by the projector is a grating of a specific period. The grating pattern can be directly obtained by projecting the grating with a specific period, and the world coordinates are obtained by avoiding phase analysis, so that the calculated amount is reduced, and the calculation process is simplified.
S3, acquiring angular point pixel coordinates of the grating pattern and discrete coordinates of the grating pattern, fitting a space plane equation of the binocular camera according to the angular point pixel coordinates of the grating pattern and parameters of the binocular camera, and fitting a space plane equation of the projector according to the discrete coordinates of the grating pattern.
In one embodiment of the invention, according to the corner pixel coordinates of the grating pattern and the parameters of the binocular camera, the world coordinates corresponding to the corner pixel coordinates of the grating pattern can be obtained by combining the conversion relation between the corner pixel coordinates and the world coordinate system.
Further, by the least square method, the world coordinates corresponding to the pixel coordinates of the corner points of the grating pattern can be fitted to a spatial plane equation of the binocular camera, and specifically, the fitted spatial plane equation of the binocular camera is:
A 1 x a +B 1 y a +C 1 z a +D 1 ＝0
A 2 x b +B 2 y b +C 2 z b +D 2 ＝0，
wherein A is 1 、B 1 、C 1 、D 1 And A is a 2 、B 2 、C 2 、D 2 The value of (2) is determined by the relationship between the angular point pixel coordinate system established according to the perspective transformation principle of the binocular camera and the binocular camera coordinate system.
In one embodiment of the present invention, the discretized coordinate information of the grating pattern is obtained by discretizing the grating pattern and the checkerboard pattern and separating the grating pattern according to the discretization result.
Further, a spatial plane equation of the projector may be fitted according to the discrete coordinates of the grating pattern, and specifically, the fitted spatial plane equation of the projector is:
conversion to world coordinate system: />
Wherein, (X s ，Y s ，Z s ) Is the coordinate of the grating pattern surface, alpha is the included angle between the grating pattern surface and a plane in the projector coordinate system, R sw And T sw Rotation matrix and translation matrix from projector coordinate system to world coordinate system, respectively, (x) w ，y w ，z w ) Is the coordinates of the world coordinate system.
In one embodiment of the invention, as shown in FIG. 3, the raster pattern surface may be SPQ, the projector coordinate system may be (OS-XSYSZS), the world coordinate system may be (ow-xwywzw), and one plane in the projector coordinate system may be OSYSZS.
And S4, obtaining world coordinates corresponding to the discrete coordinates of the grating pattern according to the space plane equation of the binocular camera and the space plane equation of the projector.
In one embodiment of the present invention, referring to fig. 3, the spatial plane equation of the binocular camera and the spatial plane equation of the projector intersect, and the intersecting plane equations in this embodiment may be solved simultaneously to obtain world coordinates corresponding to the discrete coordinates of the grating pattern.
S5, repeating the steps S2 to S4 to obtain corner pixel coordinate-world coordinate data sets with more than 10 groups, and obtaining parameters of the projector according to the data sets.
In one embodiment of the present invention, constraint targets are set for each set of corner pixel coordinate-world coordinate data sets, and the constraint targets are specific to the minimum world coordinate error of each set of corner pixel coordinate-world coordinate data sets.
Further, according to the constraint target and through a maximum likelihood method, a transformation matrix can be obtained and SVD (Singular Value Decomposition ) decomposition is performed to obtain parameters of the projector. By setting constraint conditions and taking the minimum reprojection error as a target, the projector internal parameters and external parameters can be optimized.
The implementation of an embodiment of the present invention will be further described with reference to fig. 4.
Fig. 4 is a schematic diagram of the overall principle of an embodiment of the present invention.
In one particular embodiment of the invention, as shown in fig. 4, the stereoscopic vision system includes a projector, a first camera, and a second camera, wherein the first camera and the second camera are CCD (charge coupled device ) cameras.
In a specific embodiment of the present invention, the first camera and the second camera are calibrated by a Zhengyou calibration method, parameters thereof are obtained, then a grating pattern shown in fig. 2 is obtained by projecting a grating through a projector, angular points of the grating pattern are extracted, world coordinates are restored, more than 10 sets of angular point pixel coordinates-world coordinate data sets of the grating pattern are obtained, and parameters of the projector are obtained again by a Zhang Zhengyou calibration method. By adopting the mature camera calibration technology to calibrate the first camera and the second camera, the error existing when binarizing the grating patterns acquired by the first camera and the second camera and extracting the corner points can be reduced.
According to the virtual target calibration method combining grating projection and stereoscopic vision, parameters of the virtual target calibration method are firstly obtained through a calibration camera, then a grating pattern is projected to a checkerboard through a projector, the grating pattern and angular point pixel coordinates and discrete coordinates of the grating pattern are obtained, a space plane equation of a binocular camera is further fitted according to the parameters of the binocular camera and the angular point pixel coordinates of the grating pattern, a space plane equation of the projector is fitted according to the discrete coordinates of the grating pattern, finally world coordinates corresponding to the discrete coordinates of the grating pattern are calculated according to the space plane equation of the binocular camera and the space plane equation of the projector, a angular point pixel coordinate-world coordinate data set is obtained, more than 10 groups of angular point pixel coordinate-world coordinate data sets are obtained, and constraint targets are set to obtain parameters of the projector.
In the description of the present invention, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
In the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured," and the like are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements or interaction relationship between the two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.
In the present invention, unless expressly stated or limited otherwise, a first feature "up" or "down" a second feature may be the first and second features in direct contact, or the first and second features in indirect contact via an intervening medium. Moreover, a first feature being "above," "over" and "on" a second feature may be a first feature being directly above or obliquely above the second feature, or simply indicating that the first feature is level higher than the second feature. The first feature being "under", "below" and "beneath" the second feature may be the first feature being directly under or obliquely below the second feature, or simply indicating that the first feature is less level than the second feature.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (5)
1. A virtual target calibration method combining grating projection and stereoscopic vision is characterized by comprising the following steps:
s1, calibrating a binocular camera based on a checkerboard to obtain parameters of the binocular camera;
s2, taking the checkerboard as a background, projecting a grating to the checkerboard through a projector, and acquiring a grating pattern on the checkerboard through the binocular camera;
s3, acquiring angular point pixel coordinates of the grating pattern and discrete coordinates of the grating pattern, fitting a space plane equation of the binocular camera according to the angular point pixel coordinates of the grating pattern and parameters of the binocular camera, and fitting a space plane equation of the projector according to the discrete coordinates of the grating pattern;
s4, obtaining world coordinates corresponding to the discrete coordinates of the grating pattern according to the space plane equation of the binocular camera and the space plane equation of the projector;
s5, repeating the steps S2 to S4 to obtain corner pixel coordinate-world coordinate data sets of more than 10 groups, obtaining parameters of the projector according to the data sets, specifically setting constraint targets, wherein the constraint targets are minimum world coordinate errors of each group of corner pixel coordinate-world coordinate data sets, obtaining a conversion matrix according to the constraint targets through a maximum likelihood method, and carrying out SVD decomposition to obtain the parameters of the projector.
2. The method for calibrating a virtual target by combining grating projection and stereoscopic vision according to claim 1, wherein the grating projected by the projector is a grating with a specific period.
3. The method for calibrating a virtual target by combining raster projection and stereoscopic vision according to claim 2, wherein the step S3 comprises the steps of:
discretizing the grating pattern and the checkerboard pattern, and separating the grating pattern according to the discretization result to obtain the discrete coordinates of the grating pattern;
according to the corner pixel coordinates of the grating pattern and the parameters of the binocular camera, world coordinates corresponding to the corner pixel coordinates of the grating pattern are obtained, and the world coordinates corresponding to the corner pixel coordinates of the grating pattern are fitted to a space plane equation of the binocular camera through a least square method;
and fitting a space plane equation of the projector according to the discrete coordinates of the grating pattern.
4. The virtual target calibration method combining grating projection and stereoscopic vision according to claim 3, wherein the fitted spatial plane equation of the binocular camera is:
A 1 x a +B 1 y a +C 1 z a +D 1 ＝0
A 2 x b +B 2 y b +C 2 z b +D 2 ＝0，
the fitted spatial plane equation of the projector is:
conversion to world coordinate system: />
Wherein, (X s ，Y s ，Z s ) Is the coordinate of the grating pattern surface, alpha is the included angle between the grating pattern surface and a plane in the projector coordinate system, R sw And T sw Rotation matrix and translation matrix from projector coordinate system to world coordinate system, respectively, (x) w ，y w ，z w ) Is the coordinates of the world coordinate system.
5. The virtual target calibration method combining grating projection and stereoscopic vision according to claim 4, wherein a space plane equation of the binocular camera and a space plane equation of the projector are intersected, and the intersected plane equations are solved simultaneously to obtain world coordinates corresponding to discrete coordinates of the grating pattern.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910537131.6A CN110378967B (en) | 2019-06-20 | 2019-06-20 | Virtual target calibration method combining grating projection and stereoscopic vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910537131.6A CN110378967B (en) | 2019-06-20 | 2019-06-20 | Virtual target calibration method combining grating projection and stereoscopic vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110378967A CN110378967A (en) | 2019-10-25 |
| CN110378967B true CN110378967B (en) | 2023-09-22 |
Family
ID=68249047
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910537131.6A Active CN110378967B (en) | 2019-06-20 | 2019-06-20 | Virtual target calibration method combining grating projection and stereoscopic vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110378967B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111080713B (en) * | 2019-12-11 | 2023-03-28 | 四川深瑞视科技有限公司 | Camera calibration system and method |
| CN113409399B (en) * | 2021-06-10 | 2023-04-11 | 武汉库柏特科技有限公司 | Dual-camera combined calibration method, system and device |
| CN114049304A (en) * | 2021-10-14 | 2022-02-15 | 五邑大学 | 3D grating detection method and device, computer equipment and readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107657643A (en) * | 2017-08-28 | 2018-02-02 | 浙江工业大学 | A kind of parallax calculation method based on space plane constraint |
| EP3648055A1 (en) * | 2017-06-30 | 2020-05-06 | Hangzhou Hikvision Digital Technology Co., Ltd. | Camera parameter calibration method, device, apparatus, and system |
-
2019
- 2019-06-20 CN CN201910537131.6A patent/CN110378967B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3648055A1 (en) * | 2017-06-30 | 2020-05-06 | Hangzhou Hikvision Digital Technology Co., Ltd. | Camera parameter calibration method, device, apparatus, and system |
| CN107657643A (en) * | 2017-08-28 | 2018-02-02 | 浙江工业大学 | A kind of parallax calculation method based on space plane constraint |
Non-Patent Citations (1)
| Title |
|---|
| 多摄像机光栅投影三维测量系统的标定方法研究;汤明;《中国优秀硕士论文电子期刊网》;20180415(第04期);正文第三章 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110378967A (en) | 2019-10-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107194972B (en) | Camera calibration method and system | |
| CN106780590B (en) | Method and system for acquiring depth map | |
| CN104596502B (en) | Object posture measuring method based on CAD model and monocular vision | |
| CN105091849B (en) | A kind of non-parallel binocular distance-finding method of optical axis | |
| CN103106688B (en) | Based on the indoor method for reconstructing three-dimensional scene of double-deck method for registering | |
| CN102032878B (en) | Accurate on-line measurement method based on binocular stereo vision measurement system | |
| CN110378967B (en) | Virtual target calibration method combining grating projection and stereoscopic vision | |
| CN108520537B (en) | Binocular depth acquisition method based on luminosity parallax | |
| CN110390719A (en) | Based on flight time point cloud reconstructing apparatus | |
| CN105654547B (en) | Three-dimensional rebuilding method | |
| CN103530880B (en) | Based on the camera marking method of projection Gaussian network pattern | |
| CN107886547B (en) | Fisheye camera calibration method and system | |
| CN110728715A (en) | Camera angle self-adaptive adjusting method of intelligent inspection robot | |
| CN102665086B (en) | Method for obtaining parallax by using region-based local stereo matching | |
| CN108474658B (en) | Ground form detection method and system, unmanned aerial vehicle landing method and unmanned aerial vehicle | |
| CN106651794A (en) | Projection speckle correction method based on virtual camera | |
| CN107358631A (en) | A kind of binocular vision method for reconstructing for taking into account three-dimensional distortion | |
| CN110473262A (en) | Outer ginseng scaling method, device, storage medium and the electronic equipment of more mesh cameras | |
| CN102980526B (en) | Spatial digitizer and the scan method thereof of coloured image is obtained with black and white camera | |
| CN104596439A (en) | Speckle matching and three-dimensional measuring method based on phase information aiding | |
| US10771776B2 (en) | Apparatus and method for generating a camera model for an imaging system | |
| CN107545586B (en) | Depth obtaining method and system based on light field polar line plane image local part | |
| WO2020199439A1 (en) | Single- and dual-camera hybrid measurement-based three-dimensional point cloud computing method | |
| CN110349257B (en) | Phase pseudo mapping-based binocular measurement missing point cloud interpolation method | |
| CN113205592A (en) | Light field three-dimensional reconstruction method and system based on phase similarity |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |