Nothing Special   »   [go: up one dir, main page]

CN113793270A - Aerial image geometric correction method based on unmanned aerial vehicle attitude information - Google Patents

Aerial image geometric correction method based on unmanned aerial vehicle attitude information Download PDF

Info

Publication number
CN113793270A
CN113793270A CN202110896628.4A CN202110896628A CN113793270A CN 113793270 A CN113793270 A CN 113793270A CN 202110896628 A CN202110896628 A CN 202110896628A CN 113793270 A CN113793270 A CN 113793270A
Authority
CN
China
Prior art keywords
image
coordinates
correction
pixel
corrected
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110896628.4A
Other languages
Chinese (zh)
Other versions
CN113793270B (en
Inventor
赵辽英
向罗巧
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Dianzi University
Original Assignee
Hangzhou Dianzi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Dianzi University filed Critical Hangzhou Dianzi University
Priority to CN202110896628.4A priority Critical patent/CN113793270B/en
Publication of CN113793270A publication Critical patent/CN113793270A/en
Application granted granted Critical
Publication of CN113793270B publication Critical patent/CN113793270B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/80Geometric correction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/80Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10032Satellite or aerial image; Remote sensing

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

本发明公开了一种基于无人机姿态信息的航拍图像几何校正方法。本发明利用航拍图像的影像内方位元素,以及将航拍图像的POS参数直接作为影像外方位元素,通过三维重建空间坐标转换和间接法影像校正,实现对经过预处理的无人机遥感影像的几何校正。本发明充分利用了原始图像的像素信息,将多个姿态参数同时参与校正,避免了传统方法多个参数不通顺参与校正造成的校正误差,在无法进行人工控制点对选取时对无人机航拍图像进行几何校正,使得图像更加符合标准,并获得像点规则排列的校正影像,便于后续的航拍图像配准以及拼接等处理。

Figure 202110896628

The invention discloses an aerial image geometric correction method based on unmanned aerial vehicle attitude information. The present invention utilizes the azimuth element in the image of the aerial image and the POS parameter of the aerial image directly as the azimuth element outside the image, and realizes the geometric analysis of the preprocessed UAV remote sensing image through three-dimensional reconstruction space coordinate transformation and indirect method image correction. Correction. The invention makes full use of the pixel information of the original image, and participates in the correction of multiple attitude parameters at the same time, thereby avoiding the correction error caused by the inconsistent participation of multiple parameters in the traditional method. The image is geometrically corrected to make the image more in line with the standard, and a corrected image with regular arrangement of image points is obtained, which is convenient for subsequent aerial image registration and stitching.

Figure 202110896628

Description

一种基于无人机姿态信息的航拍图像几何校正方法A Geometric Correction Method of Aerial Image Based on UAV Attitude Information

技术领域technical field

本发明涉及图像处理领域,尤其涉及一种基于无人机姿态信息的航拍图像几何校正方法。The invention relates to the field of image processing, in particular to an aerial image geometric correction method based on UAV attitude information.

背景技术Background technique

随着无人机的快速发展,无人机低空遥感技术逐渐成为一种获取空间信息的重要手段。通过运用特定的处理系统,对无人机航拍图像中的像素进行系列操作,最终实现信息处理与信息提取的目的。至今,无人机已在建筑测量、故障检测、风电巡检等各种工程得到了很好的应用。With the rapid development of UAVs, UAV low-altitude remote sensing technology has gradually become an important means of obtaining spatial information. By using a specific processing system, a series of operations are performed on the pixels in the aerial image of the UAV, and the purpose of information processing and information extraction is finally realized. So far, drones have been well used in various projects such as building surveying, fault detection, and wind power inspection.

然而,无人机在执行航拍任务中,其飞行运动可能会因一些外在因素的影响比如风和气流等,使无人机成像传感器的姿态发生变换,以及无人机拍摄平台会出现俯仰、翻滚和偏航的情况,因而会导致获取的航拍图像发生俯仰变形、翻滚变形和偏航变形。正因为这些几何变形的存在,使得影像不能直接表示地物的形状和平面位置,从而导致获取的航拍图像产生几何畸变。对于同一地物区域,不同的飞行姿态所获取的航拍图像会存在比较大的差异,因此需要对获取的图像进行几何校正,以得到基于同一基准投影面上的航拍图像,从而提高无人机遥感影像数据处理的速度和定量提取信息的质量。现有的无人机航拍影像几何校正通常考虑人工布置控制点进行校正,然而这种方法虽然校正精度高,但需要投入大量的人力、物力、财力,尤其在山地、水面、沙漠等地布置控制点难度极大,不宜实施,且考虑到部分航拍工作人员不具备相关知识,在航拍后的数据集中存在的相关畸变,技术人员无法通过控制点方法进行校正,而无人机航拍影像的几何校正的质量和速度将会直接影响到后续的数据处理及分析决策。因此,如何快速准确的消除原始影像存在的几何畸变是无人机航拍影像预处理和应用的一项关键技术。However, when the drone is performing aerial photography tasks, its flight movement may be affected by some external factors, such as wind and airflow, which may change the attitude of the drone's imaging sensor, and the drone's shooting platform will appear pitching, roll and yaw, which will result in pitch, roll and yaw deformation of the acquired aerial image. Because of the existence of these geometric distortions, the image cannot directly represent the shape and plane position of the objects, resulting in geometric distortion of the acquired aerial images. For the same object area, the aerial images obtained by different flight attitudes will have relatively large differences. Therefore, it is necessary to perform geometric correction on the obtained images to obtain aerial images based on the same datum projection plane, thereby improving the remote sensing of UAVs. The speed of image data processing and the quality of quantitatively extracted information. Existing UAV aerial image geometry correction usually considers manually arranging control points for correction. However, although this method has high correction accuracy, it requires a lot of manpower, material resources, and financial resources, especially in mountains, water, deserts and other places. It is extremely difficult to implement, and considering that some aerial photography staff do not have the relevant knowledge, the relevant distortion in the data set after aerial photography cannot be corrected by the technical personnel through the control point method, and the geometric correction of UAV aerial photography images The quality and speed of the data will directly affect the subsequent data processing and analysis decisions. Therefore, how to quickly and accurately eliminate the geometric distortion of the original image is a key technology for the preprocessing and application of UAV aerial photography images.

考虑到无人机上装有惯性导航单元(INU)和GPS,可以实时保存无人机航行时的姿态角和坐标。根据无人机飞行过程中的成像传感器姿态、航拍时间、辅助导航定位数据及其各种空中传感器所获取的数据进行融合,完成无需控制点或正射参考的图像校正,实际应用中易于实现。为了使无人机遥感数据能够快速有效的应用于信息的提取和分析,需要建立相应的处理模型,在缺少地面控制点的情况下能够完成无人机遥感影像几何校正。影像校正的实质是实现两个二维影像之间的几何变换,因此,在进行影像校正之前,须建立原始影像与校正影像之间的几何关系,通常采用两种方法即直接法和间接法进行校正,李峥提出了一种仅仅利用航拍飞行信息进行图像几何校正的方法 (李峥,“缺少控制点的无人机遥感影像几何校正技术研究”,电子科技大学硕士论文,2010),便是通过直接法根据原始影像上每个像点的像平面坐标,通过姿态信息建立的几何关系,逐个解算其在校正影像上对应像点的像平面坐标,但该方法校正后的影像上的像点非规则排列,可能会出现空白或者重复的像素,对像素值的内插造成一定程度的困难,因此也就难以获得像点规则排列的校正影像。Considering that the UAV is equipped with an inertial navigation unit (INU) and GPS, the attitude angle and coordinates of the UAV during navigation can be saved in real time. According to the image sensor attitude, aerial photography time, auxiliary navigation and positioning data and the data obtained by various aerial sensors during the flight of the UAV, image correction without control points or orthophoto reference is completed, which is easy to implement in practical applications. In order to make UAV remote sensing data can be quickly and effectively applied to information extraction and analysis, it is necessary to establish a corresponding processing model, which can complete the geometric correction of UAV remote sensing images in the absence of ground control points. The essence of image correction is to realize the geometric transformation between two two-dimensional images. Therefore, before performing image correction, the geometric relationship between the original image and the corrected image must be established. Usually, two methods are used, namely, the direct method and the indirect method. Correction, Li Zheng proposed a method of image geometric correction only using aerial photography flight information (Li Zheng, "Research on Geometric Correction Technology of UAV Remote Sensing Image Lack of Control Points", University of Electronic Science and Technology Master Thesis, 2010), which is Through the direct method, according to the image plane coordinates of each image point on the original image, and through the geometric relationship established by the attitude information, the image plane coordinates of the corresponding image points on the corrected image are calculated one by one. If the dots are irregularly arranged, there may be blank or repeated pixels, which causes a certain degree of difficulty in the interpolation of pixel values, so it is difficult to obtain a corrected image with regular arrangement of pixels.

发明内容SUMMARY OF THE INVENTION

由于直接法存在上述缺点,考虑到航拍图像原有信息的真实性,本发明给出在无地面控制点的情况下,根据相机检校报告可知影像内方位元素,即像主点在框标坐标系中坐标和相机焦距,同时,将航拍图像的POS参数直接作为影像外方位元素,通过三维重建空间坐标转换和间接法影像校正,实现对经过预处理的无人机遥感影像的几何校正,将存在几何变形的无人机遥感影像平面变换至大地水平面。Due to the above-mentioned shortcomings of the direct method, and considering the authenticity of the original information of the aerial image, the present invention provides that in the absence of ground control points, the orientation elements in the image can be known according to the camera calibration report, that is, the main point in the frame coordinates At the same time, the POS parameter of the aerial image is directly used as the external orientation element of the image, and the geometric correction of the preprocessed UAV remote sensing image is realized through three-dimensional reconstruction space coordinate transformation and indirect image correction. The plane of the UAV remote sensing image with geometric deformation is transformed to the ground level.

本发明方法实现过程中,景物在相机传感器上成像,形成一张二维图像。记空间某点在校正影像上的像平面坐标为(u',v'),在原始影像上的像平面坐标为(u,v)。During the implementation of the method of the present invention, the scene is imaged on the camera sensor to form a two-dimensional image. Note that the image plane coordinates of a point in space on the corrected image are (u', v'), and the image plane coordinates on the original image are (u, v).

为实现上述目的,本发明提供了如下技术方案:For achieving the above object, the present invention provides the following technical solutions:

步骤1:求校正影像坐标对应的物理等价坐标Step 1: Find the physical equivalent coordinates corresponding to the corrected image coordinates

根据相机参数计算单位像素的物理长度(dx,dy),根据所述物理长度和相机焦距f,求内参数矩阵K1,校正后像点与世界坐标系坐标之间不存在旋转、平移的畸变,校正后像点(u',v')对应的物理等价坐标为(X,Y,Z),所述物理等价坐标指物理坐标系的坐标值除以飞机飞行高度的坐标,用公式表示为:Calculate the physical length (d x , dy ) of the unit pixel according to the camera parameters, and calculate the internal parameter matrix K 1 according to the physical length and the camera focal length f. There is no rotation or translation between the corrected image point and the coordinates of the world coordinate system The distortion of the corrected image point (u', v') corresponds to the physical equivalent coordinates (X, Y, Z), and the physical equivalent coordinates refer to the coordinates of the physical coordinate system divided by the coordinates of the flight height of the aircraft, The formula is expressed as:

Figure BDA0003198194580000031
Figure BDA0003198194580000031

步骤2:求物理等价坐标对应的校正前影像平面坐标Step 2: Find the image plane coordinates before correction corresponding to the physical equivalent coordinates

根据步骤1求得的物理等价坐标和飞行姿态参数,按如下公式计算校正前像平面坐标;According to the physical equivalent coordinates and flight attitude parameters obtained in step 1, calculate the image plane coordinates before correction according to the following formula;

Figure BDA0003198194580000032
Figure BDA0003198194580000032

其中R为根据飞行姿态参数求的旋转矩阵,T为平移向量;Among them, R is the rotation matrix calculated according to the flight attitude parameters, and T is the translation vector;

步骤3:计算校正后的影像Step 3: Calculate the corrected image

对每个校正后像点坐标(u',v'),按步骤1和2求得的坐标变换关系,得到校正前图像的像点坐标(u,v);通过对校正前图像像素插值,得到校正后图像中坐标为(u',v')的像素值;For each corrected image point coordinates (u', v'), according to the coordinate transformation relationship obtained in steps 1 and 2, the image point coordinates (u, v) of the uncorrected image are obtained; by interpolating the uncorrected image pixels, Obtain the pixel value whose coordinates are (u', v') in the corrected image;

步骤4:选择合适的重采样方法对输出图像像元进行灰度赋值Step 4: Select an appropriate resampling method to assign grayscale values to the output image pixels

对每个校正后像点坐标(u',v'),按步骤1和2求得的坐标变换关系,得到校正前图像像素坐标(u,v),采用插值法对校正影像进行重采样,得到校正后图像中坐标为(u',v')的像素值,所述插值法包括最邻近插值法、双线性插值法以及三次立方卷积插值法。For each corrected image point coordinate (u', v'), according to the coordinate transformation relationship obtained in steps 1 and 2, obtain the uncorrected image pixel coordinate (u, v), and use the interpolation method to resample the corrected image, The pixel value whose coordinates are (u', v') in the corrected image are obtained, and the interpolation method includes the nearest neighbor interpolation method, the bilinear interpolation method and the cubic convolution interpolation method.

进一步的,步骤1和2所述内参数矩阵K1计算公式为Further, the calculation formula of the internal parameter matrix K 1 described in steps 1 and 2 is:

Figure BDA0003198194580000033
Figure BDA0003198194580000033

其中(Cx,Cy)表示图像光心在图像像素坐标下的坐标,dx和dy分别表示传感器x轴和y轴上单位像素的尺寸大小,f为相机焦距。Where (C x , Cy ) represents the coordinates of the image optical center in the image pixel coordinates, dx and dy represent the size of the unit pixel on the x-axis and y-axis of the sensor, respectively, and f is the camera focal length.

进一步的,步骤2所述旋转矩阵R计算公式为:Further, the calculation formula of the rotation matrix R described in step 2 is:

Figure BDA0003198194580000041
Figure BDA0003198194580000041

Figure BDA0003198194580000042
Figure BDA0003198194580000042

Figure BDA0003198194580000043
Figure BDA0003198194580000043

Figure BDA0003198194580000044
Figure BDA0003198194580000044

其中,

Figure BDA0003198194580000045
θ和ω分别为影像姿态参数中的俯仰角、翻滚角和偏航角。in,
Figure BDA0003198194580000045
θ and ω are the pitch angle, roll angle and yaw angle in the image attitude parameters, respectively.

本发明的有益效果:Beneficial effects of the present invention:

(1)使用遥感图像的地理姿态信息构建校正前后的映射关系,在无法进行人工控制点对选取时对无人机航拍图像进行几何校正,使得图像更加符合标准,并获得像点规则排列的校正影像,便于后续的航拍图像配准以及拼接等处理。(1) Use the geographic attitude information of the remote sensing image to construct the mapping relationship before and after correction, and perform geometric correction on the UAV aerial image when manual control point selection cannot be performed, so that the image is more in line with the standard, and the correction of regular arrangement of image points is obtained. image, which is convenient for subsequent aerial image registration and stitching.

(2)多个姿态参数同时参与校正,避免了传统方法多个参数不通顺参与校正造成的校正误差。(2) Multiple attitude parameters participate in the correction at the same time, which avoids the correction error caused by the inconsistent participation of multiple parameters in the traditional method.

附图说明Description of drawings

图1直接法与间接法校正方法对比示意图Figure 1 Schematic diagram of the comparison between the direct method and the indirect method

图2间接法影像校正图Figure 2 Indirect method image correction diagram

图3(a)为原始组件航拍图像;Figure 3(a) is an aerial image of the original assembly;

图3(b)为校正后组件图像Figure 3(b) is the component image after correction

图4(a)为原始组件航拍图像;Figure 4(a) is the aerial image of the original assembly;

图4(b)为校正后组件细节图像Figure 4(b) is the component detail image after correction

具体实施方式Detailed ways

为了使本发明的目的、技术方案和优点更加清楚,下面结合具体的实施例和附图,详细说明本发明,并描述了具体实施例以简化本发明。但是需要认识到,本发明不局限于所说明的实施例,并且在不脱离基本原理的前提下,本发明的各种修改是可能的,这些等价形式同样落于本申请所附权利要求书所限定的范围。In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention is described in detail below with reference to specific embodiments and accompanying drawings, and the specific embodiments are described to simplify the present invention. It should be recognized, however, that the present invention is not limited to the illustrated embodiments, and that various modifications of the present invention are possible without departing from the basic principles, the equivalents of which also fall within the claims appended hereto. limited range.

如图1和图2所示,一种基于无人机航拍图像姿态信息几何校正方法,具体实现如下:As shown in Figure 1 and Figure 2, a geometric correction method based on the attitude information of UAV aerial imagery, the specific implementation is as follows:

步骤1:求校正影像坐标对应的物理等价坐标Step 1: Find the physical equivalent coordinates corresponding to the corrected image coordinates

根据相机参数计算单位像素的物理长度(dx,dy),根据所述物理长度和相机焦距f,求内参数矩阵K1,校正后像点与世界坐标系坐标之间不存在旋转、平移的畸变,校正后像点(u',v')对应的物理等价坐标为(X,Y,Z),所述物理等价坐标指物理坐标系的坐标值除以飞机飞行高度的坐标,则其坐标转换公式为:Calculate the physical length (d x , dy ) of the unit pixel according to the camera parameters, and calculate the internal parameter matrix K 1 according to the physical length and the camera focal length f. There is no rotation or translation between the corrected image point and the coordinates of the world coordinate system The distortion of the corrected image point (u', v') corresponds to the physical equivalent coordinates (X, Y, Z), and the physical equivalent coordinates refer to the coordinates of the physical coordinate system divided by the coordinates of the flight height of the aircraft, Then its coordinate conversion formula is:

Figure BDA0003198194580000051
Figure BDA0003198194580000051

步骤2:求物理等价坐标对应的校正前影像平面坐标Step 2: Find the image plane coordinates before correction corresponding to the physical equivalent coordinates

根据步骤1求得的物理等价坐标和飞行姿态参数,按如下公式计算校正前像平面坐标;According to the physical equivalent coordinates and flight attitude parameters obtained in step 1, calculate the image plane coordinates before correction according to the following formula;

Figure BDA0003198194580000052
Figure BDA0003198194580000052

其中R为根据飞行姿态参数求的旋转矩阵,T为平移向量;Among them, R is the rotation matrix calculated according to the flight attitude parameters, and T is the translation vector;

步骤3:选择合适的重采样方法对输出图像像元进行灰度赋值;Step 3: Select an appropriate resampling method to assign grayscale values to the output image pixels;

对每个校正后坐标(u',v'),按步骤1和2求得的坐标变换关系,得到校正前图像像素坐标(u,v),完成从校正后的图像像素坐标到校正前图像像素坐标的映射。坐标变换完成后,采用插值法对校正影像进行重采样,得到校正后图像中坐标为(u',v')的像素值。For each corrected coordinate (u', v'), according to the coordinate transformation relationship obtained in steps 1 and 2, the uncorrected image pixel coordinates (u, v) are obtained, and the pixel coordinates of the corrected image to the uncorrected image are completed. A map of pixel coordinates. After the coordinate transformation is completed, use the interpolation method to resample the corrected image to obtain the pixel value whose coordinates are (u', v') in the corrected image.

所述的插值法的实现是把由上述步骤算得的原始图像点位上的灰度值取出并填回到空白输出图像点阵中相应的像素点位上去。由于并不一定刚好位于原始图像的某个像素中心,为此必须进行灰度内插,一般常用的有最邻近插值法、双线性插值法以及三次立方卷积插值法三种。在通过对校正前图像像素进行插值,得到校正后图像对应坐标的像素值,即可完成图像校正。The implementation of the interpolation method is to take out the gray value of the original image point calculated by the above steps and fill it back to the corresponding pixel point in the blank output image point matrix. Because it is not necessarily located at the center of a certain pixel of the original image, grayscale interpolation must be performed for this purpose. Generally, there are three commonly used methods: nearest neighbor interpolation, bilinear interpolation and cubic convolution interpolation. Image correction can be completed by interpolating the pixels of the image before correction to obtain the pixel values of the corresponding coordinates of the corrected image.

进一步的,步骤1和2所述内参数矩阵K1计算公式为Further, the calculation formula of the internal parameter matrix K 1 described in steps 1 and 2 is:

Figure BDA0003198194580000061
Figure BDA0003198194580000061

其中(Cx,Cy)表示图像光心在图像像素坐标下的坐标,dx和dy分别表示传感器x轴和y轴上单位像素的尺寸大小,f为相机焦距。Where (C x , Cy ) represents the coordinates of the image optical center in the image pixel coordinates, dx and dy represent the size of the unit pixel on the x-axis and y-axis of the sensor, respectively, and f is the camera focal length.

进一步的,步骤2所述旋转矩阵R计算公式为:Further, the calculation formula of the rotation matrix R described in step 2 is:

Figure BDA0003198194580000062
Figure BDA0003198194580000062

Figure BDA0003198194580000063
Figure BDA0003198194580000063

Figure BDA0003198194580000064
Figure BDA0003198194580000064

Figure BDA0003198194580000065
Figure BDA0003198194580000065

其中,

Figure BDA0003198194580000066
θ和ω分别为影像姿态参数中的俯仰角、翻滚角和偏航角。in,
Figure BDA0003198194580000066
θ and ω are the pitch angle, roll angle and yaw angle in the image attitude parameters, respectively.

本发明利用航拍图像的影像内方位元素,以及将航拍图像的POS 参数直接作为影像外方位元素,通过三维重建空间坐标转换和间接法影像校正,实现对经过预处理的无人机遥感影像的几何校正。本发明充分利用了原始图像的像素信息,将多个姿态参数同时参与校正,避免了传统方法多个参数不通顺参与校正造成的校正误差,在无法进行人工控制点对选取时对无人机航拍图像进行几何校正,使得图像更加符合标准,并获得像点规则排列的校正影像,便于后续的航拍图像配准以及拼接等处理。The present invention utilizes the azimuth element in the image of the aerial image, and directly uses the POS parameter of the aerial image as the azimuth element outside the image, and realizes the geometric analysis of the preprocessed UAV remote sensing image through three-dimensional reconstruction space coordinate transformation and indirect method image correction. Correction. The invention makes full use of the pixel information of the original image, and participates in the correction of multiple attitude parameters at the same time, thereby avoiding the correction error caused by the inconsistent participation of multiple parameters in the traditional method. The image is geometrically corrected to make the image more in line with the standard, and a corrected image with regular arrangement of image points is obtained, which is convenient for subsequent aerial image registration and stitching.

本发明具体实施过程如下:The specific implementation process of the present invention is as follows:

将采用无人机获取的光伏组件航拍图片,挑选出其中角度偏移稍大的图像,如图3(a)所示,对其进行几何校正,相关图像采集参数如表1.1所示:The aerial photo of the photovoltaic module obtained by the drone is used, and the image with a slightly larger angle offset is selected, as shown in Figure 3(a), and the geometric correction is performed on it. The relevant image acquisition parameters are shown in Table 1.1:

表1.1图像采集参数表Table 1.1 Image Acquisition Parameters Table

Figure BDA0003198194580000072
Figure BDA0003198194580000072

图像校正过程如下:The image correction process is as follows:

步骤1:求校正影像坐标对应的物理等价坐标Step 1: Find the physical equivalent coordinates corresponding to the corrected image coordinates

摄像机光轴与图像平面的交点,通常位于图像中心处,按下式计算图像光心坐标Cx和Cy

Figure BDA0003198194580000071
The intersection of the optical axis of the camera and the image plane is usually located at the center of the image, and the image optical center coordinates C x and C y are calculated as follows:
Figure BDA0003198194580000071

根据传感器尺寸大小通过查表可知其宽Fw为7.4mm,高Fh为 5.6mm,计算传感器x轴和y轴上单位像素的尺寸大小dx= Fw/W,dy=Fh/H。According to the size of the sensor, the width F w is 7.4mm and the height F h is 5.6mm. Calculate the size of the unit pixel on the x-axis and y-axis of the sensor. d x = F w /W, dy = F h / H.

根据公式(3)计算内参数矩阵K1,根据公式(1)逐像素计算物理等价坐标。The internal parameter matrix K 1 is calculated according to formula (3), and the physical equivalent coordinates are calculated pixel by pixel according to formula (1).

步骤2:求物理等价坐标对应的校正前像平面坐标Step 2: Find the image plane coordinates before correction corresponding to the physical equivalent coordinates

将影像姿态参数中的俯仰角

Figure BDA0003198194580000073
翻滚角θ和偏航角ω代入公式(4),计算得到旋转矩阵R,平移向量T=[0 0 0]-1,根据公式(2)逐像素计算校正前像平面坐标。Convert the pitch angle in the image attitude parameter
Figure BDA0003198194580000073
The roll angle θ and the yaw angle ω are substituted into formula (4), the rotation matrix R is calculated, and the translation vector T=[0 0 0] -1 , and the image plane coordinates before correction are calculated pixel by pixel according to formula (2).

步骤3:计算校正后的影像Step 3: Calculate the corrected image

对每个校正后坐标(u',v'),按步骤1和2求得的坐标变换关系,得到校正前图像像素坐标(u,v),采用双线性插值法对校正影像进行重采样,得到校正后图像中坐标为(u',v')的像素值,结果如图3(b) 所示。放大图像细节,得到校正前后的细节对比,如图4(a)和图4 (b),通过同样大小的矩形蒙版进行组件阴影部分的覆盖,由图4(b) 可得出校正后的组件阴影部分与蒙版的重合率更高,其光伏组件的轮廓也更具有几何特征。For each corrected coordinate (u', v'), according to the coordinate transformation relationship obtained in steps 1 and 2, the pixel coordinates (u, v) of the uncorrected image are obtained, and the corrected image is resampled by bilinear interpolation , to obtain the pixel value whose coordinates are (u', v') in the corrected image, and the result is shown in Figure 3(b). Zoom in on the details of the image to get the detail comparison before and after correction, as shown in Figure 4(a) and Figure 4(b). The shadow part of the component is covered by a rectangular mask of the same size. From Figure 4(b), the corrected image can be obtained. The shadow part of the module has a higher coincidence rate with the mask, and the outline of the PV module has more geometric features.

Claims (4)

1.一种基于无人机姿态信息的航拍图像几何校正方法,其特征在于,包括如下步骤:1. an aerial photographing image geometric correction method based on unmanned aerial vehicle attitude information, is characterized in that, comprises the steps: 步骤1:求校正影像坐标对应的物理等价坐标Step 1: Find the physical equivalent coordinates corresponding to the corrected image coordinates 根据相机参数计算单位像素的物理长度(dx,dy),根据所述物理长度和相机焦距f,求内参数矩阵K1,校正后像点与世界坐标系坐标之间不存在旋转、平移的畸变,校正后像点(u',v')对应的物理等价坐标为(X,Y,Z),所述物理等价坐标指物理坐标系的坐标值除以飞机飞行高度的坐标,用公式表示为:Calculate the physical length (d x , dy ) of the unit pixel according to the camera parameters, and calculate the internal parameter matrix K 1 according to the physical length and the camera focal length f. There is no rotation or translation between the corrected image point and the coordinates of the world coordinate system The distortion of the corrected image point (u', v') corresponds to the physical equivalent coordinates (X, Y, Z), and the physical equivalent coordinates refer to the coordinates of the physical coordinate system divided by the coordinates of the flight height of the aircraft, The formula is expressed as:
Figure FDA0003198194570000011
Figure FDA0003198194570000011
步骤2:求物理等价坐标对应的校正前影像平面坐标Step 2: Find the image plane coordinates before correction corresponding to the physical equivalent coordinates 根据步骤1求得的物理等价坐标和飞行姿态参数,按如下公式计算校正前像平面坐标;According to the physical equivalent coordinates and flight attitude parameters obtained in step 1, calculate the image plane coordinates before correction according to the following formula;
Figure FDA0003198194570000012
Figure FDA0003198194570000012
其中R为根据飞行姿态参数求的旋转矩阵,T为平移向量;Among them, R is the rotation matrix calculated according to the flight attitude parameters, and T is the translation vector; 步骤3:选择合适的重采样方法对输出图像像元进行灰度赋值;Step 3: Select an appropriate resampling method to assign grayscale values to the output image pixels; 对每个校正后坐标(u',v'),按步骤1和2求得的坐标变换关系,得到校正前图像像素坐标(u,v),完成从校正后的图像像素坐标到校正前图像像素坐标的映射;坐标变换完成后,采用插值法对校正影像进行重采样,得到校正后图像中坐标为(u',v')的像素值,所述插值法包括最邻近插值法、双线性插值法以及三次立方卷积插值法。For each corrected coordinate (u', v'), according to the coordinate transformation relationship obtained in steps 1 and 2, the uncorrected image pixel coordinates (u, v) are obtained, and the pixel coordinates of the corrected image to the uncorrected image are completed. The mapping of pixel coordinates; after the coordinate transformation is completed, the corrected image is resampled by an interpolation method, and the pixel value with the coordinates (u', v') in the corrected image is obtained. The interpolation method includes the nearest neighbor interpolation method, double line Interpolation and cubic convolution interpolation.
2.根据权利要求1所述的基于无人机航拍图像姿态信息几何校正方法,其特征在于步骤1和2所述内参数矩阵K1计算公式为2. the method for geometric correction based on the aerial photography image attitude information of unmanned aerial vehicle according to claim 1, is characterized in that the described internal parameter matrix K 1 calculation formula of steps 1 and 2 is:
Figure FDA0003198194570000013
Figure FDA0003198194570000013
其中(Cx,Cy)表示图像光心在图像像素坐标下的坐标,dx和dy分别表示传感器x轴和y轴上单位像素的尺寸大小,f为相机焦距。Where (C x , Cy ) represents the coordinates of the image optical center in the image pixel coordinates, dx and dy represent the size of the unit pixel on the x-axis and y-axis of the sensor, respectively, and f is the camera focal length.
3.根据权利要求1或2所述的基于无人机航拍图像姿态信息几何校正方法,其特征在于步骤2所述旋转矩阵R计算公式为:3. according to claim 1 and 2, it is characterized in that the rotation matrix R calculation formula described in step 2 is:
Figure FDA0003198194570000021
Figure FDA0003198194570000021
Figure FDA0003198194570000022
Figure FDA0003198194570000022
Figure FDA0003198194570000023
Figure FDA0003198194570000023
Figure FDA0003198194570000024
Figure FDA0003198194570000024
其中,
Figure FDA0003198194570000025
θ和ω分别为影像姿态参数中的俯仰角、翻滚角和偏航角。
in,
Figure FDA0003198194570000025
θ and ω are the pitch angle, roll angle and yaw angle in the image attitude parameters, respectively.
4.根据权利要求1或2所述的基于无人机航拍图像姿态信息几何校正方法,其特征在于所述的插值法的实现是把由上述步骤算得的原始图像点位上的灰度值取出并填回到空白输出图像点阵中相应的像素点位上去;由于并不一定刚好位于原始图像的某个像素中心,必须经过灰度内插确定该处的灰度值,可选择合适的插值方法对畸变图像的输出图像像元进行灰度赋值,在通过对校正前图像像素进行插值,得到校正后图像对应坐标的像素值,即可完成图像校正。4. according to claim 1 and 2, it is characterized in that the realization of described interpolation method is to take out the gray value on the original image point position that is calculated by above-mentioned steps based on UAV aerial photography image attitude information geometric correction method And fill it back to the corresponding pixel position in the blank output image dot matrix; because it is not necessarily located at the center of a certain pixel of the original image, the gray value of the place must be determined by grayscale interpolation, and an appropriate interpolation value can be selected. The method assigns grayscale to the pixels of the output image of the distorted image, and obtains the pixel values of the corresponding coordinates of the corrected image by interpolating the pixels of the image before correction, and then the image correction can be completed.
CN202110896628.4A 2021-08-05 2021-08-05 A geometric correction method for aerial images based on UAV attitude information Active CN113793270B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110896628.4A CN113793270B (en) 2021-08-05 2021-08-05 A geometric correction method for aerial images based on UAV attitude information

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110896628.4A CN113793270B (en) 2021-08-05 2021-08-05 A geometric correction method for aerial images based on UAV attitude information

Publications (2)

Publication Number Publication Date
CN113793270A true CN113793270A (en) 2021-12-14
CN113793270B CN113793270B (en) 2024-12-06

Family

ID=78877155

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110896628.4A Active CN113793270B (en) 2021-08-05 2021-08-05 A geometric correction method for aerial images based on UAV attitude information

Country Status (1)

Country Link
CN (1) CN113793270B (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114612788A (en) * 2022-03-22 2022-06-10 东北林业大学 Urban landscape plant diversity monitoring method based on neural network
CN115326054A (en) * 2022-08-24 2022-11-11 中国热带农业科学院农业机械研究所 A kind of automatic navigation method of crawler agricultural vehicle
CN115618749A (en) * 2022-12-05 2023-01-17 四川腾盾科技有限公司 Error compensation method for real-time positioning of large unmanned aerial vehicle
CN116228598A (en) * 2023-05-06 2023-06-06 四川师范大学 Geometric distortion correction device for remote sensing image of mountain unmanned aerial vehicle and application
CN117291980A (en) * 2023-10-09 2023-12-26 宁波博登智能科技有限公司 Single unmanned aerial vehicle image pixel positioning method based on deep learning
CN117671543A (en) * 2024-01-31 2024-03-08 青岛云世纪信息科技有限公司 Unmanned aerial vehicle image coordinate calibration method and system and electronic equipment
CN118037546A (en) * 2024-03-25 2024-05-14 电子科技大学 Method for estimating acquisition time of remote sensing mosaic image element of unmanned aerial vehicle

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101345554B1 (en) * 2013-10-18 2014-01-02 중앙항업(주) Method of resampling high resolution digital multi band imagery from line senser into frame type imagery to construct gis(uis), digital map and 3d spatial information using ground control point and gps/ins data
CN108109118A (en) * 2017-12-15 2018-06-01 大连理工大学 A kind of Aerial Images geometric correction method at no control point
CN111174697A (en) * 2019-12-13 2020-05-19 中国南方电网有限责任公司超高压输电公司柳州局 Stereoscopic vision image accurate measurement method based on unmanned aerial vehicle

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101345554B1 (en) * 2013-10-18 2014-01-02 중앙항업(주) Method of resampling high resolution digital multi band imagery from line senser into frame type imagery to construct gis(uis), digital map and 3d spatial information using ground control point and gps/ins data
CN108109118A (en) * 2017-12-15 2018-06-01 大连理工大学 A kind of Aerial Images geometric correction method at no control point
CN111174697A (en) * 2019-12-13 2020-05-19 中国南方电网有限责任公司超高压输电公司柳州局 Stereoscopic vision image accurate measurement method based on unmanned aerial vehicle

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114612788A (en) * 2022-03-22 2022-06-10 东北林业大学 Urban landscape plant diversity monitoring method based on neural network
CN115326054A (en) * 2022-08-24 2022-11-11 中国热带农业科学院农业机械研究所 A kind of automatic navigation method of crawler agricultural vehicle
CN115618749A (en) * 2022-12-05 2023-01-17 四川腾盾科技有限公司 Error compensation method for real-time positioning of large unmanned aerial vehicle
CN116228598A (en) * 2023-05-06 2023-06-06 四川师范大学 Geometric distortion correction device for remote sensing image of mountain unmanned aerial vehicle and application
CN116228598B (en) * 2023-05-06 2023-07-11 四川师范大学 A geometric distortion correction device and application for remote sensing images of UAVs in mountainous areas
CN117291980A (en) * 2023-10-09 2023-12-26 宁波博登智能科技有限公司 Single unmanned aerial vehicle image pixel positioning method based on deep learning
CN117291980B (en) * 2023-10-09 2024-03-15 宁波博登智能科技有限公司 Single unmanned aerial vehicle image pixel positioning method based on deep learning
CN117671543A (en) * 2024-01-31 2024-03-08 青岛云世纪信息科技有限公司 Unmanned aerial vehicle image coordinate calibration method and system and electronic equipment
CN117671543B (en) * 2024-01-31 2024-04-19 青岛云世纪信息科技有限公司 Unmanned aerial vehicle image coordinate calibration method and system and electronic equipment
CN118037546A (en) * 2024-03-25 2024-05-14 电子科技大学 Method for estimating acquisition time of remote sensing mosaic image element of unmanned aerial vehicle

Also Published As

Publication number Publication date
CN113793270B (en) 2024-12-06

Similar Documents

Publication Publication Date Title
CN113793270B (en) A geometric correction method for aerial images based on UAV attitude information
CN110648398B (en) Real-time ortho image generation method and system based on unmanned aerial vehicle aerial data
CN112767391B (en) Power grid line part defect positioning method integrating three-dimensional point cloud and two-dimensional image
WO2022142759A1 (en) Lidar and camera joint calibration method
CN105627991B (en) A kind of unmanned plane image real time panoramic joining method and system
CN107316325B (en) Airborne laser point cloud and image registration fusion method based on image registration
CN110675450B (en) Method and system for generating orthoimage in real time based on SLAM technology
CN107492069B (en) Image fusion method based on multi-lens sensor
CN104268935A (en) Feature-based airborne laser point cloud and image data fusion system and method
CN107274380B (en) A fast mosaic method for UAV multispectral images
CN106127697A (en) Unmanned aerial vehicle onboard imaging EO-1 hyperion geometric correction method
CN113739767B (en) Method for producing orthoscopic image aiming at image acquired by domestic area array swinging imaging system
CN115187798A (en) Multi-unmanned aerial vehicle high-precision matching positioning method
CN114549871B (en) A method for matching drone aerial images with satellite images
CN110555813B (en) Rapid geometric correction method and system for remote sensing image of unmanned aerial vehicle
CN113205603A (en) Three-dimensional point cloud splicing reconstruction method based on rotating platform
CN114913297B (en) Scene orthophoto image generation method based on MVS dense point cloud
CN114998545A (en) Three-dimensional modeling shadow recognition system based on deep learning
CN117665841B (en) Geographic space information acquisition mapping method and device
CN110631555A (en) Orthorectification method of historical imagery based on second-order polynomial block adjustment without control points
CN116385504A (en) Inspection and ranging method based on unmanned aerial vehicle acquisition point cloud and image registration
CN110223233A (en) A kind of unmanned plane based on image mosaic builds drawing method
CN108109118B (en) A Geometric Correction Method of Aerial Image without Control Points
US20240412332A1 (en) Joint imaging system based on unmanned aerial vehicle platform and image enhancement fusion method
JP3490774B2 (en) How to generate geospecific textures

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant