Open AccessArticle

Inclinometer Assembly Error Calibration and Horizontal Image Correction in Photoelectric Measurement Systems

Xiaofang Kong

¹,

Qian Chen

^1,*,

Jiajie Wang

¹,

Guohua Gu

¹,

Pengcheng Wang

¹,

Weixian Qian

¹,

Kan Ren

¹ and

Xiaotao Miao

Jiangsu Key Laboratory of Spectral Imaging and Intelligent Sense, School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, China

Jilin Jiangji Special Industries Co., Ltd., Zun Yi West Road 17#, Jilin 132021, China

Author to whom correspondence should be addressed.

Sensors 2018, 18(1), 248; https://doi.org/10.3390/s18010248

Submission received: 6 November 2017 / Revised: 5 January 2018 / Accepted: 13 January 2018 / Published: 16 January 2018

(This article belongs to the Special Issue Charge-Coupled Device (CCD) Sensors)

Download

Browse Figures

Figure 1
An example of the camera and inclinometer coordinate systems. "> Figure 2
The plumb line in the real world should be a vertical line on the image plane when the camera is placed horizontally in the photoelectric system. "> Figure 3
Relationship between the geodetic coordinate system and the inclinometer coordinate system. "> Figure 4
Flow chart of horizontal image correction. "> Figure 5
The schematic diagram of one situation. "> Figure 6
A photoelectric measurement system (System 1) with a high-resolution industrial camera and a dual-axis inclinometer. "> Figure 7
The photoelectric measurement system (System 2) by the Basler acA2040-25gm camera and the SCA126T inclinometer. "> Figure 8
The checkerboard used for camera calibration. "> Figure 9
Images with detected plumb lines before and after horizontal correction in System 1. (a) The original image <math display="inline"> <semantics> <mrow> <msub> <mi>I</mi> <mn>3</mn> </msub> </mrow> </semantics> </math>; (b) The horizontally corrected image <math display="inline"> <semantics> <mrow> <msub> <mi>I</mi> <mrow> <mn>3</mn> <mo>_</mo> <mi>c</mi> <mi>o</mi> <mi>r</mi> <mi>r</mi> <mi>e</mi> <mi>c</mi> <mi>t</mi> <mi>e</mi> <mi>d</mi> </mrow> </msub> </mrow> </semantics> </math>. "> Figure 10
Images before and after correction using System 2. (a) The original distorted image, <math display="inline"> <semantics> <mrow> <msub> <mi>I</mi> <mn>6</mn> </msub> </mrow> </semantics> </math>; (b) The undistorted image, <math display="inline"> <semantics> <mrow> <msub> <mi>I</mi> <mrow> <mn>6</mn> <mo>_</mo> <mi>u</mi> <mi>n</mi> <mi>d</mi> <mi>i</mi> <mi>s</mi> <mi>t</mi> <mi>o</mi> <mi>r</mi> <mi>t</mi> <mi>e</mi> <mi>d</mi> </mrow> </msub> </mrow> </semantics> </math>, with detected plumb lines; (c) The horizontally corrected image, <math display="inline"> <semantics> <mrow> <msub> <mi>I</mi> <mrow> <mn>6</mn> <mo>_</mo> <mi>c</mi> <mi>o</mi> <mi>r</mi> <mi>r</mi> <mi>e</mi> <mi>c</mi> <mi>t</mi> <mi>e</mi> <mi>d</mi> </mrow> </msub> </mrow> </semantics> </math>, with detected plumb lines. "> Figure 11
Euler angles for each experiment. (a) Yaw; (b) Pitch; (c) Roll. "> Figure 12
Images with lines detected before (first row) and after (second row) horizontal correction. "> Figure 13
Images with lines detected after horizontal correction using the four methods. (a) Proposed method; (b) PCA method; (c) Hough method; (d) Radon method. ">

Versions Notes

Abstract

Inclinometer assembly error is one of the key factors affecting the measurement accuracy of photoelectric measurement systems. In order to solve the problem of the lack of complete attitude information in the measurement system, this paper proposes a new inclinometer assembly error calibration and horizontal image correction method utilizing plumb lines in the scenario. Based on the principle that the plumb line in the scenario should be a vertical line on the image plane when the camera is placed horizontally in the photoelectric system, the direction cosine matrix between the geodetic coordinate system and the inclinometer coordinate system is calculated firstly by three-dimensional coordinate transformation. Then, the homography matrix required for horizontal image correction is obtained, along with the constraint equation satisfying the inclinometer-camera system requirements. Finally, the assembly error of the inclinometer is calibrated by the optimization function. Experimental results show that the inclinometer assembly error can be calibrated only by using the inclination angle information in conjunction with plumb lines in the scenario. Perturbation simulation and practical experiments using MATLAB indicate the feasibility of the proposed method. The inclined image can be horizontally corrected by the homography matrix obtained during the calculation of the inclinometer assembly error, as well.

Keywords:

photoelectric measurement; inclinometer assembly error calibration; plumb line; horizontal image correction; Euler algorithm

1. Introduction

The photoelectric measurement system detects and measures objects by placing a visible CCD or an infrared CCD in a fixed site. Due to terrain variations in the shooting range, a variety of inertial sensors or inclinometers are needed to assist attitude calculation [1,2,3,4,5] and image stabilization [6] for the detectors in the system. With the rapid development and wide application of these devices, the calibration of photoelectric measurement systems is becoming a hot research topic both domestically and overseas, as they are widely used in aerospace, navigation [7,8,9,10,11], automotive robots [12,13], and many other fields [14]. Low cost [3,4,5], high precision, small size, low power dissipation, high overload and high reliability are the directions in which the development of photoelectric measurement systems is focused [15,16,17,18].

Detectors in photoelectric measurement systems should have high angular resolution, which increases the measurement accuracy. During the process of calculating the camera attitude information, the assembly error of the inertial sensor or inclinometer [10] and other system interference factors should be considered and calibrated first, and then combined into the calculation to obtain the desired system precision [15,19].

On the one hand, the inclinometer is a measuring implement used for small-angle measurement [20]. It can be fixed in photoelectric measurement systems to obtain attitude changes relative to the system’s horizontal plane [21]. The real position of the object to be measured in the three-dimensional space coordinate system can be calculated by modeling the variables obtained from the inclinometer and combining them with image processing methods [10,22].

However, the inclinometer can only measure the angle between the assembly plane and the horizontal plane, and cannot acquire complete system attitude information. Thus, in order to perform assembly error calibration, inertial sensors are often needed for assistance. Nevertheless, the drawback is that the cumulative errors of the inertial sensor itself will affect the accuracy of the system’s assembly error calibration [23]. Additionally, the cost of high-precision inertial sensors or gyroscopes is very high, and they are not convenient for system integration.

On the other hand, with regard to the field of photography [14] and image processing in computer vision, in order to meet some demands while taking photos—for example, taking panoramic [24] or distant-view pictures—the camera needs to remain a fixed degree. Additionally, when using functions based on multiple exposures or bracketed exposures for shooting, the pictures will not overlap if the levels are not the same for two exposures, creating ghosting images. Furthermore, sometimes, objects in the image are inclined due to shaking of the camera, or other factors while capturing. For example, some buildings are not perpendicular to the ground, edges of the door and wall are not perpendicular to the floor, etc. These images will not look correct if horizontal correction is not performed. Therefore, it is necessary to pay attention to the level of the camera. Through the use of an inclinometer in the photoelectric system, the image can be horizontally corrected, and the design of the picture can be optimized.

In this paper, a new method for inclinometer assembly error calibration and horizontal image correction is proposed based on plumb lines in the scenario. During the calibration, only images with plumb lines captured by the camera and inclination data from the inclinometer are needed. This paper firstly analyzes the expression of the inclinometer assembly error in photoelectric measurement systems, and then establishes the optimization function of the inclinometer assembly error according to the principle that “the plumb lines of edges of constructions in the real world should become vertical lines on the image plane after horizontal correction by the inclinometer”. Finally, the image is horizontally corrected to verify the accuracy of the calibration result, and calibration errors are discussed through simulation and practical experiments.

The structure of this paper is as follows. Section 2 provides the related work on applications of the camera and the inclinometer, as well as the contributions made by the proposed method. Section 3 explains inclinometer assembly error in photoelectric measurement systems. Section 4 introduces the proposed calibration and horizontal image correction method based on plumb lines. Section 5 shows the experimental results and analyzes the system errors. Section 6 is the conclusion.

2. Related Work and Motivation

Much progress has been witnessed in recent years in inertial and inclinometer calibration, and their applications in various areas.

Merckel [25], et al. described a method for finding the pose of an object from a single image using an inclinometer attached to the camera to reduce the number of unknown parameters during the exact pose calculation, and efficiently compute the pose by using a classical iterative optimization method.

Chang [26], et al. introduced a four-parameter mathematical computing model for the state parameters of the inclinometer, and analyzed various calculation models for the state parameters of the inclinometer.

Hirata [27], et al. presented a method for measuring the thoracic kyphosis angle in an upright standing position with a digital inclinometer and a digital camera. Results showed that the thoracic kyphosis angle of the digital inclinometer was significantly correlated with the angle derived from the digital image analysis, which suggested that a digital inclinometer is an instrument that is simple and easy to use for measuring the thoracic kyphosis angle in the clinic.

In Ref [10], Fabio, et al. proposed a method of using the altitude of celestial bodies to determine position in unknown territory. The experimental setup was similar to that used in our paper, consisting of a camera, a digital inclinometer, neutral density filters and an adjustable platform. The method obtains the position on Earth based on the principle that “a set of circles of equal altitude can be intersected to yield viewer position”. The measurements are processed by non-linear least-squares optimization, replacing the tables used by mariners [28], and which comprises one of the biggest innovations of this method. This paper also analyzed system calibration, providing much inspiration with regard to error analysis.

Motivated by the currently available methods, a new method for inclinometer assembly error calibration and horizontal image correction is proposed based on plumb lines in the scenario. The major contributions of the method proposed in this paper can be summarized as follows.

(1): Based on the principle that “the plumb lines of edges of constructions in the real world should become vertical lines on the image plane after horizontal correction of the attitude of the camera by the inclinometer”, only the angle information of the different attitudes obtained by the inclinometer and the plumb lines in the acquired images are needed to calibrate the inclinometer assembly error matrix in the photoelectric system, which is fast and easy.
(2): The inclinometer assembly error matrix expression in photoelectric systems is analyzed in this paper, and an optimization function to achieve the optimal solution for the assembly error matrix by minimizing the Sum of Squared Residuals (SSR) is established.
(3): A captured image with an arbitrary inclination angle can be horizontally corrected after the system calibration in order to test the correctness of the calibration result.
(4): Factors affecting the accuracy of the calibration results are analyzed by means of a simulation perturbation experiment and a practical experiment, which show sufficient accuracy for the proposed method.
(5): The experimental setup is simple to implement. The calibration process is easily operated. The experimental results are stable and effective.

3. Inclinometer Assembly Error

For some photoelectric measurement systems that employ inclinometers and cameras, the inclinometer coordinate system and the camera coordinate system cannot be completely overlapped during the practical installation process. Figure 1 shows an example of these two coordinate systems, where

C - X_{c} Y_{c} Z_{c}

O_{i} - X_{i} Y_{i} Z_{i}

and

c - x y

represent the camera coordinate system, the inclinometer coordinate system, and the image plane

Π

, respectively.

The transformation relationship between them can be expressed as

X_{i} = R_{i c} X_{c} + t_{i c},

(1)

where

X_{c}

and

X_{i}

represent the coordinates of the spatial point of the camera and the inclinometer coordinate system, respectively.

R_{i c}

and

t_{i c}

represent the matrices of rotation and translation from the camera to the inclinometer coordinate system, respectively.

In practice, the inclinometer is generally placed close to the camera in photoelectric measurement systems, while the distance between the measurement system and the scenario to be measured is greater than the translation distance

t_{i c}

between the camera and the inclinometer. Thus,

t_{i c}

can be ignored [29]. In this paper, we define the rotation matrix

R_{i c}

from the camera coordinate system to the inclinometer coordinate system as the inclinometer assembly error, and we are aiming to calibrate this matrix by using the information in the scenario.

4. Inclinometer Assembly Error Calibration and Horizontal Image Correction

The inclinometer can only measure angles between two axes (

x

-axis and

y

-axis) and the horizontal plane. It cannot obtain the yaw angle value between the measurement system and the north direction. Thus, this degree of freedom should be avoided during calibration. In Figure 2,

o_{1} - x_{1} y_{1}

and

o_{2} - x_{2} y_{2}

represent camera coordinate systems that are in different rotation statuses, and

C

is the optic center of the camera. When the camera is placed horizontally, no matter how the camera’s orientation changes, the plumb line

A B

in the real scenario should be a vertical line

A_{1} B_{1}

A_{2} B_{2}

on the image plane, which means that points on the plumb line

A B

should have the same

x

coordinates (

x_{A_{1}} = x_{B_{1}}, x_{A_{2}} = x_{B_{2}}

) in the image. Based on this principle, we propose a practical method aiming to calibrate the inclinometer assembly error

R_{i c}

, in order to avoid the limitation of the lack of the yaw angle information.

4.1. Relationship between the Geodetic Coordinate System and the Inclinometer Coordinate System

The relationship between the geodetic coordinate system and the inclinometer coordinate system is shown in Figure 3, where

O - X_{g} Y_{g} Z_{g}

and

O - X_{i} Y_{i} Z_{i}

represent the geodetic coordinate system and the inclinometer coordinate system, respectively. Both of these coordinate systems follow the rules of right-handed coordinate systems.

Rotate the geodetic coordinate system around the

Z_{g}

-axis by

α

degrees to make the

X_{o}

-axis the projection of the

X_{i}

-axis. Define this new coordinate system as an intermediate coordinate system

O - X_{o} Y_{o} Z_{o}

. During rotation, the inclinometer gives two angles

(θ, ϕ)

, which are shown in Figure 3.

According to the geometric relationship, the base vector

X_{i}

in the intermediate coordinate system

O - X_{o} Y_{o} Z_{o}

can be expressed as

X_{i} = {(\cos θ, 0, \sin θ)}^{T} .

(2)

According to the orthogonality between the

Y_{i}

-axis and the

X_{i}

-axis, the base vector

Y_{i}

in the intermediate coordinate system

O - X_{o} Y_{o} Z_{o}

can be expressed as

Y_{i} = {(- \tan θ \sin ϕ, ν, \sin ϕ)}^{T},

(3)

where

ν = \sqrt{1 - \frac{\sin^{2} ϕ}{\cos^{2} θ}} .

(4)

Then, the base vector

Z_{i}

Z_{i} = X_{i} \times Y_{i} = {(- ν \sin θ, - \frac{\sin ϕ}{\cos θ}, ν \cos θ)}^{T} .

(5)

The direction cosine matrix

R_{g i}

from the inclinometer coordinate system to the geodetic coordinate system can be represented as

R_{g i} = R_{z} (α) R_{i} (θ, ϕ) = [\begin{matrix} \cos α & - \sin α & 0 \\ \sin α & \cos α & 0 \\ 0 & 0 & 1 \end{matrix}] [\begin{matrix} \cos θ & - \tan θ \sin ϕ & - ν \sin θ \\ 0 & ν & - \frac{\sin ϕ}{\cos θ} \\ \sin θ & \sin ϕ & ν \cos θ \end{matrix}],

(6)

where

α

can be interpreted as the angle between the

Y_{i}

-axis of the inclinometer coordinate system and the north direction. Changing

α

will not affect the value of the inclinometer.

4.2. Horizontal Image Correction Using the Inclinometer

Using the angle information given by the inclinometer, combined with the coordinate transformation relationship, an image taken by the camera in any attitude can be corrected to an image taken with a horizontally placed camera. Figure 4 is the flow chart depicting horizontal image correction using an inclinometer, and Algorithm 1 shows the specific steps of horizontal image correction.

Algorithm 1: Horizontal image correction using the inclinometer.

Rotate the camera coordinate system to the inclinometer coordinate system using the rotation matrix $R_{i c}$ in Equation (1);
Rotate the inclinometer coordinate system to the geodetic coordinate system according to the direction cosine matrix $R_{g i}$ in Equation (6);
Rotate the geodetic coordinate system to the horizontal camera coordinate system by rotating 90° around the $X_{g}$ -axis, the rotation matrix $R_{h}$ can be indicated as

$R_{h} = [\begin{matrix} 1 & 0 & 0 \\ 0 & \cos 90^{\circ} & - \sin 90^{\circ} \\ 0 & \sin 90^{\circ} & \cos 90^{\circ} \end{matrix}] = [\begin{matrix} 1 & 0 & 0 \\ 0 & 0 & - 1 \\ 0 & 1 & 0 \end{matrix}] .$

(7)
According to homography matrix theory in multiple-view geometry [9], the homography matrix $H_{12}$ between two images can be simplified as

$H_{12} = K R_{12} K^{- 1},$

(8)

where $K$ is the intrinsic parameter matrix of the camera, which can be expressed as

$K = [\begin{matrix} f_{x} & 0 & c_{x} \\ 0 & f_{y} & c_{y} \\ 0 & 0 & 1 \end{matrix}],$

(9)

where $f_{x}$ and $f_{y}$ are focal lengths in $x$ and $y$ directions, and $c_{x}$ and $c_{y}$ are principal points.
$R_{12}$ is the rotation matrix from image 1 to image 2

$R_{12} = R_{h} R_{g i} R_{i c} = R_{h} R_{z} (α) R_{i} (θ, ϕ) R_{i c} .$

(10)

Substitute Equation (10) into Equation (8); the homography matrix $H_{12}$ can be written as

$H_{12} = K R_{12} K^{- 1} = K R_{h} R_{z} (α) R_{i} (θ, ϕ) R_{i c} K^{- 1} .$

(11)

Substitute Equations (7) and (9) into Equation (11); we have

$H_{12} = [\begin{matrix} f_{x} & 0 & c_{x} \\ 0 & f_{y} & c_{y} \\ 0 & 0 & 1 \end{matrix}] [\begin{matrix} 1 & 0 & 0 \\ 0 & 0 & - 1 \\ 0 & 1 & 0 \end{matrix}] [\begin{matrix} \cos α & - \sin α & 0 \\ \sin α & \cos α & 0 \\ 0 & 0 & 1 \end{matrix}] R_{i} (θ, ϕ) R_{i c} K^{- 1} .$

(12)

$= [\begin{matrix} f_{x} \cos α + c_{x} \sin α & - f_{x} \sin α + c_{x} \cos α & 0 \\ c_{y} \sin α & c_{y} \cos α & - f_{y} \\ \sin α & \cos α & 0 \end{matrix}] R_{i} (θ, ϕ) R_{i c} K^{- 1} .$

(13)

Finally, the horizontal corrected image can be obtained by multiplying the homography matrix $H_{12}$ with the original image.

4.3. Inclinometer Assembly Error Calibration Based on Plumb Lines

Select two points

A (x_{1}, y_{1})

and

B (x_{2}, y_{2})

on the line

L

in the image captured by the camera in any attitude, define

A^{'} (x_{1}^{'}, y_{1}^{'})

and

B^{'} (x_{2}^{'}, y_{2}^{'})

as image coordinates on the vertical line

L^{'}

after the homography transformation. Figure 5 shows a schematic diagram of such a situation. The plumb lines in the scenario should be vertical lines following horizontal image correction using Algorithm 1.

Based on the homography matrix theory [30], we have

{\begin{matrix} s_{1} [\begin{matrix} x_{1}^{'} \\ y_{1}^{'} \\ 1 \end{matrix}] = H_{12} [\begin{matrix} x_{1} \\ y_{1} \\ 1 \end{matrix}] \\ s_{2} [\begin{matrix} x_{2}^{'} \\ y_{2}^{'} \\ 1 \end{matrix}] = H_{12} [\begin{matrix} x_{2} \\ y_{2} \\ 1 \end{matrix}] \end{matrix},

(14)

where

s_{1}

and

s_{2}

are scale factors of homogeneous coordinates.

Let

{\begin{matrix} [\begin{matrix} X_{1} \\ Y_{1} \\ Z_{1} \end{matrix}] = R_{i} (θ, ϕ) R_{i c} K^{- 1} [\begin{matrix} x_{1} \\ y_{1} \\ 1 \end{matrix}] \\ [\begin{matrix} X_{2} \\ Y_{2} \\ Z_{2} \end{matrix}] = R_{i} (θ, ϕ) R_{i c} K^{- 1} [\begin{matrix} x_{2} \\ y_{2} \\ 1 \end{matrix}] \end{matrix} .

(15)

Substitute Equation (13), (15) into Equation (14), and we have

{\begin{matrix} s_{1} [\begin{matrix} x_{1}^{'} \\ y_{1}^{'} \\ 1 \end{matrix}] = [\begin{matrix} f_{x} \cos α + c_{x} \sin α & - f_{x} \sin α + c_{x} \cos α & 0 \\ c_{y} \sin α & c_{y} \cos α & - f_{y} \\ \sin α & \cos α & 0 \end{matrix}] [\begin{matrix} X_{1} \\ Y_{1} \\ Z_{1} \end{matrix}] \\ s_{2} [\begin{matrix} x_{2}^{'} \\ y_{2}^{'} \\ 1 \end{matrix}] = [\begin{matrix} f_{x} \cos α + c_{x} \sin α & - f_{x} \sin α + c_{x} \cos α & 0 \\ c_{y} \sin α & c_{y} \cos α & - f_{y} \\ \sin α & \cos α & 0 \end{matrix}] [\begin{matrix} X_{2} \\ Y_{2} \\ Z_{2} \end{matrix}] \end{matrix} .

(16)

Expand Equation (16), and since

x_{1}^{'} = x_{2}^{'}

, finally we have

X_{1} Y_{2} - X_{2} Y_{1} = 0 .

(17)

From Equation (17), we can see that angle

α

—defined in Equation (6)—can be eliminated during the deduction, which verifies that the changing of

α

will not affect the value of the inclinometer. No matter what the value

α

is, the plumb line will always be a vertical line following the horizontal image correction.

In terms of actual measurement error, Equation (17) will not be strictly established. Thus, a method that minimizes the Sum of Squared Residuals (SSR) [31] is used to calculate the inclinometer assembly error

R_{i c}

. By rotating the photoelectric measurement system, multiple inclinometer values

(θ_{i}, ϕ_{i}), i = 1, 2, \dots, n

are recorded. At the same time, relative images for different inclinometer angles are captured. The Hough line-detection method [32] is applied to detect straight lines that are projections of plumb lines in the same scenario. The coordinates of the lines are recorded as

(x_{1}^{i}, y_{1}^{i}), (x_{2}^{i}, y_{2}^{i}), i = 1, 2, \dots, n

. The optimization function can be formed as

\min_{R_{i c}} \frac{1}{2} \sum_{i = 1}^{n} {(X_{1}^{i} Y_{2}^{i} - X_{2}^{i} Y_{1}^{i})}^{2}, s . t . R_{i c}^{T} R_{i c} = I,

(18)

where

I

is a

3 \times 3

identity matrix.

In the optimization problem in Equation (18), the objective function is the Sum of Squared Residuals, and the restricted condition is the orthogonality constraint of the inclinometer assembly error matrix

R_{i c}

. Therefore, this is a nonlinear programming problem with constraints, and the matrix to be optimized,

R_{i c}

, has three degrees of freedom. In order to increase the calibration precision, it should satisfy

n \geq 3

[33].

5. Experimental Results and Analyses

This section explains the experimental investigation of the inclinometer assembly error calibration in two different photoelectric measurement systems with two different cameras and inclinometers, as well as the results of the horizontal corrected image. Additionally, the calibration error is analyzed by means of both a perturbation simulation experiment [34] and a practical experiment. We also compare the proposed method with other methods.

5.1. Photoelectric Measurement System

The camera in the first photoelectric measurement system (defined as System 1) is a high-resolution industrial CCD VC-12MC-65 made by Vieworks Company. The lens is a Distagon 35 mm prime lens made by the Zeiss Company, which has a broad perspective, and negligible radial distortion. The camera parameters are shown in Table 1.

The dual-axis inclinometer in System 1 is made by Xi’an Sicong Chuangwei Optoelectronic Co. Ltd., which has a 0.05° angular accuracy. The inclination data acquired are all 16-bit signed numbers. The angle data in degrees can be calculated by multiplying by a scale factor of

1 / 3600

The camera and the inclinometer are fixed on an adjustable platform. A diagram of System 1 is shown in Figure 6.

From Figure 6, we can see that the

X_{i}

-axis and the

Y_{i}

-axis of the inclinometer are orthogonal along the horizontal direction, while the

Z_{i}

-axis points to the zenith. The

X_{c}

-axis and the

Y_{c}

-axis of the camera are parallel to the

x

-axis and the

y

-axis of the image plane, respectively, while the

Z_{c}

-axis is parallel to the optic axis. Thus, a probable initial value for the nonlinear programming problem in Equation (18) for the inclinometer assembly error

{\hat{R}}_{i c}

of System 1 could be

{\hat{R}}_{i c} = [\begin{matrix} 1 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & - 1 & 0 \end{matrix}] .

(19)

In the second photoelectric measurement system (defined as System 2), we use a Basler acA2040-25gm camera with the 8 mm fish-eye lens. This lens has significant radial distortion that needs to be corrected before the calibration. The SCA126T dual-axis inclinometer is from China Shenzhen Rion Technology Co., Ltd., with a 0.01° angular resolution. The camera parameters are shown in Table 2, and the photoelectric measurement system (System 2) is shown in Figure 7.

From Figure 7, we can also obtain one probable initial value of the nonlinear programming problem in Equation (18) for the inclinometer assembly error

{\hat{R}}_{i c}

of System 2:

{\hat{R}}_{i c} = [\begin{matrix} 1 & 0 & 0 \\ 0 & 0 & - 1 \\ 0 & 1 & 0 \end{matrix}] .

(20)

5.2.Calibration of Camera Intrinsic Parameter and Estimation of Lens Radial Distortion Parameter

To solve the nonlinear programming problem in Equation (18), the camera intrinsic parameter matrix

K

and the lens distortion parameter [35] are necessary. A commonly used lens distortion model can be written as

{\begin{cases} x_{u} - x_{0} = \frac{x_{d} - x_{0}}{1 + λ_{1} r_{d}^{2}} \\ y_{u} - y_{0} = \frac{y_{d} - y_{0}}{1 + λ_{1} r_{d}^{2}} \\ r_{d}^{2} = {(x_{d} - x_{0})}^{2} + α^{2} {(y_{d} - y_{0})}^{2} \end{cases},

(21)

where

r_{d}

is the distance from the distorted point

(x_{d}, y_{d})

to the distortion center

(x_{0}, y_{0})

λ_{1}

is the radial distortion parameter, and

α

is the pixel aspect ratio. In this paper, we only consider the first term of the radial distortion, since this is enough for tasks in computer vision, according to Tsai [36].

We use the classical calibration method proposed by Zhang [37] to obtain the intrinsic parameter matrix and the radial distortion parameter. Figure 8 shows the checkerboard used during camera calibration.

The intrinsic parameter matrices

K_{1}

and

K_{2}

of the two cameras are

K_{1} = [\begin{matrix} 6428.2619 & 0 & 2061.0438 \\ 0 & 6425.1360 & 1546.1847 \\ 0 & 0 & 1 \end{matrix}]

(22)

for System 1, with 0.1013 pixels mean re-projection error [34]; and

K_{2} = [\begin{matrix} 1343.8891 & 0 & 984.6998 \\ 0 & 1345.1382 & 1051.7659 \\ 0 & 0 & 1 \end{matrix}]

(23)

for System 2, with 0.0966 pixels mean re-projection error.

The radial distortion parameters are

k_{1} = - 0.0994

(24)

for System 1; and

k_{2} = - 0.3487

(25)

for System 2.

Images captured by System 2 will be corrected for radial distortion using the parameters in Equations (23) and (25) before Hough line detection, to reduce the coordinate detection error caused by the fish-eye lens distortion.

5.3. Experimental Data Measurement

During the experiment, changing the attitude of the photoelectric measurement system, taking images of each scenario

I_{i} (i = 1, 2, \dots, n)

, and recording the inclination data

(θ_{i}, ϕ_{i}), i = 1, 2, \dots, n

all takes place at the same time. When saving camera images, we make sure that there are obvious building edges in the image. Table 3a shows 12 pairs of the inclination data in degrees for System 1, and Table 3b shows the data for System 2.

The Hough line detection method is applied to detect straight lines that are projections of plumb lines in the scenario. Other lines that are not projections of plumb lines are removed. Additionally, in order to ensure precision, only lines longer than

D

pixels are retained (in our experiments,

D = 200

pixels in System 1, and

D = 100

pixels in System 2).

5.4. Inclinometer Assembly Error Calculation and Horizontal Image Correction

Using Equation (18), inclinometer assembly error matrices are calculated as

R_{i c} = [\begin{matrix} 0.9996 & - 0.0014 & 0.0297 \\ - 0.0296 & 0.0127 & 0.9995 \\ - 0.0017 & - 0.9999 & 0.0127 \end{matrix}] .

(26)

for System 1; and

R_{i c} = [\begin{matrix} 0.9990 & 0.0197 & 0.0390 \\ 0.0396 & - 0.0317 & - 0.9987 \\ - 0.0184 & 0.9993 & - 0.0325 \end{matrix}] .

(27)

for System 2.

For the original image, homography matrices

H_{12}

are calculated with the inclination data in Table 3 by substituting Equation (22) or (23) and (26) or (27) into Equation (11):

H_{12} = [\begin{matrix} 0.9837 & 0.0970 & 100.0557 \\ - 0.1194 & 0.9824 & 521.4895 \\ - 0.0000 & - 0.0000 & 1.0186 \end{matrix}] .

(28)

for System 1; and

H_{12} = [\begin{matrix} 0.9551 & 0.1814 & - 84.4734 \\ - 0.2094 & 0.9826 & 222.5300 \\ - 0.0000 & - 0.0000 & 1.0274 \end{matrix}] .

(29)

for System 2.

Finally, the horizontally corrected image is obtained by transformation using the corresponding homography matrix

H_{12}

Figure 9a shows the original image,

I_{3}

, in which nine lines were detected in System 1; Figure 9b shows the horizontally corrected image,

I_{3_c o r r e c t e d}

, which can be seen to be horizontally corrected, as the edges of the buildings are vertical in the corrected image. The horizontally corrected image looks better than the original one.

Figure 10a–c shows the horizontal correction results of image

I_{6}

using System 2. The original image,

I_{6}

, has serious radial distortion, which is apparent due to the plumb lines of the building having turned into curves (Figure 10a). Firstly, the image was undistorted,

I_{6_u n d i s t o r t e d}

(Figure 10b), to reduce the error caused by coordinate detection, and then horizontally corrected,

I_{6_c o r r e c t e d}

(Figure 10c).

Table 4 gives the corresponding coordinates and angle for every detected line in one image using System 2. After the horizontal correction, coordinates on the same plumb line in image

I_{6_c o r r e c t e d}

all have the same

x

-axis coordinate except for one, obtained by testing coordinates on the same line, and the corresponding angles with

x

direction are all 90°.

5.5. System Error Analyses

The inclinometer assembly error calibration result

R_{i c}

in Equation (18) is deduced under ideal conditions. In real-world conditions, due to error arising during measurement and image processing, the calibration results will be affected by image noise, camera intrinsic parameter calibration errors, inclinometer angle reading errors, coordinate detection errors, etc. In order to verify the feasibility of the proposed method, one simulation experiment and one practical experiment are performed to analyze the system error.

5.5.1. Error Analyses by Simulation Experiment

In the simulated perturbation experiment, standard Gaussian random noise with a zero mean and one-pixel standard deviation is added to the endpoints of detected lines to simulate the coordinate detection error caused by image noise, the radial distortion of the lens, and the inaccuracy of the Hough line detection method.

In order to analyze the system error, the inclinometer assembly error matrix

R_{i c}

is decomposed into Euler angles (yaw, pitch, and roll) [38]. The pseudocode of the perturbation experiment is shown in Algorithm 2.

Algorithm 2: Pseudocode of the perturbation simulation experiment.

For

i = 1 : 5

(

i

is the number of groups in the experiment)
For

j = 1 : 5

(

j

is the number of images taken in each group)
For

k = 1 : 10

(

k

is the duration over which Gaussian noise was added)
(I) Obtain image

I_{j}

, and record values of the inclinometer

(θ_{j}, ϕ_{j})

.
(II) Detect plumb lines in the image

I_{j}

using the Hough line detection method.
(III) Add Gaussian noise to the endpoints of detected lines

(x_{n}^{j, k}, y_{n}^{j, k})

.
End
End
(IV) Calculate the inclinometer assembly error

R_{i c}^{i}

, and decompose the matrix into Euler angles.
End
(V) Calculate the average of the Euler angles as the final assembly error for each group.
(VI) Calculate the mean and the standard deviation of the Euler angle.

Figure 11 shows Euler angles for each experiment. In addition, Table 5 shows the mean and the standard deviation of the Euler angles.

From Figure 11 and Table 5, we can see that the mean of the Euler angle with the Gaussian white noise illustrates the angle between the inclinometer and the camera coordinate systems, while the standard deviation with the Gaussian white noise illustrates the calibration precision, indicating the feasibility of the proposed method.

5.5.2. Error Analyses by Practical Experiment

In the practical experiment, the calibration precision is tested by horizontally correcting images using the calibrated system. The pseudocode of the practical experiment is shown in Algorithm 3.

Bulleted lists look like this:

Algorithm 3: Pseudocode for the practical experiment.

(I) Simultaneously capture the images $I_{i} (i = 1, 2, \dots, n)$ and record the corresponding inclination data $(θ_{i}, ϕ_{i}), i = 1, 2, \dots, n$ .
(II) Detect plumb lines in the captured image $I_{i} (i = 1, 2, \dots, N, N = \frac{2}{3} n)$ using the Hough line detection method, and extract the endpoints of each line $(x_{i}, y_{i}), (i = 1, 2, \dots, N)$ .
(III) Calculate the inclinometer assembly error $R_{i c}$ using the $N$ images and corresponding information.
(IV) Horizontally correct the $n - N$ retained images using the calibrated matrix $R_{i c}$ .
(V) Detect the endpoints of the horizontal corrected images and calculate the angle with the $x$ -axis on the same line.

We first captured 9 images using the camera and inclinometer in System 1, and recorded the corresponding inclination data. Then, we used 6 of the 9 images to calibrate the inclinometer assembly error

R_{i c}

. Finally, we horizontally corrected the remaining 3 images using the calibrated

R_{i c}

. Figure 12 shows the correction results with the lines detected on them, which can be seen to be horizontally corrected, as the edges of the building are vertical in the corrected image. Table 6 provides the coordinates of the endpoints and the angle with the x direction of each line, from which we can see that most of the coordinates on the plumb line have the same x-axis coordinate after horizontal correction. It also shows that angles with the direction are 90°, which means that the calibrated result is of sufficient accuracy.

5.6. Comparison with Other Methods

The precision of horizontal image correction reflects the accuracy of the calibration of inclinometer assembly error in our photoelectric measurement system, since the homography matrix used for horizontal image correction is related to the inclinometer assembly error in the proposed method. In this section, we compare the proposed horizontal image correction method with three other classical image processing methods—the slant correction method based on Principal Component Analysis (PCA) [39], the image tilt correction method based on the Hough line detection method [40], and the horizontal image correction method based on the Radon method [41]—to evaluate our proposed method.

We compare the above four methods in terms of the aspects of computation time and horizontal image correction error. In our experiment, the horizontal image correction error is defined as

E = \frac{1}{k N} \sum_{j = 1}^{k} \sum_{i = 1}^{n} | x_{i + 1}^{j} - x_{i}^{j} |,

(30)

where

N

is the number of plumb lines detected in the corrected image,

k

is the number of experiments,

n = 2 N

is the number of endpoints on the detected lines, and

x_{i + 1}^{j}

and

x_{i}^{j}

represent the starting point and the ending point, respectively, of the same detected line.

The comparative experiment is carried out using System 1. Figure 13 shows the horizontal correction results with lines detected using the four methods. Table 7 shows the comparison of the results, with the optimal results in boldface.

From the comparison of the results, we can see that all the methods are able to horizontally correct the image by rotating using the inclination data, with the proposed method using the calibrated inclinometer assembly matrix to obtain the homography matrix for horizontal correction, while the other three methods use image processing methods without calibrating the inclinometer assembly error. The PCA method uses the PCA algorithm to acquire the correcting geometrical transform matrix by covariance matrix calculation and eigenvalue decomposition. The Hough method uses the horizontal lines by finding the most prominent line or edge in the image. However, it traverses the entire image for image interpolation, resulting in a longer calculation time. The Radon method is based on the rotation and projection transformation [42,43], which uses the distance from the image center pixel to the image boundary to calculate the image inclination angle.

Therefore, compared with these methods, the proposed method has the advantages of calibrating the inclinometer assembly error of photoelectric measurement systems and horizontally correcting the tilt image at the same time. The horizontal image correction error is very small compared with other methods, while the computation time is not increased, which shows the feasibility and accuracy of the proposed method.

6. Conclusions

This paper analyzes the expression of inclinometer assembly error in photoelectric systems. Aiming to solve the problem of the lack of yaw angle information, an inclinometer assembly error calibration method is proposed that utilizes the plumb lines in the scenario. An optimization function for calibration by deriving transformation relationships between different coordinate systems is established, and the constraint equation satisfies the inclinometer-camera system.

The experimental results of the simulated photoelectric measurement systems show that the assembly error can be calibrated only by using the inclinometer angle information in conjunction with the plumb lines in the scenario. The perturbation experiment for the input parameter indicates the feasibility of the proposed method, which provides a theoretical basis for future work on the correction of and compensation for inclinometer assembly error, as well as improvements in accuracy and precision for photoelectric measurement systems.

Future work may include the decreasing of calibration error, and the application of the proposed method in the photoelectric detection system.

Acknowledgments

This research was funded by the National Natural Science Foundation of China (61701233).

Author Contributions

Xiaofang Kong and Pengcheng Wang proposed the method and designed the experiments; Xiaofang Kong, Pengcheng Wang and Jiajie Wang performed the experiments; Xiaofang Kong and Pengcheng Wang analyzed the data; Qian Chen, Guohua Gu, Weixian Qian, Kan Ren and Xiaotao Miao contributed analysis tools; Xiaofang Kong and Pengcheng Wang wrote the paper.

Conflicts of Interest

The authors declare no conflict of interest.

References

Ma, D.M.; Shiau, J.K.; Wang, I.C.; Lin, Y.H. Attitude Determination Using a MEMS-Based Flight Information Measurement Unit. Sensors 2012, 12, 1–23. [Google Scholar] [CrossRef] [PubMed]
Nadarajah, N.; Teunissen, P.J.G.; Raziq, N. BeiDou Inter-Satellite-Type Bias Evaluation and Calibration for Mixed Receiver Attitude Determination. Sensors 2013, 13, 9435–9463. [Google Scholar] [CrossRef] [PubMed]
Cong, L.; Li, E.; Qin, H.; Voon Ling, K.; Xue, R. A Performance Improvement Method for Low-Cost Land Vehicle GPS/MEMS-INS Attitude Determination. Sensors 2015, 15, 5722–5746. [Google Scholar] [CrossRef] [PubMed]
Antonio, A.; Mark, P.; Giovanni, P. Benefits of Combined GPS/GLONASS with Low-Cost MEMS IMUs for Vehicular Urban Navigation. Sensors 2012, 12, 5134–5158. [Google Scholar]
Kai-Wei, C.; Trung, D.T.; Liao, J.K.; Lai, Y.C.; Chang, C.C.; Cai, J.M.; Huang, S.C. On-Line Smoothing for an Integrated Navigation System with Low-Cost MEMS Inertial Sensors. Sensors 2012, 12, 17372–17389. [Google Scholar]
Hao, Q.; Cheng, X.; Kang, J.; Jiang, Y. An Image Stabilization Optical System Using Deformable Freeform Mirrors. Sensors 2015, 15, 1736–1749. [Google Scholar] [CrossRef] [PubMed]
Sun, T.; Xing, F.; You, Z. Optical System Error Analysis and Calibration Method of High-Accuracy Star Trackers. Sensors 2013, 13, 4598–4623. [Google Scholar] [CrossRef] [PubMed]
Gao, W.; Zhang, Y.; Wang, J. Research on Initial Alignment and Self-Calibration of Rotary Strapdown Inertial Navigation Systems. Sensors 2015, 15, 3154–3171. [Google Scholar] [CrossRef] [PubMed]
Xian, Z.; Hu, X.; Lian, J.; Zhang, L.; Cao, J.; Wang, Y.; Ma, T. A novel angle computation and calibration algorithm of bio-inspired sky-light polarization navigation sensor. Sensors 2014, 14, 17068–17088. [Google Scholar] [CrossRef] [PubMed]
Cozman, F.; Krotkov, E. Robot localization using a computer vision sextant. In Proceedings of the 1995 IEEE International Conference on Robotics and Automation, Nagoya, Japan, 21–27 May 1995; Volume 1, pp. 106–111. [Google Scholar]
Barbour, N.M. Inertial navigation sensors. Available online: http://www.dtic.mil/dtic/tr/fulltext/u2/a581016.pdf (accessed on 6 November 2017).
Olivares, A.; Ramírez, J.; Górriz, J.M.; Olivares, G.; Damas, M. Detection of (in) activity Periods in Human Body Motion Using Inertial Sensors: A Comparative Study. Sensors 2012, 12, 5791–5814. [Google Scholar] [CrossRef] [PubMed]
Beṣdok, E. 3D Vision by Using Calibration Pattern with Inertial Sensor and RBF Neural Networks. Sensors 2009, 9, 4572–4585. [Google Scholar] [CrossRef] [PubMed]
Balletti, C.; Guerra, F.; Tsioukas, V.; Vernier, P. Calibration of action cameras for photogrammetric purposes. Sensors 2014, 14, 17471–17490. [Google Scholar] [CrossRef] [PubMed]
Niu, X.; You, L.; Zhang, H.; Wang, Q.; Ban, Y. Fast Thermal Calibration of Low-Grade Inertial Sensors and Inertial Measurement Units. Sensors 2013, 13, 12192–12217. [Google Scholar] [CrossRef] [PubMed]
Ding, J. Research on Attitude Algorithm Based on Micro Inertial Sensors. Master’s Thesis, Shanghai Jiao Tong University, Shanghai, China, 2013. [Google Scholar]
Wu, Z.; Sun, Z.; Zhang, W.; Chen, Q. A novel approach for attitude estimation using MEMS inertial sensors. In Proceedings of the 2014 IEEE SENSORS, Valencia, Spain, 2–5 November 2014; pp. 1022–1025. [Google Scholar]
Liu, X. The Attitude Test Algorithm Based on MEMS Multidimensional Inertial Sensors. Master’s Thesis, Harbin Engineering University, Harbin, China, 2013. [Google Scholar]
Bonnet, S.; Bassompierre, C.; Godin, C.; Lesecq, S.; Barraud, A. Calibration methods for inertial and magnetic sensors. Sensors Actuators A Phys. 2009, 156, 302–311. [Google Scholar] [CrossRef]
Hedrich, F.; Frech, J.; Auber, J.; Sandmaier, H.; Wimmer, W.; Lang, W. Micro-machined inclinometer with high sensitivity and very good stability. Sens. Actuators A Phys. 2002, 97–98, 125–130. [Google Scholar]
Cao, J.; Zhang, L.; Wu, H.; Wang, L.; Miao, L. Analytical approach for measurement of spatial angle with inclination sensor. J. Xian Jiao Tong Univ. 2013, 47, 109–114. [Google Scholar]
Tong, G.; Wang, T.; Wu, Z.; Li, Z.; Chen, T. Application of high accuracy inclinometer to deformation measurement for vehicular platform. Opt. Precis. Eng. 2010, 18, 1347–1353. [Google Scholar]
Lambrecht, S.; Nogueira, S.L.; Bortole, M.; Siqueira, A.A.; Terra, M.H.; Rocon, E.; Pons, J.L. Inertial Sensor Error Reduction through Calibration and Sensor Fusion. Sensors 2016, 16, 235. [Google Scholar] [CrossRef] [PubMed]
Benosman, R.; Kang, S.B. Panoramic Vision, Sensors Theory and Applications. Sensors 2001, 29, 1989–1997. [Google Scholar]
Merckel, L.; Nishida, T. Solution of the perspective-three-point problem: Calculation from video image by using inclinometers attached to the camera. In Proceedings of the International Conference on Industrial, Engineering, and Other Applications of Applied Intelligent Systems, Kyoto, Japan, 26–29 June 2007; Springer: Berlin/Heidelberg, Germany; pp. 324–333. [Google Scholar]
Chang, J. Calculation of state parameters of inclinometer in digital zenith camera. Petrochem. Ind. Technol. 2016, 24, 2325–2331. [Google Scholar]
Hirata, H.; Yagura, C.; Oka, S.; Yoshimura, K.; Hamachi, N.; Tahara, H. Measurement of the Thoracic Kyphosis Angle with a Digital Inclinometer. Rigakuryoho Kagaku 2012, 27, 115–118. [Google Scholar] [CrossRef]
Hobbs, R.R. Marine Navigation, 3rd ed.; Naval Institute Press: Annapolis, MD, USA, 1974. [Google Scholar]
Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision; Cambridge University Press: Cambridge, UK, 2003. [Google Scholar]
Rousso, B.; Avidan, S.; Shashua, A.; Peleg, S. Robust Recovery of Camera Rotation from Three Frames. In Proceedings of the CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 18–20 June 1996; pp. 796–802. [Google Scholar]
Simon, D.; Simon, D.L. Analytic Confusion Matrix Bounds for Fault Detection and Isolation Using a Sum-of-squared-residuals Approach. IEEE Trans. Reliab. 2010, 59, 287–296. [Google Scholar] [CrossRef]
Ballard, D.H. Generalizing the Hough transform to detect arbitrary shapes. Pattern Recognit. 1981, 13, 111–122. [Google Scholar] [CrossRef]
Bertsekas, D.P. Nonlinear programming. J. Oper. Res. Soc. 1997, 48, 334. [Google Scholar] [CrossRef]
Stewart, G.W.; Sun, J. Matrix Perturbation Theory; Academic Press: Cambridge, MA, USA, 1990. [Google Scholar]
Strand, R.; Hayman, E. Correcting Radial Distortion by Circle Fitting. In Proceedings of the British Machine Vision Conference, Oxford, UK, 5–8 September 2005. [Google Scholar]
Tsai, R.Y. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Autom. 2003, 3, 323–344. [Google Scholar] [CrossRef]
Zhang, Z. Flexible camera calibration by viewing a plane from unknown orientations. In Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece, 20–27 September 1999; pp. 666–673. [Google Scholar]
Pio, R.L. Euler angle transformations. IEEE Trans. Autom. Control 1967, 11, 707–715. [Google Scholar] [CrossRef]
Wang, H.; Leng, Y.; Wang, Z.; Wu, X. Application of image correction and bit-plane fusion in generalized PCA based face recognition. Pattern Recognit. Lett. 2007, 28, 2352–2358. [Google Scholar] [CrossRef]
Zafarifar, B.; Weda, H.; With, P.H.N.D. Horizon detection based on sky-color and edge features. In Proceedings of the SPIE—The International Society for Optical Engineering, San Jose, CA, USA, 27–31 January 2008. [Google Scholar]
Chen, L.; Li, X.; Lu, L. Research and modification of license plate tilt correction algorithm. Comput. Mod. 2013, 12, 91–97. [Google Scholar]
Zisserman, A. Geometric Framework for Vision I: Single View and Two-View Geometry; Cambridge University Press: Cambridge, UK, 1997. [Google Scholar]
Criminisi, A.; Reid, I.; Zisserman, A. A plane measuring device. Image Vis. Comput. 1999, 17, 625–634. [Google Scholar] [CrossRef]

Figure 1. An example of the camera and inclinometer coordinate systems.

Figure 2. The plumb line in the real world should be a vertical line on the image plane when the camera is placed horizontally in the photoelectric system.

Figure 3. Relationship between the geodetic coordinate system and the inclinometer coordinate system.

Figure 4. Flow chart of horizontal image correction.

Figure 5. The schematic diagram of one situation.

Figure 6. A photoelectric measurement system (System 1) with a high-resolution industrial camera and a dual-axis inclinometer.

Figure 7. The photoelectric measurement system (System 2) by the Basler acA2040-25gm camera and the SCA126T inclinometer.

Figure 8. The checkerboard used for camera calibration.

Figure 9. Images with detected plumb lines before and after horizontal correction in System 1. (a) The original image

I_{3}

; (b) The horizontally corrected image

I_{3_c o r r e c t e d}

Figure 9. Images with detected plumb lines before and after horizontal correction in System 1. (a) The original image

I_{3}

; (b) The horizontally corrected image

I_{3_c o r r e c t e d}

Figure 10. Images before and after correction using System 2. (a) The original distorted image,

I_{6}

; (b) The undistorted image,

I_{6_u n d i s t o r t e d}

, with detected plumb lines; (c) The horizontally corrected image,

I_{6_c o r r e c t e d}

, with detected plumb lines.

Figure 10. Images before and after correction using System 2. (a) The original distorted image,

I_{6}

; (b) The undistorted image,

I_{6_u n d i s t o r t e d}

, with detected plumb lines; (c) The horizontally corrected image,

I_{6_c o r r e c t e d}

, with detected plumb lines.

Figure 11. Euler angles for each experiment. (a) Yaw; (b) Pitch; (c) Roll.

Figure 12. Images with lines detected before (first row) and after (second row) horizontal correction.

Figure 13. Images with lines detected after horizontal correction using the four methods. (a) Proposed method; (b) PCA method; (c) Hough method; (d) Radon method.

Table 1. Camera parameters of the high-resolution industrial CCD in System 1.

Camera Parameter	Value
Resolution	4090 × 3072
Pixel size	5.5 μm × 5.5 μm
Device dimension	22.5 mm 16.9 mm
Focal length	35 mm
Frame frequency	64 fps
angular resolution	0.16 mrad

Table 2. Camera parameters of the Basler acA2040-25gm camera in System 2.

Camera Parameter	Value
Resolution	2048 × 2048
Pixel size	5.5 μm × 5.5 μm
Device dimension	11.3 mm 11.3 mm
Focal length	8 mm
Frame frequency	25 fps
angular resolution	4 MP

Table 3. The measured inclination data.

(a) Inclination Data in System 1 (°)
No.	$X_{i}$ -axis Inclination	$Y_{i}$ -axis Inclination	No.	$X_{i}$ -axis Inclination	$Y_{i}$ -axis Inclination
1	1°41′21″	−0°26′12″	7	0°44′33″	−0°14′51″
2	7°15′8″	2°48′38″	8	−2°53′53″	−6°11′21″
3	−5°36′24″	−6°37′34″	9	−6°20′5″	−0°41′4″
4	1°14′16″	4°16′53″	10	−6°36′41″	−6°26′12″
5	−0°21′50″	−6°16′36″	11	3°42′48″	−7°30′0″
6	5°7′34″	−3°24′27″	12	3°51′33″	4°59′42″
(b) Inclination Data in System 2 (°)
No.	$X_{i}$ -axis Inclination	$Y_{i}$ -axis Inclination	No.	$X_{i}$ -axis Inclination	$Y_{i}$ -axis Inclination
1	0.01	−0.03	10	−1.53	−5.75
2	−3.15	−1.94	11	−4.54	−6.57
3	−1.53	−1.43	12	6.12	−6.88
4	1.78	−0.20	13	2.18	−5.05
5	2.94	−0.26	14	−0.42	5.77
6	4.09	−0.59	15	3.05	3.25
7	5.75	−0.28	16	−2.00	4.07
8	7.56	−0.57	17	−0.04	2.95
9	−9.36	−1.85	18	0.75	0.89

Table 4. Coordinates and angles of each detected plumb line before and after horizontal correction.

Line No.	Before Horizontal Correction			After Horizontal Correction
Line No.	Starting Point	End Point	Angle (°)	Starting Point	End Point	Angle (°)
1	(1043, 1116)	(1004, 1321)	−76.30	(1247, 334)	(1247, 518)	90
2	(1003, 1326)	(920, 1749)	−78.89	(1392, 133)	(1392, 239)	90
3	(1534, 640)	(1503, 816)	−80.01	(1115, 423)	(1115, 667)	90
4	(855, 1124)	(776, 1531)	−79.02	(1588, 133)	(1588, 402)	90
5	(775, 1536)	(755, 1639)	−79.01	(381, 485)	(382, 599)	89.50
6	(632, 1103)	(564, 1453)	−79.00	(303, 134)	(303, 357)	90
7	(263, 564)	(203, 875)	−79.08	(248, 117)	(248, 235)	90

Table 5. The mean and the standard deviation of the Euler angles (°).

(a) Mean of the Euler Angles
No.	Yaw	Pitch	Roll
1	0.6415	0.2888	−178.0544
2	8.2894	0.4158	−178.0544
3	18.7872	0.5699	179.2778
4	−11.8215	−1.7134	172.4610
5	−18.1659	1.5106	−178.4386
(b) Standard Deviation of the Euler Angles
No.	Yaw	Pitch	Roll
1	0.9574	0.0731	0.5204
2	0.4007	0.1694	0.8484
3	1.3761	0.2176	0.6053
4	0.4516	0.5442	1.8206
5	1.4514	1.0103	1.5745
AVE	0.9276	0.4029	1.0732

Table 6. Coordinates and angles of each detected plumb line before and after horizontal correction.

(a) Image (a)
Line No.	Before Horizontal Correction			After Horizontal Correction
Line No.	Starting Point	End Point	Angle (°)	Starting Point	End Point	Angle (°)
1	(1345, 1043)	(1327, 1169)	-81.87	(1511, 1093)	(1510, 1185)	89.38
2	(1564, 1031)	(1544, 1162)	-81.32	(1457, 1006)	(1457, 1120)	90
3	(1194, 1025)	(1178, 1137)	-80.37	(1737, 1258)	(1737, 1359)	90
4	(1135, 1042)	(1123, 1125)	-81.77	(1533, 1269)	(1533, 1357)	90
5	(1615, 1016)	(1589, 1185)	-81.25	(1735, 1374)	(1735, 1461)	90
6	(1519, 1134)	(1501, 1248)	-81.03	(1127, 1160)	(1127, 1348)	90
(b) Image (b)
Line No.	Before Horizontal Correction			After Horizontal Correction
Line No.	Starting Point	End Point	Angle (°)	Starting Point	End Point	Angle (°)
1	(1646, 940)	(1710, 1348)	81.09	(1827, 1250)	(1827, 1537)	90
2	(1630, 1004)	(1642, 1085)	81.57	(1864, 1108)	(1864, 1246)	90
3	(1643, 1090)	(1663, 1213)	80.76	(1864, 1252)	(1864, 1348)	90
4	(1336, 1063)	(1366, 1254)	81.07	(1912, 1156)	(1912, 1399)	90
5	(1762, 1070)	(1785, 1221)	81.34	(2056, 1319)	(2056, 1566)	90
6	(1787, 1232)	(1807, 1359)	81.05	(1458, 1319)	(1458, 1427)	90
7	(1717, 1138])	(1738, 1272)	81.09	(1801, 1258)	(1801, 1416)	90
8	(1739, 1277)	(1753, 1370)	81.44	(1510, 1374)	(1510, 1470)	90
9	(1801, 1010)	(1836, 1236)	81.19	(1771, 1307)	(1771, 1484)	90
10	(1369, 1058)	(1384, 1153)	81.03	(1487, 1184)	(1487, 1292)	90
(c) Image (c)
Line No.	Before Horizontal Correction			After Horizontal Correction
Line No.	Starting Point	End Point	Angle (°)	Starting Point	End Point	Angle (°)
1	(1941, 1036)	(2017, 1470)	80.07	(2284, 1350)	(2284, 1402)	90
2	(1966, 1032)	(2001, 1232)	80.07	(2285, 1408)	(2285, 1464)	90
3	(2005, 1254)	(2041, 1463)	80.23	(1998, 1296)	(1998, 1352)	90
4	(2173, 998)	(2235, 1353)	80.09	(2027, 1181)	(2027, 1237)	90
5	(1886, 1047)	(1948, 1402)	80.09	(2030, 1353)	(2030, 1409)	90
6	(2050, 1158)	(2092, 1400)	80.15	(2081, 1409)	(2081, 1466)	90
7	(1580, 1179)	(1598, 1282)	80.09	(1675, 1356)	(1675, 1412)	90
8	(1599, 1287)	(1620, 1407)	80.07	(1707, 1298)	(1707, 1350)	90
9	(1878, 1150)	(1902, 1290)	80.27	(1708, 1355)	(1708, 1412)	90
10	(1634, 1166)	(1675, 1401)	80.23	(1774, 1355)	(1774, 1411)	90
11	(1677, 1159)	(1725, 1433)	80.06	(2056, 1410)	(2056, 1466)	90
12	(1934, 1143)	(1967, 1331)	80.04	(2142, 1466)	(2142, 1522)	90

Table 7. Comparison of results of the four methods.

Method	Proposed Method	PCA Method	Hough Method	Radon Method
Computation time (s)	7.6137	35.3295	561.3933	9.3188
Correction error (pixel)	0.50	3.00	0.77	0.91

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Kong, X.; Chen, Q.; Wang, J.; Gu, G.; Wang, P.; Qian, W.; Ren, K.; Miao, X. Inclinometer Assembly Error Calibration and Horizontal Image Correction in Photoelectric Measurement Systems. Sensors 2018, 18, 248. https://doi.org/10.3390/s18010248

AMA Style

Kong X, Chen Q, Wang J, Gu G, Wang P, Qian W, Ren K, Miao X. Inclinometer Assembly Error Calibration and Horizontal Image Correction in Photoelectric Measurement Systems. Sensors. 2018; 18(1):248. https://doi.org/10.3390/s18010248

Chicago/Turabian Style

Kong, Xiaofang, Qian Chen, Jiajie Wang, Guohua Gu, Pengcheng Wang, Weixian Qian, Kan Ren, and Xiaotao Miao. 2018. "Inclinometer Assembly Error Calibration and Horizontal Image Correction in Photoelectric Measurement Systems" Sensors 18, no. 1: 248. https://doi.org/10.3390/s18010248

APA Style

Kong, X., Chen, Q., Wang, J., Gu, G., Wang, P., Qian, W., Ren, K., & Miao, X. (2018). Inclinometer Assembly Error Calibration and Horizontal Image Correction in Photoelectric Measurement Systems. Sensors, 18(1), 248. https://doi.org/10.3390/s18010248

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

Inclinometer Assembly Error Calibration and Horizontal Image Correction in Photoelectric Measurement Systems

Abstract

1. Introduction

2. Related Work and Motivation

3. Inclinometer Assembly Error

4. Inclinometer Assembly Error Calibration and Horizontal Image Correction

4.1. Relationship between the Geodetic Coordinate System and the Inclinometer Coordinate System

4.2. Horizontal Image Correction Using the Inclinometer

4.3. Inclinometer Assembly Error Calibration Based on Plumb Lines

5. Experimental Results and Analyses

5.1. Photoelectric Measurement System

5.2.Calibration of Camera Intrinsic Parameter and Estimation of Lens Radial Distortion Parameter

5.3. Experimental Data Measurement

5.4. Inclinometer Assembly Error Calculation and Horizontal Image Correction

5.5. System Error Analyses

5.5.1. Error Analyses by Simulation Experiment

5.5.2. Error Analyses by Practical Experiment

5.6. Comparison with Other Methods

6. Conclusions

Acknowledgments

Author Contributions

Conflicts of Interest

References

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI