Fengli Zhang
© 2024 The author. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
3D image reconstruction technology holds significant potential for applications in medical imaging, industrial inspection, and virtual reality, offering more intuitive and precise internal structure visualization. However, due to the complexity of human anatomy and the diversity of medical imaging data, traditional 3D reconstruction methods often struggle to achieve optimal results in terms of reconstruction accuracy, computational efficiency, and structural continuity simultaneously. The application of multi-objective optimization in 3D image reconstruction can comprehensively consider multiple objectives, providing more comprehensive and optimized reconstruction results. However, current research methods still have some deficiencies, primarily neglecting the trade-offs between different objectives and experiencing high computational load and low efficiency when handling complex medical imaging data. This study includes the development of image-target 3D reconstruction algorithms in trajectory space and the establishment and solution of a multi-objective optimization-based 3D image reconstruction model. The research content of this paper aims to improve the quality of reconstruction results and provide more reliable technical support for practical applications, in the hopes of enriching the theoretical foundation of 3D image reconstruction as well as offering new technical approaches for practical applications, having significant theoretical and practical value.
3D image reconstruction, multi-objective optimization, medical imaging, computational efficiency, structural continuity
With the rapid development of computer vision and image processing technologies, 3D image reconstruction has become an important research field, widely applied in medical imaging, industrial inspection, virtual reality, and other fields [1-4]. In medical imaging, 3D image reconstruction technology can provide more intuitive and precise internal structure visualization, playing an important auxiliary role in the diagnosis and treatment of diseases [5-7]. However, due to the complexity of the target structures and the diversity of imaging data, how to improve the accuracy and efficiency of 3D reconstruction has always been a challenging and key research area.
The application of multi-objective optimization in 3D image reconstruction is of great significance. Traditional 3D reconstruction methods often consider only a single objective, making it difficult to achieve optimal results in terms of reconstruction accuracy, computational efficiency, and structural continuity simultaneously [8, 9]. Multi-objective optimization methods, on the other hand, can comprehensively consider multiple objectives, providing more comprehensive and optimized reconstruction results [10, 11]. Therefore, introducing multi-objective optimization into 3D image reconstruction can effectively improve the quality of reconstruction results and provide more reliable technical support for practical applications.
However, current research methods still have some deficiencies and shortcomings [12-14]. For example, traditional reconstruction methods based on single-objective optimization often ignore the trade-offs between different objectives, resulting in suboptimal performance in certain aspects of the reconstruction results [15-17]. Additionally, existing multi-objective optimization methods have high computational loads and low efficiency when processing complex imaging data, making it difficult to meet the requirements of real-time applications [18-22]. Therefore, how to improve reconstruction accuracy while also considering computational efficiency and structural continuity remains an urgent problem to be solved.
This paper's main research content includes two parts: first, image-target 3D reconstruction algorithms in trajectory space, and second, the establishment and solution of a multi-objective optimization-based 3D image reconstruction model. By studying 3D reconstruction methods in trajectory space, it is possible to better capture and describe the motion trajectories of image targets, providing more accurate reconstruction results. The reconstruction model based on multi-objective optimization can comprehensively consider multiple objectives, such as reconstruction accuracy, computational efficiency, and structural continuity, providing more comprehensive and optimized reconstruction solutions. This research not only enriches the theoretical foundation of 3D image reconstruction but also provides new technical means for practical applications, having significant theoretical and practical value.
The application of multi-objective optimization in 3D image reconstruction is extensive and significant. A typical application scenario is the reconstruction and analysis of medical images. In medical imaging, it is often required to reconstruct 3D anatomical structures from 2D CT or MRI scan images. In this process, it is necessary to optimize the image resolution, noise level, and computation time simultaneously, and these objectives are often conflicting. Multi-objective optimization technology can find the best balance among these objectives, generating high-quality 3D images to assist doctors in accurate diagnosis and treatment planning. Another important application scenario is industrial inspection and quality control. In manufacturing, 3D image reconstruction technology can be used for non-destructive testing of products to ensure that the dimensions and structures of the products meet design requirements. Multi-objective optimization in such applications can help optimize image clarity and processing speed while reducing errors caused by noise or other interference factors, thereby improving the accuracy and efficiency of the inspection. In addition, in the field of cultural heritage protection and virtual display, 3D image reconstruction can be used for the digital preservation and virtual reconstruction of cultural relics and historical sites. Multi-objective optimization technology can optimize data storage space and processing time while ensuring reconstruction accuracy, making the digitalization of large-scale cultural relics possible. This not only helps in cultural relic protection but also provides the public with a more intuitive cultural experience through virtual reality technology. In the field of autonomous driving and robot navigation, 3D image reconstruction can also be used for environmental perception and path planning. Multi-objective optimization can generate accurate environmental models while optimizing processing speed and resource consumption, thereby improving the real-time response capability and safety of autonomous vehicles or robots. Therefore, multi-objective optimization has a wide range of applications in 3D image reconstruction, significantly enhancing image quality and processing efficiency, and providing strong support for practical applications in various fields.
To achieve 3D image reconstruction based on multi-objective optimization, this paper first performs 3D reconstruction of image targets in trajectory space, which can be refined into the following steps. First, obtain a sequence of D-frame medical images, where each frame contains O feature points. The coordinates of all feature points in all frames form a measurement matrix Q, with dimensions 2D×O. To extract 3D structure and motion information from it, singular value decomposition (SVD) is used to decompose the measurement matrix Q. Through SVD, Q can be decomposed into the product of three matrices, namely Q = IΣNS, where I and N are orthogonal matrices, and Σ is a diagonal matrix. The expression of Q is:
$Q=\left[\begin{array}{l}I \\ N\end{array}\right]=\left[\begin{array}{ccc}a_{11} & \cdots & a_{1o} \\ b_{11} & \cdots & b_{1 o} \\ \vdots & \ddots & \vdots \\ a_{D 1} & \cdots & a_{D o} \\ b_{D 1} & \cdots & b_{D o}\end{array}\right]_{2 D \times O}$ (1)
Based on SVD, by truncating the first 3 columns of I and Σ and the first 3 rows of N, we can initially obtain the motion matrix E2D×3l and the structure matrix T3l×o. The purpose of this process is to convert the original 2D measurement data into a low-dimensional trajectory space, retaining the main motion and structural features of the data. The motion matrix E contains information on non-rigid body motion, while the structure matrix T represents the relative positions of feature points in 3D space, wherein:
$E=\left[\begin{array}{ccccc}E_1 & & & & \\ & \ddots & & & \\ & & E_u & & \\ & & & \ddots & \\ & & & & E_l\end{array}\right]_{2 D \times 3 v}$ (2)
Let Θ=EΦ, according to the equation $T_{3 l \times o}=\Phi_{3 l \times 3 j} \wedge_{3 j \times o}$, we can get:
$Q=E T=E \Theta \Lambda=\Theta \Lambda$ (3)
If the reconstructed image is a medical image, the aforementioned steps are crucial because they provide an initial 3D reconstruction result, helping us understand the basic 3D structure and dynamic changes of the measured object, such as human organs or lesions. In this way, the original image data can be converted into 3D information that is easy to process and analyze, laying the foundation for subsequent optimization and accurate reconstruction. The initial solution obtained from the decomposition of Q is as follows:
$Q=\bar{\Theta} \bar{\Lambda}$ (4)
Because any invertible matrix W3j×3j can satisfy Q=(Θ-Q)(W-1Λ-)=ΘΛ, the result of matrix decomposition is not unique, which means that the decomposed Θ- and Λ- are not the Θ and Λ that are truly needed. Therefore, if this transformation matrix W can be obtained, the reconstruction can be completed. The matrix Θ can be written as:
$\Theta=\left[\begin{array}{ccc}\varphi_{11} E_1 & \cdots & \varphi_{1 j} E_1 \\ \vdots & \ddots & \vdots \\ \varphi_{D 1} E_D & \cdots & \varphi_{D J} E_D\end{array}\right]$ (5)
To solve the transformation matrix W, only any three columns of it are needed. Therefore, the first column, the 1-th column, the j+1-th column, and the 2j+1-th column of the transformation matrix W can be chosen to form a new matrix W. Next, the orthogonality constraint of the rotation matrix Eu can be used to solve W. Here, Θ-2u-1:2u represents the 2u-1-th row and the 2u-th row of matrix Θ. According to the above equation, we can get:
$\bar{\Theta} W=\left[\begin{array}{l}\varphi_{11} E_1 \\ \vdots \\ \varphi_{m 1} E_m\end{array}\right]$ (6)
Further, we can get the following equation:
$\bar{\Theta}_{2 u-1: 2 u} W W^S \bar{\Theta}_{2 u-1: 2 u}^S=z_{u j}^2 U_{2 \times 2}$ (7)
Taking the reconstruction of medical images as an example, the above process can be better understood. In medical image reconstruction, it is usually necessary to reconstruct 3D structures from multi-angle 2D images, such as CT images or MRI images taken from different angles. In the above steps, by solving the transformation matrix W, the trajectory basis information of these 2D images can be converted into 3D structural information. Specifically, by selecting specific columns of the transformation matrix W and using the orthogonality constraint of the rotation matrix, the 3D structure of the target can be accurately restored.
Further attention is paid to the constraint conditions in each frame of the image. For each frame of the image, assuming there are three independent constraints, there are 3D constraints for the entire image sequence, where D represents the number of frames. At the same time, assuming there are 9j unknowns in matrix W, where j represents the dimension or number of certain parameters, at least 3j images are needed to solve matrix W. Once W is obtained, the rotation matrix E can be calculated using equation (5). Next, the matrix Φ2D×3D can be obtained using the relationship Θ2D×3D=E2D×3DΦ2D×3D. The coefficient matrix Λ can be calculated based on the following equation:
$\bar{\Theta}_{2 D \times 3 j} \bar{\Lambda}_{3 j \times o}=Q_{2 D \times o}$ (8)
In the dynamic MRI image reconstruction of the heart, a series of 2D image sequences of the heart is first obtained. Each frame of the image provides three independent constraints, which may come from key point matching, edge detection, or feature point tracking in the image. For the entire sequence, a system containing 3D constraints can be established to describe the movement and deformation of the entire heart in the time sequence. Due to the complexity of the unknown W, at least 3k frames are needed to obtain sufficient information to solve W.
In the process of medical imaging, multiple frames of images are often needed from different angles to perform 3D reconstruction of non-rigid body tissues inside the body. The measurement matrix can be extracted from these images, and the transformation matrix W can be solved by optimizing the orthogonality of the rotation matrix. Specifically, the rotation matrix E needs to satisfy the unit orthogonal requirement, i.e., the rank constraint of the matrix.
$\bar{\Theta}_{2 u-1: 2 u} W_k \bar{\Theta}_{2 u-1: 2 u}^S=\varphi_{u j}^2 U_{2 \times 2}^2, u=1 \ldots D$ (9)
Since ϕuk is unknown and Wj is symmetric, two equivalent non-diagonal constraints are needed to linearly constrain W. For all D frames of medical images in the entire image sequence during the medical imaging process, 2D linear constraints can be obtained, expressed as follows:
$\bar{\Theta}_{2 u-1} W_j \bar{\Theta}_{2 u}^S-\bar{\Theta}_{2 u} W_j \bar{\Theta}_{2 u}^S=0, u=1 \ldots D$ (10)
$\bar{\Theta}_{2 u-1} W_j \bar{\Theta}_{2 u}^S=0, u=1 \ldots D$ (11)
Next, the least squares method is used to solve these linear equations to obtain the transformation matrix W. Once W is solved, the motion matrix and the structure matrix can be further calculated. The solution process of these matrices is an important step in the 3D reconstruction algorithm, as they determine the spatial relationship between each frame in the image sequence and the 3D shape of the target structure.
In the 3D reconstruction of image targets in trajectory space, the construction of the multi-objective optimization function is a crucial step to achieving high-precision reconstruction. Figure 1 shows the flowchart for establishing a 3D image reconstruction model based on multi-objective optimization. Taking the reconstruction of medical images as an example, the optimization process can be described as follows:
Figure 1. Flowchart for establishing a 3D image reconstruction model based on multi-objective optimization
The first objective of the reconstruction is to minimize the back-projection error. The setting of this objective means that when the reconstructed 3D structure is projected back to the 2D image, the resulting projection should be as close as possible to the original image. The core of this objective is to ensure the accuracy of the 3D reconstruction, making it truly reflect the target objects in the image. This objective can be achieved by minimizing the sum of squared distances between all projection points and the actual image points. Assuming the real 2D measurement matrix of the reconstruction target is represented by Ql and the measurement matrix obtained by back-projection is represented by Q~l, we have the expression:
$\gamma_{e r}=\sum_{l=1}^D\left\|Q_l-\tilde{Q}_l\right\|^2, \tilde{Q}=E_l T_l$ (12)
The second objective is the smoothness of the motion trajectory. In medical imaging, especially in the reconstruction of dynamic organs such as the heart, it is required that the motion trajectory be smooth to reflect the natural motion of the organs. This can be achieved by introducing a smoothness constraint on the motion trajectory, ensuring that the changes in the motion trajectory between each frame in the time series are continuous and smooth. Specifically, this objective can be achieved by minimizing the differences in motion parameters between adjacent frames.
$\gamma_o=\sum_{v=1}^o \sum_{l=1}^{D-1}[f(S H A) / f A+d(S H B) / f B+f(S H C) / f C]$ (13)
The third objective is the continuity of the structure. In 3D image reconstruction, especially for complex structures such as human organs, it is required that the reconstructed 3D structure has spatial continuity and consistency. This means that during the reconstruction process, the geometric relationships between adjacent points should remain unchanged, avoiding unreasonable distortions or breaks. This objective can be achieved by adding constraints on structural continuity during the optimization process, such as minimizing the changes in distances between 3D points. Assuming the original 3D structure is represented by T, and the reconstructed 3D structure data is represented by SH, with the number of frames in the image sequence and the number of feature points represented by D and O respectively, and the coordinate matrices of the A, B, and C axes in the 3D structure Shat represented by SHA, SHB, and SHC respectively, we have:
${{\gamma }_{t}}=\sum\limits_{v=1}^{O-1}{\sum\limits_{l=1}^{D-1}{\left[ {f\left( {f\left( SHA \right)}/{fz}\; \right)}/{fe}\;+{f\left( {f\left( SHB \right)}/{fz}\; \right)}/{fe}\;+{f\left( {f\left( SHC \right)}/{fz}\; \right)}/{fe}\; \right]}}$ (14)
Combining the above three objectives, a multi-objective cost function can be constructed. This cost function comprehensively considers the back-projection error, the smoothness of the motion trajectory, and the continuity of the structure, integrating these objectives into a unified optimization framework through weighted summation. Specifically, assuming the scalars controlling the importance of each objective constraint are represented by ψ1, ψ2, and ψ3, the cost function is expressed as:
$d=M I N\left(\psi_1 \lambda_{e r}+\psi_2 \gamma_o+\psi_3 \gamma_t\right)$ (15)
In the multi-objective optimization function, the computational load of function γt is large, which is not conducive to real-time computation. Let W(u,k)=d(u,k)/∑Lu=1∑Vk=1d(u,k), and discarding higher-order terms, we can obtain an approximate calculation formula for γt:
$\gamma_t(a) \approx-\sum_{u=1}^L \sum_{k=1}^V W(u, k)(W(u, k)-1)=1-\sum_{u=1}^L \sum_{k=1}^V W(u, k)^2$ (16)
The original objective function is uniformly minimized to convert it into a more manageable form. In this process, each objective function in the multi-objective optimization problem is normalized to ensure they are compared and optimized on the same scale. Specifically, by adjusting the weights of each objective function, their influence in the optimization process is reasonably balanced. The purpose of this step is to simplify the complex multi-objective optimization problem into a single-objective optimization problem, making it easier to solve subsequently.
$\begin{cases}\operatorname{MIN} & d_1(a)=\sum_{u=1}^U\left(o_u-\sum_{k=1}^K e_{u k} a_k\right)^2 \\ \operatorname{MIN} & d_2(a)=\sum_{u=1}^L \sum_{k=1}^V\left(\frac{d(u, k)}{\sum_{u=1}^L \sum_{k=1}^V d(u, k)}\right)^2-1 \\ \operatorname{MIN} & d_3(a)=\sum_{k=1}^K\left(a_k-\bar{a}\right)^2\end{cases}$ (17)
In the equation, the feasible region A- is defined as:
$\bar{A}\overset{\Delta }{\mathop{=}}\left\{A \in E^V \mid E a=O, a \geq 0\right\}$ (18)
Through the method of linear weighted sum, multiple objective functions are transformed into an optimization problem of a scalar-valued function. This process specifically involves multiplying each objective function by an appropriate weight and then summing them to form a comprehensive optimization objective function. The selection of these weights needs to be adjusted according to the specific application scenario and the importance of the objective functions. In medical image reconstruction, these weights can be reasonably set based on factors such as reconstruction accuracy and computational efficiency, thereby improving computational efficiency while ensuring accuracy. Specifically, assuming the weighting coefficients are represented by μm, we have:
$MIN \quad d(a)=\sum_{m=1}^3 \mu_m d_m(a)$ (19)
At the same time:
$\mu \overset{\Delta }{\mathop{=}}\,\left\{ {{\mu }_{s}}\ge 0,m=1,2,3\begin{matrix} {} & \sum\limits_{m=1}^{3}{{{\mu }_{m}}}=1 \\\end{matrix} \right\}$ (20)
Once the weighting coefficients μm are determined, our multi-objective optimization reconstruction problem can be equivalent to a single-objective optimization problem. Specifically, by assigning different weights to multiple objective functions based on their importance, and then linearly combining these weighted objective functions, a comprehensive optimization objective function is formed. The key to this step is to reasonably select the weights, so that the optimization problem can consider the needs of each objective while simplifying to an easily solvable optimization problem.
$\begin{cases} { MIN } & d(a)=\sum_{m=1}^3 \mu_m d_m(a) \\ { t.s. } & o-E a=0 \\ & a \geq 0\end{cases}$ (21)
After determining the weighting coefficients, the optimization problem is transformed into a typical nonlinear programming problem with equality constraints. To solve this problem, we use the method of Lagrange multipliers. By introducing Lagrange multipliers, the constrained optimization problem is transformed into an unconstrained problem. Specifically, we construct the Lagrangian function, which combines the objective function and the constraint conditions. By optimizing this unconstrained Lagrangian function, we can simultaneously satisfy the minimization of the objective function and the constraint conditions. Assuming the K-dimensional Lagrange multiplier vector is represented by η and the L-dimensional multiplier vector is represented by I, we have:
$M(a, I, \eta)=\mu_1 \sum_{u=1}^U\left(o-\sum_{k=1}^K e_{u k} a_k\right)+\mu_2 \sum_{k=1}^K\left(a_k-\bar{a}\right)^2+\mu_3\left(\sum_{u=1}^L \sum_{k=1}^V\left(\frac{d(u, j)}{\sum_{u=1}^M \sum_{k=1}^N d(u, j)}\right)-1\right)+I^S(O-O a)+\eta^S a$ (22)
By taking the partial derivative of the Lagrangian function M(a,I,η) with respect to the vector a and setting its derivative to zero, we can obtain the necessary conditions for the optimization problem. These conditions can be transformed into the multi-objective iterative reconstruction formulas. In practical applications, we iteratively solve these formulas to gradually approach the solution to the optimization problem. Specifically, in medical image reconstruction, by continuously adjusting and optimizing the 3D structural parameters of the image, we can gradually obtain a high-precision reconstruction result that meets the requirements of multi-objective optimization. The multi-objective iterative reconstruction formulas obtained after transformation are as follows:
$a_k^{(j+1)}=a_k^j \frac{\mu_2 \sum_{u=1}^U e_{u k} o_u+\mu_3 \sum_{u=1}^U i_u e_{u k}}{\mu_2 a_k^{(j)}+\mu_2 \sum_{u=1}^L \sum_{k=1}^V e_{u k} a_k^{(j)}}$ (23)
According to mathematical programming theory, the parameters ij are adjusted as follows:
$i_u^{(j+1)}=i_u^{(j)}+\left(o_u-\sum_{k=1}^V e_{u k} a_k^{(j)}\right) \quad u=1,2, \cdots, U$ (24)
Let ∆wu=|dm(aj)-dm(aj+1)|(m=1,2,...,M), and the selection of the weight coefficients follows the rules below:
$\left\{\begin{array}{l}\Delta w=\left|e_m\left(a^j\right)-d_m\left(a^{j+1}\right)\right| \\ \mu_m^{j+1}=\Delta w / \sum_{m=1}^M \Delta w_m\end{array} m=1,2,3\right.$ (25)
In Table 1, the noise-free normalized mean squared distance (NMSE) between the reconstructed images and the original images at angles of 120°, 140°, and 160° is compared for three different methods. The results show that this method exhibits the lowest NMSE values at all angles, which are 0.2236, 0.1568, and 0.0365, respectively. This is significantly better than adaptive voxel reconstruction (0.3654, 0.2875, and 0.1278) and GPU-accelerated volume rendering (0.2985, 0.2016, and 0.0389). Particularly at 160°, the NMSE of this method is 0.0365, which is lower than 0.1278 of adaptive voxel reconstruction and 0.0389 of GPU-accelerated volume rendering, indicating that this method is especially outstanding in reconstruction accuracy at larger angles. The experimental results demonstrate the effectiveness of the 3D reconstruction algorithm in trajectory space and the 3D reconstruction model based on multi-objective optimization proposed in this paper. By capturing the motion trajectories of image targets, this method can reconstruct image structures more accurately, while the multi-objective optimization model helps achieve a comprehensive balance of reconstruction accuracy, computational efficiency, and structural continuity.
Table 1. Comparison of the noise-free normalized mean squared distance between reconstructed images and original images
|
|
120° |
140° |
160° |
|
Adaptive Voxel Reconstruction |
0.3654 |
0.2875 |
0.1278 |
|
GPU-Accelerated Volume Rendering |
0.2985 |
0.2016 |
0.0389 |
|
The Proposed Method |
0.2236 |
0.1568 |
0.0365 |
Figure 2. 3D structural error of reconstructed image sequences using multi-objective optimization method with different weights
In Figure 2, the 3D structural error data of reconstructed image sequences using the multi-objective optimization method under different weights (0.3, 0.4, 0.3) and (0.4, 0.3, 0.3) show the changes in error. For the weights (0.3, 0.4, 0.3), the error ranges from 0.04 to 0.095 from frame 0 to frame 1000, with overall error fluctuating as the number of frames increases but remaining at a relatively low level for most frames. For the weights (0.4, 0.3, 0.3), the error range is similar to the previous set, also between 0.04 and 0.095, with error values very close at the same frame numbers. For example, at frame 0 and frame 200, the error values for both sets of weights are 0.04 and 0.045, showing that the error variation trends under both sets of weights are basically consistent. This indicates that different weight settings have limited impact on the error, but the weights (0.4, 0.3, 0.3) have slightly lower error at certain frames (such as frame 400 and frame 600). The above error data indicates that the proposed 3D image reconstruction model based on multi-objective optimization has significant stability and adaptability. The error data of both sets of weights remain at a low level under different frames, especially at key frames such as frame 400 and frame 600, where the error is relatively small, showing high reconstruction accuracy. Furthermore, although the error variation trends of the two sets of weights are similar, the reconstruction quality of certain specific frames can be further optimized by adjusting the weights.
In Figure 3, the back-projection error data of reconstructed image sequences using the multi-objective optimization method under two sets of weights (0.3, 0.4, 0.3) and (0.4, 0.3, 0.3) are shown. For the weights (0.3, 0.4, 0.3), the error ranges from 0.006 to 0.088 from frame 0 to frame 1000. Specifically, at frame 200 the error is 0.036 and 0.035, at frame 400 the error is 0.016 and 0.02, at frame 600 the error is 0.025 and 0.022, at frame 800 the error is 0.021 and 0.014, and at frame 1000 the error is 0.088. For the weights (0.4, 0.3, 0.3), the error ranges from 0.005 to 0.082. At the same frame numbers, the error is 0.038 and 0.035 at frame 200, 0.016 and 0.02 at frame 400, 0.022 and 0.021 at frame 600, 0.02 and 0.012 at frame 800, and 0.082 at frame 1000. Overall, the error trends for the two sets of weights are similar across different frames, with close error values and low error at almost all frames. Particularly at key frames such as frame 400 and frame 600, the error difference between the two is minimal. The above error data shows that the proposed 3D image reconstruction model based on multi-objective optimization performs well under different weight settings, with error values remaining low for most frames, indicating high stability and reliability in reconstruction accuracy. Although the error differences between different weights are small, the reconstruction results of specific frames can still be further optimized by reasonably adjusting the weights. This shows that the multi-objective optimization method can effectively balance and integrate multiple objectives such as reconstruction accuracy, computational efficiency, and structural continuity, providing a more comprehensive and optimized reconstruction solution.
In Figure 4, the relative trajectory smoothness error data of reconstructed image sequences using the multi-objective optimization method under two sets of weights (0.3, 0.4, 0.3) and (0.4, 0.3, 0.3) are shown. For the weights (0.3, 0.4, 0.3), the error fluctuates greatly from frame 0 to frame 1000, ranging from 1.4 to 6.5. At frame 200, the error ranges from 2.8 to 2.4; at frame 400, from 0.6 to 2.4; at frame 600, from 0.6 to 1.5; at frame 800, from 0 to 3.2; and at frame 1000, from 0.4 to 6.5. For the weights (0.4, 0.3, 0.3), the error variation range is relatively narrower, from 1.4 to 5. At frame 200, the error ranges from 3 to 2.4; at frame 400, from 0.4 to 2.4; at frame 600, from 0.4 to 1.6; at frame 800, from 0 to 3.2; and at frame 1000, from 0.2 to 5. Overall, although the errors for the two sets of weights differ under different frame numbers, the error values are relatively low for most frames, especially at frames 600 and 800, where the error differences between the two sets of weights are small. The above error data show that the proposed 3D image reconstruction model based on multi-objective optimization exhibits good trajectory smoothness effects under different weight settings. Although the adjustment of weights may affect the error of certain frames, the overall error value remains at a relatively low level in most cases, indicating that the model performs with high stability and reliability in terms of trajectory smoothness. Especially at key points such as frame 400 and frame 600, the error differences between the two sets of weights are small, further proving the robustness and adaptability of the model. Additionally, the highest error value appears at frame 1000, indicating that further optimization and adjustment of weights may be needed in the reconstruction of long-time sequences to reduce cumulative errors.
Figure 3. Back-projection error of reconstructed image sequences using multi-objective optimization method with different weights
Figure 4. Relative trajectory smoothness error of reconstructed image sequences using multi-objective optimization method with different weights
Figure 5. Relative structural continuity error of reconstructed image sequences using multi-objective optimization method with different weights
In Figure 5, the relative structural continuity error data of reconstructed image sequences using the multi-objective optimization method under two sets of weights (0.3, 0.4, 0.3) and (0.4, 0.3, 0.3) are shown. For the weights (0.3, 0.4, 0.3), the error values fluctuate within the range of 0 to 1000 frames but generally remain at a low level. For example, the error value is 0.93 to 1.50 at frame 0, 0.13 to 1.40 at frame 200, 0.40 to 1.60 at frame 400, 0.40 to 0.80 at frame 600, 0 to 2.13 at frame 800, and 0.80 to 4.33 at frame 1000. For the weights (0.4, 0.3, 0.3), the error variation range is slightly different but generally stable. At frame 0, the error value is 0.93 to 1.60; at frame 200, 0.13 to 1.20; at frame 400, 0.27 to 1.60; at frame 600, 0.27 to 0.80; at frame 800, 0 to 2.13; and at frame 1000, 0.93 to 3.33. Although the errors for the two sets of weights differ under different frame numbers, the error values are relatively low in most cases, showing that the model has high trajectory smoothness and structural continuity. The above error data indicate that the proposed 3D image reconstruction model based on multi-objective optimization exhibits good structural continuity effects under different weight settings. Although different weight settings have a certain impact on error values, the overall error value remains at a low level in most cases, indicating that the model performs with high stability and reliability in terms of structural continuity. Especially at frames 600 and 800, the error differences between the two sets of weights are small, further proving the robustness and adaptability of the model. Additionally, although the error value is relatively high at frame 1000, this also suggests that further optimization of weight settings may be needed in the reconstruction of long-time sequences to reduce cumulative errors.
Table 2 shows the mean 3D structural error and its variance for different types of image sequences under different weight settings (0.3, 0.4, 0.3) and (0.4, 0.3, 0.3). For T1-weighted images, the mean structural error is 0.1621 and the variance is 0.0897 with weights (0.3, 0.4, 0.3); with weights (0.4, 0.3, 0.3), the mean structural error slightly decreases to 0.1562 and the variance decreases to 0.0879. For T2-weighted images, the mean structural error and variance are very close for both weight settings, being 0.2345 and 0.1065, and 0.2341 and 0.1032, respectively. For FLAIR images, the mean error and variance are 0.1078 and 0.1056 with weights (0.3, 0.4, 0.3), while with weights (0.4, 0.3, 0.3), the mean error is 0.0945 and the variance is 0.1045. Lastly, for diffusion-weighted imaging, the mean structural error is 0.0245 for both weight settings, with variances of 0.0107 and 0.0104, respectively. Overall, the adjustment of weights has some impact on the structural error of different types of image sequences, but the error values and variances remain at a low level. The above experimental results clearly demonstrate the influence of different weight parameters on the 3D structural error, and indicate that the proposed multi-objective optimization reconstruction model performs stably and excellently across various image sequences. The structural errors for T1 and T2-weighted images show little difference under different weights, demonstrating the robustness and reliability of the model in handling these types of images. For FLAIR images, the mean error and variance are lower with weights (0.4, 0.3, 0.3), indicating that this weight setting is more effective in maintaining structural continuity and accuracy. Diffusion-weighted imaging shows almost consistent performance under both weight settings, further proving the stability and efficiency of the model.
Table 2. Mean 3D structural error for different types of image sequences
|
Type of Image Sequence |
(0.3 0.4 0.3) |
(0.4 0.3 0.3) |
||
|
|
Mean Structural Error |
Variance of Structural Error |
Mean Structural Error |
Variance of Structural Error |
|
T1-weighted images |
0.1621 |
0.0897 |
0.1562 |
0.0879 |
|
T2-weighted images |
0.2345 |
0.1065 |
0.2341 |
0.1032 |
|
FLAIR images |
0.1078 |
0.1056 |
0.0945 |
0.1045 |
|
Diffusion-weighted images |
0.0245 |
0.0107 |
0.0245 |
0.0104 |
Table 3. Relative trajectory smoothness error and structural continuity error for different types of image sequences
|
Type of Image Sequence |
Trajectory Smoothness |
Structural Continuity |
||
|
|
(0.3 0.4 0.3) |
(0.4 0.3 0.3) |
(0.3 0.4 0.3) |
(0.4 0.3 0.3) |
|
T1-weighted images |
0.0021 |
0.0011 |
0.0013 |
0.0008 |
|
T2-weighted images |
0.0038 |
0.0027 |
0.00097 |
0.00048 |
|
FLAIR images |
0.0028 |
0.0021 |
0.00092 |
0.00057 |
|
Diffusion-weighted images |
0.00072 |
0.00071 |
0.00041 |
0.00042 |
Table 3 shows the relative trajectory smoothness error and structural continuity error for different types of image sequences under different weight settings (0.3, 0.4, 0.3) and (0.4, 0.3, 0.3). For T1-weighted images, the trajectory smoothness error is 0.0021 and the structural continuity error is 0.0013 with weights (0.3, 0.4, 0.3); with weights (0.4, 0.3, 0.3), the trajectory smoothness error decreases to 0.0011 and the structural continuity error also decreases to 0.0008. For T2-weighted images, the trajectory smoothness error and structural continuity error are 0.0038 and 0.00097 with weights (0.3, 0.4, 0.3); with weights (0.4, 0.3, 0.3), they decrease to 0.0027 and 0.00048, respectively. For FLAIR images, the trajectory smoothness error is 0.0028 and the structural continuity error is 0.00092 with weights (0.3, 0.4, 0.3); with weights (0.4, 0.3, 0.3), they decrease to 0.0021 and 0.00057, respectively. For diffusion-weighted imaging, the trajectory smoothness error and structural continuity error change slightly with weights (0.3, 0.4, 0.3) and (0.4, 0.3, 0.3), being 0.00072 and 0.00071, and 0.00041 and 0.00042, respectively. From the results in Table 3, it can be seen that different weight settings have a significant impact on trajectory smoothness and structural continuity. Overall, with weights (0.4, 0.3, 0.3), the trajectory smoothness error and structural continuity error of various types of image sequences decrease, especially for T1 and T2-weighted images, where the errors significantly reduce, indicating that the model can better balance the smoothness of the motion trajectory and the continuity of the structure under this weight setting. The errors for FLAIR and diffusion-weighted imaging also show a similar trend, although the changes are smaller, they still exhibit a certain degree of optimization, further proving the effectiveness of this weight combination.
This study focuses on two main aspects: firstly, proposing an image target 3D reconstruction algorithm in trajectory space aimed at more accurately capturing and describing the motion trajectories of image targets; secondly, establishing and solving a 3D image reconstruction model based on multi-objective optimization theory to balance multiple objectives such as reconstruction accuracy, computational efficiency, and structural continuity. The experimental results cover the following aspects: comparison of noise-free normalized mean squared distance between reconstructed images and original images, 3D structural error and back-projection error of reconstructed image sequences using the multi-objective optimization method under different weights, as well as the relative values of trajectory smoothness error and structural continuity error, and the mean 3D structural error and relative values of trajectory smoothness error and structural continuity error for different types of image sequences. The experimental results show that the multi-objective optimization method with different weight settings can effectively reduce the trajectory smoothness error and structural continuity error, especially with the weight setting (0.4, 0.3, 0.3), significantly reducing the reconstruction error of various image sequences. This not only validates the effectiveness and broad applicability of the multi-objective optimization method in 3D image reconstruction but also provides valuable data and methodological support for subsequent research.
However, this study also has certain limitations. Firstly, the experiments mainly focus on specific types of medical image sequences, and future research needs to verify the applicability of this method to other types of images. Secondly, although the multi-objective optimization method shows good performance, its computational complexity is relatively high, and further exploration is needed on how to improve the computational efficiency of the algorithm. Additionally, this study is mainly based on simulated data, and in practical applications, the robustness and adaptability of the algorithm need to be further verified under the influence of noise and other uncertainties. Future research directions can include the following points: firstly, extending the application range of the multi-objective optimization method to verify its effectiveness in more types of image sequences; secondly, studying more efficient optimization algorithms to reduce computational complexity and improve processing speed; thirdly, further enhancing the robustness of the algorithm in practical applications, especially under the influence of noise and other uncertainties. These studies will help to further improve the practicality and broad applicability of 3D image reconstruction technology.
[1] Wang, J., Niu, G. (2024). Application of 3D image reconstruction on landscape architecture in environmental design system. Computer-Aided Design and Applications, 21(S1): 46-60. https://doi.org/10.14733/cadaps.2024.S1.46-60
[2] Wang, F., Ni, W., Liu, S., Xu, Z., Qiu, Z., Wan, Z. (2023). A 2D image 3D reconstruction function adaptive denoising algorithm. PeerJ Computer Science, 9: e1604. https://doi.org/10.7717/peerj-cs.1604
[3] Lu, Y., Wang, S., Fan, S., Lu, J., Li, P., Tang, P. (2024). Image-based 3D reconstruction for multi-scale civil and infrastructure projects: A review from 2012 to 2022 with new perspective from deep learning methods. Advanced Engineering Informatics, 59: 102268. https://doi.org/10.1016/j.aei.2023.102268
[4] Zhou, Y., Chen, D., Ye, H., Wang, X., Deng, H., Wang, D., Liu, J., Du, X., Liu, L., Zhang, Z., Zhang, J. (2023). 3D reconstruction method of metal pipe inner wall based on multi-camera sequential image acquisition. In China Conference on Command and Control, Springer, Singapore, 1124: 527-539. https://doi.org/10.1007/978-981-99-9021-4_49
[5] Rao, C., Wu, Q., Zhou, P., Yu, J., Zhang, Y., Lou, X. (2022). An energy-efficient accelerator for medical image reconstruction from implicit neural representation. IEEE Transactions on Circuits and Systems I: Regular Papers, 70(4): 1625-1638. https://doi.org/10.1109/TCSI.2022.3231863
[6] Wu, T., Huang, C., Jia, S., Li, W., Chan, R., Zeng, T., Zhou, S.K. (2024). Medical image reconstruction with multi-level deep learning denoiser and tight frame regularization. Applied Mathematics and Computation, 477: 128795. https://doi.org/10.1016/j.amc.2024.128795
[7] Nakao, M. (2023). Medical image synthesis and statistical reconstruction methods. Advanced Biomedical Engineering, 12: 21-27. https://doi.org/10.14326/abe.12.21
[8] Wang, C., Xia, Y., Wang, J., Zhao, K., Peng, W., Yu, W. (2024). An interactive method based on multi-objective optimization for limited-angle CT reconstruction. Physics in Medicine & Biology, 69(9): 095019. https://doi.org/10.1088/1361-6560/ad3724
[9] Kamisli, F., Racape, F., Choi, H. (2024). Variable-rate learned image compression with multi-objective optimization and quantization-reconstruction offsets. arXiv preprint arXiv: 2402.18930. https://doi.org/10.48550/arXiv.2402.18930
[10] Tan, X., Yin, C., Huang, X., Dadras, S., Shi, A. (2024). Design of multi-objective guided filter infrared thermal image fusion for hypervelocity impact damages detection. Journal of the Franklin Institute, 361(2): 712-731. https://doi.org/10.1016/j.jfranklin.2023.12.020
[11] Abouhawwash, M., Alessio, A.M. (2022). Evolutionary optimization of multiple machine-learned objectives for PET image reconstruction. IEEE Transactions on Radiation and Plasma Medical Sciences, 7(3): 273-283. https://doi.org/10.1109/TRPMS.2022.3205283
[12] He, Y., Peng, H., Deng, C., Dong, X., Wu, Z., Guo, Z. (2023). Reference point reconstruction-based firefly algorithm for irregular multi-objective optimization. Applied Intelligence, 53(1): 962-983. https://doi.org/10.1007/s10489-022-03561-w
[13] Abouhawwash, M., Alessio, A.M. (2021). Multi-objective evolutionary algorithm for PET image reconstruction: Concept. IEEE Transactions on Medical Imaging, 40(8): 2142-2151. https://doi.org/10.1109/TMI.2021.3073243
[14] Chai, X., Fu, J., Gan, Z., Lu, Y., Zhang, Y. (2022). An image encryption scheme based on multi-objective optimization and block compressed sensing. Nonlinear Dynamics, 108(3): 2671-2704. https://doi.org/10.1007/s11071-022-07328-3
[15] Han, W., Li, H., Gong, M. (2023). Multi-regularization sparse reconstruction based on multifactorial multiobjective optimization. Applied Soft Computing, 136: 110122. https://doi.org/10.1016/j.asoc.2023.110122
[16] Yang, G., Zhang, L., Liu, A., Fu, X., Chen, X., Wang, R. (2023). MGDUN: An interpretable network for multi-contrast MRI image super-resolution reconstruction. Computers in Biology and Medicine, 167: 107605. https://doi.org/10.1016/j.compbiomed.2023.107605
[17] Huang, H., Zhang, Z., Li, Z., Li, Y., Lin, X., Huang, Q., Sun, J. (2024). Noise reduction of axial piston motors using multi-objective optimization of reinforcing ribs based on identification of acoustic sources. Applied Acoustics, 222: 110044. https://doi.org/10.1016/j.apacoust.2024.110044
[18] Bhasha, A.V., Reddy, B.V. (2022). A multi-objective opposition-based barnacles mating optimization for image super resolution using hyper-Spectral images. Journal of Engineering, Design and Technology, 20(6): 1538-1564. https://doi.org/10.1108/JEDT-01-2021-0030
[19] Xu, J., Li, Q., Guan, Z., Wang, X. (2022). Nonlinear reconstruction for target density based on randomly perturbed optimization and multi-models fusion. Acta Photonica Sinica, 51(3): 0310006. http://doi.org/10.3788/gzxb20225103.0310006
[20] Wang, J., Yang, Q., Yang, Q., Xu, L., Cai, C., Cai, S. (2022). Joint optimization of Cartesian sampling patterns and reconstruction for single-contrast and multi-contrast fast magnetic resonance imaging. Computer Methods and Programs in Biomedicine, 226: 107150. https://doi.org/10.1016/j.cmpb.2022.107150
[21] Liu, P., Fang, C., Qiao, Z. (2024). An edge-preserving total nuclear variation minimization algorithm in EPR image reconstruction. Biomedical Signal Processing and Control, 87: 105426. https://doi.org/10.1016/j.bspc.2023.105426
[22] Zhang, X., Li, M., Hilton, A., Pal, A., Dey, S., Debroy, S. (2022). End-to-end latency optimization of multi-view 3D reconstruction for disaster response. In 2022 10th IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (MobileCloud), San Francisco, CA, USA, pp. 17-24. https://doi.org/10.1109/MobileCloud55333.2022.00010