Search Results (326)

To address the challenge of rapid geometric model development in the digital twin industry, this paper presents a comprehensive pipeline for constructing 3D models from images using monocular vision imaging principles. Firstly, a structure-from-motion (SFM) algorithm generates a 3D point cloud from photographs. The feature detection methods scale-invariant feature transform (SIFT), speeded-up robust features (SURF), and KAZE are compared across six datasets, with SIFT proving the most effective (matching rate higher than 0.12). Using K-nearest-neighbor matching and random sample consensus (RANSAC), refined feature point matching and 3D spatial representation are achieved via antipodal geometry. Then, the Poisson surface reconstruction algorithm converts the point cloud into a mesh model. Additionally, texture images are enhanced by leveraging a visual geometry group (VGG) network-based deep learning approach. Content images from a dataset provide geometric contours via higher-level VGG layers, while textures from style images are extracted using the lower-level layers. These are fused to create texture-transferred images, where the image quality assessment (IQA) metrics SSIM and PSNR are used to evaluate texture-enhanced images. Finally, texture mapping integrates the enhanced textures with the mesh model, improving the scene representation with enhanced texture. The method presented in this paper surpassed a LiDAR-based reconstruction approach by $20\%$ in terms of point cloud density and number of model facets, while the hardware cost was only $1\%$ of that associated with LiDAR. Full article

Image matching-based visual simultaneous localization and mapping (vSLAM) extracts low-level pixel features to reconstruct camera trajectories and maps through the epipolar geometry method. However, it fails to achieve correct trajectories and mapping when there are low-quality feature correspondences in several challenging environments. Although the RANSAC-based framework can enable better results, it is computationally inefficient and unstable in the presence of a large number of outliers. A Faster R-CNN learning-based semantic filter is proposed to explore the semantic information of inliers to remove low-quality correspondences, helping vSLAM localize accurately in our previous work. However, the semantic filter learning method generalizes low precision for low-level and dense texture-rich scenes, leading the semantic filter-based vSLAM to be unstable and have poor geometry estimation. In this paper, a GGCM-E-based semantic filter using YOLOv8 is proposed to address these problems. Firstly, the semantic patches of images are collected from the KITTI dataset, the TUM dataset provided by the Technical University of Munich, and real outdoor scenes. Secondly, the semantic patches are classified by our proposed GGCM-E descriptors to obtain the YOLOv8 neural network training dataset. Finally, several semantic filters for filtering low-level and dense texture-rich scenes are generated and combined into the ORB-SLAM3 system. Extensive experiments show that the semantic filter can detect and classify semantic levels of different scenes effectively, filtering low-level semantic scenes to improve the quality of correspondences, thus achieving accurate and robust trajectory reconstruction and mapping. For the challenging autonomous driving benchmark and real environments, the vSLAM system with respect to the GGCM-E-based semantic filter demonstrates its superiority regarding reducing the 3D position error, such that the absolute trajectory error is reduced by up to approximately 17.44%, showing its promise and good generalization. Full article

In traditional 3D reconstruction using UAV images, only radiance information, which is treated as a geometric constraint, is used in feature matching, allowing for the restoration of the scene’s structure. After introducing radiance supervision, NeRF can adjust the geometry in the fixed-ray direction, resulting in a smaller search space and higher robustness. Considering the lack of NeRF construction methods for aerial scenarios, we propose a new NeRF point sampling method, which is generated using a UAV imaging model, compatible with a global geographic coordinate system, and suitable for a UAV view. We found that NeRF is optimized entirely based on the radiance while ignoring the direct geometry constraint. Therefore, we designed a radiance correction strategy that considers the incidence angle. Our method can complete point sampling in a UAV imaging scene, as well as simultaneously perform digital surface model construction and ground radiance information recovery. When tested on self-acquired datasets, the NeRF variant proposed in this paper achieved better reconstruction accuracy than the original NeRF-based methods. It also reached a level of precision comparable to that of traditional photogrammetry methods, and it is capable of outputting a surface albedo that includes shadow information. Full article

This work presents a numerical mesh generation method for 3D urban scenes that could be easily converted into any 3D format, different from most implementations which are limited to specific environments in their applicability. The building models have shaped roofs and faces with static colors, combining the buildings with a ground grid. The building generation uses geographic positions and shape names, which can be extracted from OpenStreetMap. Additional steps, like a computer vision method, can be integrated into the generation optionally to improve the quality of the model, although this is highly time-consuming. Its function is to classify unknown roof shapes from satellite images with adequate resolution. The generation can also use custom geographic information. This aspect was tested using information created by procedural processes. The method was validated by results generated for many realistic scenarios with multiple building entities, comparing the results between using computer vision and not. The generated models were attempted to be rendered under Graphics Library Transmission Format and Unity Engine. In future work, a polygon-covering algorithm needs to be completed to process the building footprints more effectively, and a solution is required for the missing height values in OpenStreetMap. Full article

A free-boundary equilibrium solver for an axisymmetric tokamak geometry was developed based on the finite difference method and Picard iteration in a rectangular computational area. The solver can run either in forward mode, where external coil currents are prescribed until the converged magnetic flux function $\psi (R,Z)$ map is achieved, or in inverse mode, where the desired plasma boundary, with or without an X-point, is prescribed to determine the required coil currents. The equilibrium solutions are made consistent with prescribed plasma parameters, such as the total plasma current, poloidal beta, or safety factor at a specified flux surface. To verify the mathematical correctness and accuracy of the solver, the solution obtained using this numerical solver was compared with that from an analytic fixed-boundary equilibrium solver based on the EAST geometry. Additionally, the proposed solver was benchmarked against another numerical solver based on the finite-element and Newton-iteration methods in a triangular-based mesh. Finally, the proposed solver was compared with equilibrium reconstruction results in DIII-D experiments. Full article

In this paper, we propose a 3D Digital Surface Model (DSM) reconstruction method from uncalibrated Multi-view Satellite Stereo (MVSS) images, where Rational Polynomial Coefficient (RPC) sensor parameters are not available. While recent investigations have introduced several techniques to reconstruct high-precision and high-density DSMs from MVSS images, they inherently depend on the use of geo-corrected RPC sensor parameters. However, RPC parameters from satellite sensors are subject to being erroneous due to inaccurate sensor data. In addition, due to the increasing data availability from the internet, uncalibrated satellite images can be easily obtained without RPC parameters. This study proposes a novel method to reconstruct a 3D DSM from uncalibrated MVSS images by estimating and integrating RPC parameters. To do this, we first employ a structure from motion (SfM) and 3D homography-based geo-referencing method to reconstruct an initial DSM. Second, we sample 3D points from the initial DSM as references and reproject them to the 2D image space to determine 3D–2D correspondences. Using the correspondences, we directly calculate all RPC parameters. To overcome the memory shortage problem while running the large size of satellite images, we also propose an RPC integration method. Image space is partitioned to multiple tiles, and RPC estimation is performed independently in each tile. Then, all tiles’ RPCs are integrated into the final RPC to represent the geometry of the whole image space. Finally, the integrated RPC is used to run a true MVSS pipeline to obtain the 3D DSM. The experimental results show that the proposed method can achieve 1.455 m Mean Absolute Error (MAE) in the height map reconstruction from multi-view satellite benchmark datasets. We also show that the proposed method can be used to reconstruct a geo-referenced 3D DSM from uncalibrated and freely available Google Earth imagery. Full article

The 3D reconstruction of point cloud trees and the acquisition of stand factors are key to supporting forestry regulation and urban planning. However, the two are usually independent modules in existing studies. In this work, we extended the AdTree method for 3D modeling of trees by adding a quantitative analysis capability to acquire stand factors. We used unmanned aircraft LiDAR (ALS) data as the raw data for this study. After denoising the data and segmenting the single trees, we obtained the single-tree samples needed for this study and produced our own single-tree sample dataset. The scanned tree point cloud was reconstructed in three dimensions in terms of geometry and topology, and important stand parameters in forestry were extracted. This improvement in the quantification of model parameters significantly improves the utility of the original point cloud tree reconstruction algorithm and increases its ability for quantitative analysis. The tree parameters obtained by this improved model were validated on 82 camphor pine trees sampled from the Northeast Forestry University forest. In a controlled experiment with the same field-measured parameters, the root mean square errors (RMSEs) and coefficients of determination (R²s) for diameters at breast height (DBHs) and crown widths (CWs) were 4.1 cm and 0.63, and 0.61 m and 0.74, and the RMSEs and coefficients of determination (R²s) for heights at tree height (THs) and crown base heights (CBHs) were 0.55 m and 0.85, and 1.02 m and 0.88, respectively. The overall effect of the canopy volume extracted based on the alpha shape is closest to the original point cloud and best estimated when alpha = 0.3. Full article

In practical applications, integrating three-dimensional models of bionic devices with simulation systems can predict their behavior and performance under various operating conditions, providing a basis for subsequent engineering optimization and improvements. This study proposes a framework for characterizing three-dimensional models of objects, focusing on extracting 3D structures and generating high-quality 3D models. The core concept involves obtaining the density output of the model from multiple images to enable adaptive boundary surface detection. The framework employs a hierarchical octree structure to partition the 3D space based on surface and geometric complexity. This approach includes recursive encoding and decoding of the octree structure and surface geometry, ultimately leading to the reconstruction of the 3D model. The framework has been validated through a series of experiments, yielding positive results. Full article

Short-range MEMS-based (Micro Electronical Mechanical System) LiDAR provides precise point cloud datasets for rock fragment surfaces. However, there is more vibrational noise in MEMS-based LiDAR signals, which cannot guarantee that the reconstructed point cloud data are not distorted with a high compression ratio. Many studies have illustrated that wavelet-based clustered compressive sensing can improve reconstruction precision. The k-means clustering algorithm can be conveniently employed to obtain clusters; however, estimating a meaningful k value (i.e., the number of clusters) is challenging. An excessive quantity of clusters is not necessary for dense point clouds, as this leads to elevated consumption of memory and CPU resources. For sparser point clouds, fewer clusters lead to more distortions, while excessive clusters lead to more voids in reconstructed point clouds. This study proposes a local clustering method to determine a number of clusters closer to the actual number based on GMM (Gaussian Mixture Model) observation distances and density peaks. Experimental results illustrate that the estimated number of clusters is closer to the actual number in four datasets from the KEEL public repository. In point cloud compression and recovery experiments, our proposed approach compresses and recovers the Bunny and Armadillo datasets in the Stanford 3D repository; the experimental results illustrate that our proposed approach improves reconstructed point clouds’ geometry and curvature similarity. Furthermore, the geometric similarity increases to 0.9 above in our complete rock fragment surface datasets after selecting a better wavelet basis for each dimension of MEMS-based LiDAR signals. In both experiments, the sparsity of signals was 0.8 and the sampling ratio was 0.4. Finally, a rock outcrop point cloud data experiment is utilized to verify that the proposed approach is applicable for large-scale research objects. All of our experiments illustrate that the proposed adaptive clustered compressive sensing approach can better reconstruct MEMS-based LiDAR point clouds with a lower sampling ratio. Full article

Humans inherently perceive 3D scenes using prior knowledge and visual perception, but 3D reconstruction in computer graphics is challenging due to complex object geometries, noisy backgrounds, and occlusions, leading to high time and space complexity. To addresses these challenges, this study introduces 3DRecNet, a compact 3D reconstruction architecture optimized for both efficiency and accuracy through five key modules. The first module, the Human-Inspired Memory Network (HIMNet), is designed for initial point cloud estimation, assisting in identifying and localizing objects in occluded and complex regions while preserving critical spatial information. Next, separate image and 3D encoders perform feature extraction from input images and initial point clouds. These features are combined using a dual attention-based feature fusion module, which emphasizes features from the image branch over those from the 3D encoding branch. This approach ensures independence from proposals at inference time and filters out irrelevant information, leading to more accurate and detailed reconstructions. Finally, a Decoder Branch transforms the fused features into a 3D representation. The integration of attention-based fusion with the memory network in 3DRecNet significantly enhances the overall reconstruction process. Experimental results on the benchmark datasets, such as ShapeNet, ObjectNet3D, and Pix3D, demonstrate that 3DRecNet outperforms existing methods. Full article

We introduce a high-throughput 3D scanning system designed to accurately measure cattle phenotypes. This scanner employs an array of depth sensors, i.e., time-of-flight (ToF) sensors, each controlled by dedicated embedded devices. The sensors generate high-fidelity 3D point clouds, which are automatically stitched using a point could segmentation approach through deep learning. The deep learner combines raw RGB and depth data to identify correspondences between the multiple 3D point clouds, thus creating a single and accurate mesh that reconstructs the cattle geometry on the fly. In order to evaluate the performance of our system, we implemented a two-fold validation process. Initially, we quantitatively tested the scanner for its ability to determine accurate volume and surface area measurements in a controlled environment featuring known objects. Next, we explored the impact and need for multi-device synchronization when scanning moving targets (cattle). Finally, we performed qualitative and quantitative measurements on cattle. The experimental results demonstrate that the proposed system is capable of producing high-quality meshes of untamed cattle with accurate volume and surface area measurements for livestock studies. Full article

Infrared binocular cameras, leveraging their distinct thermal imaging capabilities, are well-suited for visual measurement and 3D reconstruction in challenging environments. The precision of camera calibration is essential for leveraging the full potential of these infrared cameras. To overcome the limitations of traditional calibration techniques, a novel method for calibrating infrared binocular cameras is introduced. By creating a virtual target plane that closely mimics the geometry of the real target plane, the method refines the feature point coordinates, leading to enhanced precision in infrared camera calibration. The virtual target plane is obtained by inverse projecting the centers of the imaging ellipses, which are estimated at sub-pixel edge, into three-dimensional space, and then optimized using the RANSAC least squares method. Subsequently, the imaging ellipses are inversely projected onto the virtual target plane, where its centers are identified. The corresponding world coordinates of the feature points are then refined through a linear optimization process. These coordinates are reprojected onto the imaging plane, yielding optimized pixel feature points. The calibration procedure is iteratively performed to determine the ultimate set of calibration parameters. The method has been validated through experiments, demonstrating an average reprojection error of less than 0.02 pixels and a significant 24.5% improvement in calibration accuracy over traditional methods. Furthermore, a comprehensive analysis has been conducted to identify the primary sources of calibration error. Ultimately, this achieves an error rate of less than 5% in infrared stereo ranging within a 55-m range. Full article

Atrial fibrillation (AF) is the most common sustained cardiac arrhythmia associated with significant morbidity and mortality. Managing risk of stroke and AF burden are pillars of AF management. Atrial geometry has long been recognized as a useful measure in achieving these goals. However, traditional diagnostic approaches often overlook the complex spatial dynamics of the atria. This review explores the emerging role of three-dimensional (3D) atrial geometry in the evaluation and management of AF. Advancements in imaging technologies and computational modeling have enabled detailed reconstructions of atrial anatomy, providing insights into the pathophysiology of AF that were previously unattainable. We examine current methodologies for interpreting 3D atrial data, including qualitative, basic quantitative, global quantitative, and statistical shape modeling approaches. We discuss their integration into clinical practice, highlighting potential benefits such as personalized treatment strategies, improved outcome prediction, and informed treatment approaches. Additionally, we discuss the challenges and limitations associated with current approaches, including technical constraints and variable interpretations, and propose future directions for research and clinical applications. This comprehensive review underscores the transformative potential of leveraging 3D atrial geometry in the evaluation and management of AF, advocating for its broader adoption in clinical practice. Full article

Three-dimensional echocardiography (3D ECHO) and magnetic resonance (MR) imaging are frequently used in patients and animals to evaluate heart functions. Inverse finite element (FE) modeling is increasingly applied to MR images to quantify left ventricular (LV) function and estimate myocardial contractility and other cardiac biomarkers. It remains unclear, however, as to whether myocardial contractility derived from the inverse FE model based on 3D ECHO images is comparable to that derived from MR images. To address this issue, we developed a subject-specific inverse FE model based on 3D ECHO and MR images acquired from seven healthy swine models to investigate if there are differences in myocardial contractility and LV geometrical features derived using these two imaging modalities. We showed that end-systolic and end-diastolic volumes derived from 3D ECHO images are comparable to those derived from MR images ($R^2=0.805$ and $0.969$, respectively). As a result, ejection fraction from 3D ECHO and MR images are linearly correlated ($R^2=0.977$) with the limit of agreement (LOA) ranging from −17.95% to 45.89%. Using an inverse FE modeling to fit pressure and volume waveforms in subject-specific LV geometry reconstructed from 3D ECHO and MR images, we found that myocardial contractility derived from these two imaging modalities are linearly correlated with an $R^2$ value of 0.989, a gradient of 0.895, and LOA ranging from −6.11% to 36.66%. This finding supports using 3D ECHO images in image-based inverse FE modeling to estimate myocardial contractility. Full article

Proper documentation is fundamental to providing structural health monitoring, damage identification and failure assessment for Cultural Heritage (CH). Three-dimensional models from photogrammetric and laser scanning surveys usually provide 3D point clouds that can be converted into meshes. The point clouds usually contain noise data due to different causes: non-cooperative material or surfaces, bad lighting, complex geometry and low accuracy of the instruments utilized. Point cloud denoising has become one of the hot topics of 3D geometric data processing, removing these noise data to recover the ground-truth point cloud and adding smoothing to the ideal surface. These cleaned point clouds can be converted in volumes with different algorithms, suitable for different uses, mainly for structural analysis. This paper aimed to analyse the geometric accuracy of algorithms available for the conversion of 3D point clouds into volumetric models that can be used for structural analyses through the FEA process. The process is evaluated, highlighting problems and difficulties that lie in poor reconstruction results of volumes from denoised point clouds due to the geometric complexity of the objects. Full article