Search Results (100)

This study examines how different filming techniques can enhance the quality of 3D reconstructions with a particular focus on their use in indoor crime scene investigations. Using Neural Radiance Fields (NeRF) and Gaussian Splatting, we explored how factors like camera orientation, filming speed, data layering, and scanning path affect the detail and clarity of 3D reconstructions. Through experiments in a mock crime scene apartment, we identified optimal filming methods that reduce noise and artifacts, delivering clearer and more accurate reconstructions. Filming in landscape mode, at a slower speed, with at least three layers and focused on key objects produced the most effective results. These insights provide valuable guidelines for professionals in forensics, architecture, and cultural heritage preservation, helping them capture realistic high-quality 3D representations. This study also highlights the potential for future research to expand on these findings by exploring other algorithms, camera parameters, and real-time adjustment techniques. Full article

Recent advancements in 3D scene representation have underscored the potential of Neural Radiance Fields (NeRFs) for producing high-fidelity renderings of complex scenes. However, NeRFs are hindered by the significant computational burden of volumetric rendering. To address this, 3D Gaussian Splatting (3DGS) has emerged as an efficient alternative, utilizing Gaussian-based representations and rasterization techniques to achieve faster rendering speeds without sacrificing image quality. Despite these advantages, the large number of Gaussian points and associated internal parameters result in high storage demands. To address this challenge, we propose a pruning strategy applied during the Gaussian densification and pruning phases. Our approach integrates learnable Gaussian masks with a contribution-based pruning mechanism, further enhanced by an opacity update strategy to facilitate the pruning process. This method effectively eliminates redundant Gaussian points and those with minimal contributions to scene construction. Additionally, during the Gaussian parameter compression phase, we employ a combination of teacher–student models and vector quantization to compress the spherical harmonic coefficients. Extensive experimental results demonstrate that our approach reduces the storage requirements of original 3D Gaussian models by over 30 times, with only a minor degradation in rendering quality. Full article

High-fidelity driving scenario reconstruction can generate a lot of realistic virtual simulation environment samples, which can support effective training and testing for autonomous vehicles. Neural radiance fields (NeRFs) have demonstrated their excellence in high-fidelity scenario reconstruction; however, they still rely on dense-view data and precise camera poses, which are difficult to obtain in autonomous vehicles. To address the above issues, we propose a novel approach called SRNeRF, which can eliminate pose-based operations and perform scenario reconstruction from sparse views. To extract more scene knowledge from limited views, we incorporate an image super-resolution module based on a fully convolutional neural network and introduce a new texture loss to capture scene details for higher-quality scene reconstruction. On both object-centric and scene-level datasets, SRNeRF performs comparably to previous methods with ground truth poses and significantly outperforms methods with predicted poses, with a PSNR improvement of about 30%. Finally, we evaluate SRNeRF on our custom autonomous driving dataset, and the results show that SRNeRF can still generate stable images and novel views in the face of sparse views, demonstrating its scalability in autonomous driving scenario synthesis. Full article

This article investigates the continuity of derivatives of real-valued functions from a topological perspective. This is achieved by the characterization of their sets of discontinuity. The same principle is applied to Gateaux derivatives and gradients in Euclidean spaces. This article also introduces a generalization of the derivatives from the perspective of the modulus of continuity and characterizes their sets of discontinuities. There is a need for such generalizations when dealing with physical phenomena, such as fractures, shock waves, turbulence, Brownian motion, etc. Full article

The advancement of neural radiance fields (NeRFs) has facilitated the high-quality 3D reconstruction of complex scenes. However, for most NeRFs, reconstructing 3D tissues from endoscopy images poses significant challenges due to the occlusion of soft tissue regions by invalid pixels, deformations in soft tissue, and poor image quality, which severely limits their application in endoscopic scenarios. To address the above issues, we propose a novel framework to reconstruct high-fidelity soft tissue scenes from low-quality endoscopic images. We first construct an EndoTissue dataset of soft tissue regions in endoscopic images and fine-tune the Segment Anything Model (SAM) based on EndoTissue to obtain a potent segmentation network. Given a sequence of monocular endoscopic images, this segmentation network can quickly obtain the tissue mask images. Additionally, we incorporate tissue masks into a dynamic scene reconstruction method called Tensor4D to effectively guide the reconstruction of 3D deformable soft tissues. Finally, we propose adopting the image enhancement model EDAU-Net to improve the quality of the rendered views. The experimental results show that our method can effectively focus on the soft tissue regions in the image, achieving higher fidelity in detail and geometric structural integrity in reconstruction compared to state-of-the-art algorithms. Feedback from the user study indicates high participant scores for our method. Full article

Forest mapping provides critical observational data needed to understand the dynamics of forest environments. Notably, tree diameter at breast height (DBH) is a metric used to estimate forest biomass and carbon dioxide (${\mathrm{CO}}_2$) sequestration. Manual methods of forest mapping are labor intensive and time consuming, a bottleneck for large-scale mapping efforts. Automated mapping relies on acquiring dense forest reconstructions, typically in the form of point clouds. Terrestrial laser scanning (TLS) and mobile laser scanning (MLS) generate point clouds using expensive LiDAR sensing and have been used successfully to estimate tree diameter. Neural radiance fields (NeRFs) are an emergent technology enabling photorealistic, vision-based reconstruction by training a neural network on a sparse set of input views. In this paper, we present a comparison of MLS and NeRF forest reconstructions for the purpose of trunk diameter estimation in a mixed-evergreen Redwood forest. In addition, we propose an improved DBH-estimation method using convex-hull modeling. Using this approach, we achieved 1.68 cm RMSE (2.81%), which consistently outperformed standard cylinder modeling approaches. Full article

The evolution of indoor skydiving from military origins to a globally recreational pursuit has attracted individuals without prior specific training, exposing them to the heightened risk of muscle–joint injuries associated with indoor skydiving. This study aims to assess the muscular excitation patterns in highly skilled instructors to determine the optimal activation strategies for preventing musculoskeletal injuries. Nine expert indoor skydiving instructors (M/F: 8/1; age 31 ± 7 years; body mass: 70.5 ± 11.1 kg; stature: 1.74 ± 0.09 m) were enrolled. The surface electromyographic signal was recorded on the anterior deltoid, AD; posterior deltoid, PD; pectoralis major, PM; latissimus dorsi, LD; rectus abdominis, Rabd; erector spinae, ES; rectus femoris, RF; and biceps femoris, BF for each position: PRONE, SUPINE, SIT, and head-down (HD). A synchronous heart rate was recorded to assess the cardiac engagement. The neuromuscular load (NeLo), representative of the excitation amplitude of a muscle set, was determined for each position across different body regions. PRONE and SUPINE were the least demanding in terms of cardiac engagement (67 ± 6 and 85 ± 8 bpm, respectively) and exhibited the lowest neuromuscular excitation (24 ± 10 and 24 ± 8% Max, respectively). By contrast, HD exhibited the highest cardiac (127 + 18 bpm) and neuromuscular (71 + 11% Max) engagement and particularly in the lower-trunk and lower-limbs muscles. SIT predominantly engaged upper-trunk and shoulder muscles. The findings indicate the significant engagement of the musculature surrounding the shoulder joint and that responsible for lower-trunk stabilization in maintaining the investigated positions. A targeted training regimen on strengthening these muscles is advised before practicing indoor skydiving to prevent shoulder joint injuries or lower back muscle overloads. Full article

This study addresses the prevalent challenges of inefficiency and suboptimal quality in indoor 3D scene generation and rendering by proposing a parameter-tuning strategy for 3D Gaussian Splatting (3DGS). Through a systematic quantitative analysis of various performance indicators under differing resolution conditions, threshold settings for the average magnitude of spatial position gradients, and adjustments to the scaling learning rate, the optimal parameter configuration for the 3DGS model, specifically tailored for indoor modeling scenarios, is determined. Firstly, utilizing a self-collected dataset, a comprehensive comparison was conducted among COLLI-SION-MAPping (abbreviated as COLMAP (V3.7), an open-source software based on Structure from Motion and Multi-View Stereo (SFM-MVS)), Context Capture (V10.2) (abbreviated as CC, a software utilizing oblique photography algorithms), Neural Radiance Fields (NeRF), and the currently renowned 3DGS algorithm. The key dimensions of focus included the number of images, rendering time, and overall rendering effectiveness. Subsequently, based on this comparison, rigorous qualitative and quantitative evaluations are further conducted on the overall performance and detail processing capabilities of the 3DGS algorithm. Finally, to meet the specific requirements of indoor scene modeling and rendering, targeted parameter tuning is performed on the algorithm. The results demonstrate significant performance improvements in the optimized 3DGS algorithm: the PSNR metric increases by 4.3%, and the SSIM metric improves by 0.2%. The experimental results prove that the improved 3DGS algorithm exhibits superior expressive power and persuasiveness in indoor scene rendering. Full article

Audio-driven cross-modal talking head generation has experienced significant advancement in the last several years, and it aims to generate a talking head video that corresponds to a given audio sequence. Out of these approaches, the NeRF-based method can generate videos featuring a specific person with more natural motion compared to the one-shot methods. However, previous approaches failed to distinguish the importance of different regions, resulting in the loss of information-rich region features. To alleviate the problem and improve video quality, we propose MLDF-NeRF, an end-to-end method for talking head generation, which can achieve better vector representation through multi-level feature dynamic fusion. Specifically, we designed two modules in MLDF-NeRF to enhance the cross-modal mapping ability between audio and different facial regions. We initially developed a multi-level tri-plane hash representation that uses three sets of tri-plane hash networks with varying resolutions of limitation to capture the dynamic information of the face more accurately. Then, we introduce the idea of multi-head attention and design an efficient audio-visual fusion module that explicitly fuses audio features with image features from different planes, thereby improving the mapping between audio features and spatial information. Meanwhile, the design helps to minimize interference from facial areas unrelated to audio, thereby improving the overall quality of the representation. The quantitative and qualitative results indicate that our proposed method can effectively generate talk heads with natural actions and realistic details. Compared with previous methods, it performs better in terms of image quality, lip sync, and other aspects. Full article

Novel view synthesis aims to generate new perspectives from a limited number of input views. Neural Radiance Field (NeRF) is a key method for this task, and it produces high-fidelity images from a comprehensive set of inputs. However, a NeRF’s performance drops significantly with sparse views. To mitigate this, depth information can be used to guide training, with coarse depth maps often readily available in practical settings. We propose an improved sparse view NeRF model, ATGANNeRF, which integrates an enhanced U-Net generator with a dual-discriminator framework, CBAM, and Multi-Head Self-Attention mechanisms. The symmetric design enhances the model’s ability to capture and preserve spatial relationships, ensuring a more consistent generation of novel views. Additionally, local depth ranking is employed to ensure depth consistency with coarse maps, and spatial continuity constraints are introduced to synthesize novel views from sparse samples. SSIM loss is also added to preserve local structural details like edges and textures. Evaluation on LLFF, DTU, and our own datasets shows that ATGANNeRF significantly outperforms state-of-the-art methods, both quantitatively and qualitatively. Full article

Paranasal sinus surgery, a common treatment for chronic rhinosinusitis, requires exceptional precision due to the proximity of critical anatomical structures. To ensure accurate instrument control and clear visualization of the surgical site, surgeons utilize computer-aided surgical navigation (CSN). A key component of CSN is the registration process, which is traditionally reliant on manual or marker-based techniques. However, there is a growing shift toward marker-less registration methods. In previous work, we investigated a mesh-based registration approach using a Mixed Reality Head-Mounted Display (MR-HMD), specifically the Microsoft HoloLens 2. However, this method faced limitations, including depth holes and invalid values. These issues stemmed from the device’s low-resolution camera specifications and the 3D projection steps required to upscale RGB camera spaces. In this study, we propose a novel automated marker-less registration method leveraging Neural Radiance Field (NeRF) technology with an MR-HMD. To address insufficient depth information in the previous approach, we utilize rendered-depth images generated by the trained NeRF model. We evaluated our method against two other techniques, including prior mesh-based registration, using a facial phantom and three participants. The results demonstrate our proposed method achieves at least a 0.873 mm (12%) improvement in registration accuracy compared to others. Full article

Maize lodging is a prevalent stress that can significantly diminish corn yield and quality. Unmanned aerial vehicles (UAVs) remote sensing is a practical means to quickly obtain lodging information at field scale, such as area, severity, and distribution. However, existing studies primarily use machine learning (ML) methods to qualitatively analyze maize lodging (lodging and non-lodging) or estimate the maize lodging percentage, while there is less research using deep learning (DL) to quantitatively estimate maize lodging parameters (type, severity, and direction). This study aims to introduce advanced DL algorithms into the maize lodging classification task using UAV-multispectral images and investigate the advantages of DL compared with traditional ML methods. This study collected a UAV-multispectral dataset containing non-lodging maize and lodging maize with different lodging types, severities, and directions. Additionally, 22 vegetation indices (VIs) were extracted from multispectral data, followed by spatial aggregation and image cropping. Five ML classifiers and three DL models were trained to classify the maize lodging parameters. Finally, we compared the performance of ML and DL models in evaluating maize lodging parameters. The results indicate that the Random Forest (RF) model outperforms the other four ML algorithms, achieving an overall accuracy (OA) of 89.29% and a Kappa coefficient of 0.8852. However, the maize lodging classification performance of DL models is significantly better than that of ML methods. Specifically, Swin-T performs better than ResNet-50 and ConvNeXt-T, with an OA reaching 96.02% and a Kappa coefficient of 0.9574. This can be attributed to the fact that Swin-T can more effectively extract detailed information that accurately characterizes maize lodging traits from UAV-multispectral data. This study demonstrates that combining DL with UAV-multispectral data enables a more comprehensive understanding of maize lodging type, severity, and direction, which is essential for post-disaster rescue operations and agricultural insurance claims. Full article

Accurate collection of plant phenotyping is critical to optimising sustainable farming practices in precision agriculture. Traditional phenotyping in controlled laboratory environments, while valuable, falls short in understanding plant growth under real-world conditions. Emerging sensor and digital technologies offer a promising approach for direct phenotyping of plants in farm environments. This study investigates a learning-based phenotyping method using neural implicit surfaces reconstruction to achieve accurate in situ phenotyping of pepper plants in greenhouse environments. To quantitatively evaluate the performance of this method, traditional point cloud registration on 3D scanning data is implemented for comparison. Experimental result shows that NIR (neural implicit surfaces reconstruction) achieves competitive accuracy compared to the 3D scanning method. The mean distance error between the scanner-based method and the NeRF (neural radiance fields)-based method is 0.811 mm. This study shows that the learning-based NeRF method has similar accuracy to the 3D scanning-based method but with greater scalability and faster deployment capabilities. Full article

Since conventional photogrammetric approaches struggle with with low-texture, reflective, and transparent regions, this study explores the application of Neural Radiance Fields (NeRFs) for large-scale 3D reconstruction of outdoor scenes, since NeRF-based methods have recently shown very impressive results in these areas. We evaluate three approaches: Mega-NeRF, Block-NeRF, and Direct Voxel Grid Optimization, focusing on their accuracy and completeness compared to ground truth point clouds. In addition, we analyze the effects of using multiple sub-modules, estimating the visibility by an additional neural network and varying the density threshold for the extraction of the point cloud. For performance evaluation, we use benchmark datasets that correspond to the setting off standard flight campaigns and therefore typically have nadir camera perspective and relatively little image overlap, which can be challenging for NeRF-based approaches that are typically trained with significantly more images and varying camera angles. We show that despite lower quality compared to classic photogrammetric approaches, NeRF-based reconstructions provide visually convincing results in challenging areas. Furthermore, our study shows that in particular increasing the number of sub-modules and predicting the visibility using an additional neural network improves the quality of the resulting reconstructions significantly. Full article

Accurate classification of three-dimensional (3D) point clouds in real-world environments is often impeded by sensor noise, occlusions, and incomplete data. To overcome these challenges, we propose SMCNet, a robust multimodal framework for 3D point cloud classification. SMCNet combines multi-view projection and neural radiance fields (NeRFs) to generate high-fidelity 2D representations with enhanced texture realism, addressing occlusions and lighting inconsistencies effectively. The Mamba model is further refined within this framework by integrating a depth perception module to capture long-range point interactions and adopting a dual-channel structure to enhance point-wise feature extraction. Fine-tuning adapters for the CLIP and Mamba models are also introduced, significantly improving cross-domain adaptability. Additionally, an intelligent voting mechanism aggregates predictions from multiple viewpoints, ensuring enhanced classification robustness. Comprehensive experiments demonstrate that SMCNet achieves state-of-the-art performance, outperforming the PointNet++ baseline with a 0.5% improvement in mean overall accuracy (mOA) on ModelNet40 and a 7.9% improvement on ScanObjectNN. In corruption resistance, SMCNet reduces the mean corruption error (mCE) by 0.8% on ModelNet40-C and 3.6% on ScanObjectNN-C. These results highlight the effectiveness of SMCNet in tackling real-world classification scenarios with noisy and corrupted data. Full article