Abstract
Point clouds captured by 3D scanning are usually sparse and noisy. Reconstructing a high-resolution 3D model of an object is a challenging task in computer vision. Recent point cloud upsampling approaches aim to generate a dense point set, while achieving both distribution uniformity and proximity-to-surface directly via an end-to-end network. Although dense reconstruction from low to high resolution can be realized by using these techniques, it lacks abundant details for dense outputs. In this work, we propose a coarse-to-fine network PUGL-Net for point cloud reconstruction that first predicts a coarse high-resolution point cloud via a global dense reconstruction module and then increases the details by aggregating local point features. On the one hand, a transformer-based mechanism is designed in the global dense reconstruction module. It aggregates residual learning in a self-attention scheme for effective global feature extraction. On the other hand, the coordinate offset of points is learned in a local refinement module. It further refines the coarse points by aggregating KNN features. Evaluated through extensive quantitative and qualitative evaluation on synthetic data set, the proposed coarse-to-fine architecture generates point clouds that are accurate, uniform and dense, it outperforms most existing state-of-the-art point cloud reconstruction works.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chen, X., Ma, H., Wan, J., Li, B., Xia, T.: Multi-view 3D object detection network for autonomous driving. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017, pp. 6526–6534 (2017). https://doi.org/10.1109/CVPR.2017.691
Cole, D.M., Newman, P.M.: Using laser range data for 3D SLAM in outdoor environments. In: Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006, ICRA 2006, 2006, pp. 1556–1563 (2006). https://doi.org/10.1109/ROBOT.2006.1641929
Orts-Escolano, S., et al.: Holoportation: virtual 3D teleportation in real-time. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, pp. 741–754, January 2016
Alexa, M., Behr, J., Cohen-Or, D., Fleishman, S., Levin, D., Silva, C.T.: Computing and rendering point set surfaces. IEEE Trans. Vis. Comput. Graph. 9(1), 3–15 (2003). https://doi.org/10.1109/TVCG.2003.1175093
Huang, H., Wu, S., Gong, M., Cohen-Or, D., Ascher, U., (Richard) Zhang, H.: Edge-aware point set resampling. ACM Trans. Graph. 32(1), Article 9 (2013)
Lipman, Y., Cohen-Or, D., Levin, D., Tal-Ezer, H.: Parameterization-free projection for geometry reconstruction. In: ACM SIGGRAPH 2007 Papers (SIGGRAPH 2007) (2007)
Wu, S., Huang, H., Gong, M., Zwicker, M., Cohen-Or, D.: Deep points consolidation. ACM Trans. Graph. 34(6), Article 176 (2015)
Preiner, R., Mattausch, O., Arikan, M., Pajarola, R., Wimmer, M.: Continuous projection for fast L1 reconstruction. ACM Trans. Graph. 33(4), Article 47 (2014)
Yu, L., Li, X., Fu, C.-W., Cohen-Or, D., Heng, P.-A.: EC-net: an edge-aware point set consolidation network. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 398–414. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_24
Yu, L., Li, X., Fu, C., Cohen-Or, D., Heng, P.: PU-Net: point cloud upsampling network. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition 2018, pp. 2790–2799 (2018). https://doi.org/10.1109/CVPR.2018.00295
Yifan, W., Wu, S., Huang, H., Cohen-Or, D., Sorkine-Hornung, O.: Patch-based progressive 3D point set upsampling. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2019, pp. 5951–5960 (2019). https://doi.org/10.1109/CVPR.2019.00611
Li, R., Li, X., Fu, C., Cohen-Or, D., Heng, P.: PU-GAN: a point cloud upsampling adversarial network. In: IEEE/CVF International Conference on Computer Vision (ICCV) 2019, pp. 7202–7211 (2019). https://doi.org/10.1109/ICCV.2019.00730
Qian, Y., Hou, J., Kwong, S., He, Y.: PUGeo-Net: a geometry-centric network for 3D point cloud upsampling. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12364, pp. 752–769. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58529-7_44
Qian, G., Abualshour, A., Li, G., Thabet, A., Ghanem, B.: PU-GCN: point cloud upsampling using graph convolutional networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11683–11692, June 2021
Li, R., Li, X., Heng, P.-A., Fu, C.-W.: Point cloud upsampling via disentangled refinement. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2021)
Guo, M.-H.: PCT: point cloud transformer. Computational Visual Media, pp. 187–199 (2021)
Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016, pp. 1874–1883 (2016). https://doi.org/10.1109/CVPR.2016.207
Goodfellow, I.J., et al.: Generative Adversarial Networks (2014). arXiv:1406.2661 [stat.ML]
Vaswani, A.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Kingma, D., Ba. Adam, J.: A method for stochastic optimization. In: International Conference on Learning Representations (ICLR) (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, Y., Wang, S., Sun, L. (2022). Point Cloud Upsampling via a Coarse-to-Fine Network. In: Þór Jónsson, B., et al. MultiMedia Modeling. MMM 2022. Lecture Notes in Computer Science, vol 13141. Springer, Cham. https://doi.org/10.1007/978-3-030-98358-1_37
Download citation
DOI: https://doi.org/10.1007/978-3-030-98358-1_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-98357-4
Online ISBN: 978-3-030-98358-1
eBook Packages: Computer ScienceComputer Science (R0)