Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Conditional GAN for Point Cloud Generation

  • Conference paper
  • First Online:
Computer Vision – ACCV 2022 (ACCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13847))

Included in the following conference series:

  • 555 Accesses

Abstract

Recently, 3D data generation problems have attracted more and more research attention and have been addressed through various approaches. However, most of them fail to generate objects with given desired categories and tend to produce hybrids of multiple types. Thus, this paper proposes a generative model for synthesizing high-quality point clouds with conditional information, which is called Point Cloud conditional Generative Adversarial Network (PC-cGAN). The generative model of the proposed PC-cGAN consists of two main components: a pre-generator to generate rough point clouds and a conditional modifier to refine the last outputs with specific categories. To improve the performance for multi-class conditional generation for point clouds, an improved tree-structured graph convolution network, called BranchGCN, is adopted to aggregate information from both ancestor and neighbor features. Experimental results demonstrate that the proposed PC-cGAN outperforms state-of-the-art GANs in terms of conventional distance metrics and novel latent metric, Frechet Point Distance, and avoids the intra-category hybridization problem and the unbalanced issue in generated sample distribution effectively. The results also show that PC-cGAN enables us to gain explicit control over the object category while maintaining good generation quality and diversity. The implementation of PC-cGAN is available at https://github.com/zlyang3/PC-cGAN.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Achlioptas, P., Diamanti, O., Mitliagkas, I., Guibas, L.: Learning representations and generative models for 3D point clouds. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 80, pp. 40–49. PMLR (2018). https://proceedings.mlr.press/v80/achlioptas18a.html

  2. Alexiou, E., Yang, N., Ebrahimi, T.: PointXR: a toolbox for visualization and subjective evaluation of point clouds in virtual reality. In: 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX). IEEE (2020). https://doi.org/10.1109/qomex48832.2020.9123121

  3. Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein GAN. arXiv e-prints arXiv:1701.07875 (2017)

  4. Arshad, M.S., Beksi, W.J.: A progressive conditional generative adversarial network for generating dense and colored 3D point clouds (2020). https://doi.org/10.1109/3dv50981.2020.00081

  5. Bruna, J., Zaremba, W., Szlam, A., LeCun, Y.: Spectral networks and locally connected networks on graphs. In: Bengio, Y., LeCun, Y. (eds.) 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, 14–16 April 2014, Conference Track Proceedings (2014). http://arxiv.org/abs/1312.6203

  6. Chang, A.X., et al.: ShapeNet: an information-rich 3D model repository. Technical report. arXiv:1512.03012, Stanford University – Princeton University – Toyota Technological Institute at Chicago (2015)

  7. Charles, R.Q., Su, H., Kaichun, M., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2017). https://doi.org/10.1109/cvpr.2017.16

  8. Choi, Y., Choi, M., Kim, M., Ha, J., Kim, S., Choo, J.: StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, 18–22 June 2018, pp. 8789–8797. Computer Vision Foundation/IEEE Computer Society (2018). https://doi.org/10.1109/CVPR.2018.00916. http://openaccess.thecvf.com/content_cvpr_2018/html/Choi_StarGAN_Unified_Generative_CVPR_2018_paper.html

  9. Feng, M., Zhang, L., Lin, X., Gilani, S.Z., Mian, A.: Point attention network for semantic segmentation of 3D point clouds. Pattern Recognit. 107, 107446 (2020). https://doi.org/10.1016/j.patcog.2020.107446

  10. Goodfellow, I.J., et al.: Generative adversarial networks. CoRR abs/1406.2661 (2014). http://arxiv.org/abs/1406.2661

  11. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.C.: Improved training of Wasserstein GANs. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc. (2017). https://proceedings.neurips.cc/paper/2017/file/892c3b1c6dccd52936e27cbd0ff683d6-Paper.pdf

  12. Guo, M.-H., Cai, J.-X., Liu, Z.-N., Mu, T.-J., Martin, R.R., Hu, S.-M.: PCT: point cloud transformer. Comput. Vis. Media 7(2), 187–199 (2021). https://doi.org/10.1007/s41095-021-0229-5

    Article  Google Scholar 

  13. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a local nash equilibrium. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4–9 December 2017, Long Beach, CA, USA, pp. 6626–6637 (2017). https://proceedings.neurips.cc/paper/2017/hash/8a1d694707eb0fefe65871369074926d-Abstract.html

  14. Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4396–4405 (2019). https://doi.org/10.1109/CVPR.2019.00453

  15. Kingma, D.P., Welling, M.: Auto-encoding variational bayes. In: 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, 14–16 April 2014, Conference Track Proceedings (2014). http://arxiv.org/abs/1312.6114

  16. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, 24–26 April 2017, Conference Track Proceedings (2017). https://openreview.net/forum?id=SJU4ayYgl

  17. Li, R.; Li, X.; Hui, K.-H.; Fu, C.-W.: SP-GAN: sphere-guided 3D shape generation and manipulation. In: ACM Transactions on Graphics (Proc. SIGGRAPH), vol. 40. ACM (2021)

    Google Scholar 

  18. Li, Y., Bu, R., Sun, M., Wu, W., Di, X., Chen, B.: PointCNN: convolution on X-transformed points. In: Bengio, S., Wallach, H.M., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3–8 December 2018, Montréal, Canada, pp. 828–838 (2018). https://proceedings.neurips.cc/paper/2018/hash/f5f8590cd58a54e94377e6ae2eded4d9-Abstract.html

  19. Lim, S., Shin, M., Paik, J.: Point cloud generation using deep local features for augmented and mixed reality contents. In: 2020 IEEE International Conference on Consumer Electronics (ICCE). IEEE (2020). https://doi.org/10.1109/icce46568.2020.9043081

  20. Ma, K., Lu, F., Chen, X.: Robust planar surface extraction from noisy and semi-dense 3D point cloud for augmented reality. In: 2016 International Conference on Virtual Reality and Visualization (ICVRV). IEEE (2016). https://doi.org/10.1109/icvrv.2016.83

  21. Ma, X., Qin, C., You, H., Ran, H., Fu, Y.: Rethinking network design and local geometry in point cloud: a simple residual MLP framework. CoRR abs/2202.07123 (2022). https://arxiv.org/abs/2202.07123

  22. Mirza, M., Osindero, S.: Conditional generative adversarial nets. CoRR abs/1411.1784 (2014). http://arxiv.org/abs/1411.1784

  23. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learning on point sets in a metric space. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS 2017, pp. 5105–5114. Curran Associates Inc., Red Hook (2017)

    Google Scholar 

  24. Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. In: Bengio, Y., LeCun, Y. (eds.) 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, 2–4 May 2016, Conference Track Proceedings (2016). http://arxiv.org/abs/1511.06434

  25. Sagawa, H., Nagayoshi, H., Kiyomizu, H., Kurihara, T.: [POSTER] hands-free AR work support system monitoring work progress with point-cloud data processing. In: 2015 IEEE International Symposium on Mixed and Augmented Reality. IEEE (2015). https://doi.org/10.1109/ismar.2015.50

  26. Sarmad, M., Lee, H.J., Kim, Y.M.: RL-GAN-net: a reinforcement learning agent controlled GAN network for real-time point cloud shape completion. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2019). https://doi.org/10.1109/cvpr.2019.00605

  27. Shu, D., Park, S.W., Kwon, J.: 3D point cloud generative adversarial network based on tree structured graph convolutions. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE (2019). https://doi.org/10.1109/iccv.2019.00396

  28. Singh, P., Sadekar, K., Raman, S.: TreeGCN-ED: encoding point cloud using a tree-structured graph network. CoRR abs/2110.03170 (2021)

    Google Scholar 

  29. Tredinnick, R., Broecker, M., Ponto, K.: Progressive feedback point cloud rendering for virtual reality display. In: 2016 IEEE Virtual Reality (VR). IEEE (2016). https://doi.org/10.1109/vr.2016.7504773

  30. Triess, L.T., Bühler, A., Peter, D., Flohr, F.B., Zöllner, M.: Point cloud generation with continuous conditioning. In: International Conference on Artificial Intelligence and Statistics, AISTATS 2022, 28–30 March 2022, Virtual Event, pp. 4462–4481 (2022). https://proceedings.mlr.press/v151/triess22a.html

  31. Valsesia, D., Fracastoro, G., Magli, E.: Learning localized generative models for 3D point clouds via graph convolution. In: 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, 6–9 May 2019 (2019). https://openreview.net/forum?id=SJeXSo09FQ

  32. Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph CNN for learning on point clouds. ACM Trans. Graph. 38(5), 1–12 (2019). https://doi.org/10.1145/3326362

    Article  Google Scholar 

  33. Yu, X., Rao, Y., Wang, Z., Liu, Z., Lu, J., Zhou, J.: PoinTr: diverse point cloud completion with geometry-aware transformers. In: 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, 10–17 October 2021, pp. 12478–12487 (2021). https://doi.org/10.1109/ICCV48922.2021.01227

  34. Yu, Z., Zheng, X., Yang, Z., Lu, B., Li, X., Fu, M.: Interaction-temporal GCN: a hybrid deep framework for COVID-19 pandemic analysis. IEEE Open J. Eng. Med. Biol. 2, 97–103 (2021). https://doi.org/10.1109/ojemb.2021.3063890

    Article  Google Scholar 

  35. Yuan, W., Khot, T., Held, D., Mertz, C., Hebert, M.: PCN: point completion network. In: 2018 International Conference on 3D Vision, 3DV 2018, Verona, Italy, 5–8 September 2018, pp. 728–737. IEEE Computer Society (2018). https://doi.org/10.1109/3DV.2018.00088

  36. Zhao, H., Jiang, L., Jia, J., Torr, P.H.S., Koltun, V.: Point transformer. In: 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, 10–17 October 2021, pp. 16239–16248. IEEE (2021). https://doi.org/10.1109/ICCV48922.2021.01595

  37. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251 (2017). https://doi.org/10.1109/ICCV.2017.244

Download references

Acknowledgement

This work was supported by the National Natural Science Foundation of China (No. 61471229 and No. 61901116), the Natural Science Foundation of Guangdong Province (No. 2019A1515011950), the Guangdong Basic and Applied Basic Research Foundation (No. 2019A1515010789 and No. 2021A1515012289), and in part by the Key Field Projects of Colleges and Universities of Guangdong Province (No. 2020ZDZX3065), and in part by Shantou University Scientific Research Foundation for Talents under Grant NTF19031.

Author information

Authors and Affiliations

  1. Key Lab of Digital Signal and Image Processing of Guangdong Province, Shantou University, Shantou, 515063, China

    Zhulun Yang, Yijun Chen, Yadong Chang & Xutao Li

  2. Department of Electronic Engineering, Shantou University, Shantou, 515063, China

    Zhulun Yang, Yijun Chen, Yadong Chang & Xutao Li

  3. School of Mathematics and Big Data, Foshan University, Foshan, 52800, China

    Xianwei Zheng

Authors
  1. Zhulun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yijun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xianwei Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yadong Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xutao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xianwei Zheng .

Editor information

Editors and Affiliations

  1. University of Wollongong, Wollongong, NSW, Australia

    Lei Wang

  2. University of Bonn, Bonn, Germany

    Juergen Gall

  3. University of Adelaide, Adelaide, SA, Australia

    Tat-Jun Chin

  4. National Institute of Informatics, Tokyo, Japan

    Imari Sato

  5. Johns Hopkins University, Baltimore, MD, USA

    Rama Chellappa

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yang, Z., Chen, Y., Zheng, X., Chang, Y., Li, X. (2023). Conditional GAN for Point Cloud Generation. In: Wang, L., Gall, J., Chin, TJ., Sato, I., Chellappa, R. (eds) Computer Vision – ACCV 2022. ACCV 2022. Lecture Notes in Computer Science, vol 13847. Springer, Cham. https://doi.org/10.1007/978-3-031-26293-7_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-26293-7_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-26292-0

  • Online ISBN: 978-3-031-26293-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation