Nothing Special   »   [go: up one dir, main page]

Skip to main content

3D-Producer: A Hybrid and User-Friendly 3D Reconstruction System

  • Conference paper
  • First Online:
Artificial Intelligence (CICAI 2022)

Abstract

In this paper, we introduce a hybrid and user-friendly 3D reconstruction system for objects, which seamlessly integrates a deep neural network model for 3D shape reconstruction and a multi-band image blending algorithm for realistic texturing into a single encapsulated application, dubbed as 3D-Producer. Compared to expensive laser scanning devices, our 3D-Producer offers an economical and practical solution, where a textured 3D model for a target object can be easily created by general users with mobile devices, a physical turntable, and our application. An illustration video can be found here (https://github.com/winnechan/3D-Producer/blob/main/3D-Producer-Demo-480P.mp4).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/opencv/opencv.

  2. 2.

    https://github.com/alicevision/meshroom.

References

  1. Allène, C., Pons, J., Keriven, R.: Seamless image-based texture atlases using multi-band blending. In: ICPR, pp. 1–4. IEEE Computer Society (2008)

    Google Scholar 

  2. Baumberg, A.: Blending images for texturing 3d models. In: Rosin, P.L., Marshall, A.D. (eds.) BMVC, pp. 1–10 (2002)

    Google Scholar 

  3. Cernea, D.: OpenMVS: Multi-view stereo reconstruction library (2020). http://cdcseacave.github.io/openMVS

  4. Chen, X., Ma, H., Wan, J., Li, B., Xia, T.: Multi-view 3d object detection network for autonomous driving. In: CVPR, pp. 1907–1915 (2017)

    Google Scholar 

  5. Contributors, P.: Paddleseg, end-to-end image segmentation kit based on paddlepaddle. http://github.com/PaddlePaddle/PaddleSeg (2019)

  6. Deng, J., Pan, Y., Yao, T., Zhou, W., Li, H., Mei, T.: Single shot video object detector. IEEE Trans. Multimedia 23, 846–858 (2020)

    Article  Google Scholar 

  7. Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F.J., Marín-Jiménez, M.J.: Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognit. 47(6), 2280–2292 (2014)

    Google Scholar 

  8. Gkioxari, G., Johnson, J., Malik, J.: Mesh R-CNN. In: ICCV. IEEE (2019)

    Google Scholar 

  9. Griwodz, C., Gasparini, S., Calvet, L., Gurdjos, P., Castan, F., Maujean, B., Lillo, G.D., Lanthony, Y.: Alicevision Meshroom: an open-source 3D reconstruction pipeline. In: Proceedings of the 12th ACM Multimedia Systems Conference - MMSys 2021. ACM Press (2021)

    Google Scholar 

  10. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)

    Google Scholar 

  11. Li, Y., Yao, T., Pan, Y., Mei, T.: Contextual transformer networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. (2022)

    Google Scholar 

  12. Liu, Y., Chu, L., Chen, G., Wu, Z., Chen, Z., Lai, B., Hao, Y.: Paddleseg: a high-efficient development toolkit for image segmentation (2021)

    Google Scholar 

  13. Lombardi, S., Simon, T., Saragih, J.M., Schwartz, G., Lehrmann, A.M., Sheikh, Y.: Neural volumes: learning dynamic renderable volumes from images. ACM Trans. Graph. 38(4), 65:1–65:14 (2019)

    Google Scholar 

  14. Long, F., Qiu, Z., Pan, Y., Yao, T., Luo, J., Mei, T.: Stand-alone inter-frame attention in video models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3192–3201 (2022)

    Google Scholar 

  15. Lorensen, W.E., Cline, H.E.: Marching cubes: a high resolution 3d surface construction algorithm. In: Stone, M.C. (ed.) SIGGRAPH. ACM (1987)

    Google Scholar 

  16. Mescheder, L.M., Oechsle, M., Niemeyer, M., Nowozin, S., Geiger, A.: Occupancy networks: learning 3d reconstruction in function space. In: CVPR, pp. 4460–4470 (2019)

    Google Scholar 

  17. Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 405–421. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_24

    Chapter  Google Scholar 

  18. Pan, Y., Chen, Y., Bao, Q., Zhang, N., Yao, T., Liu, J., Mei, T.: Smart director: an event-driven directing system for live broadcasting. TOMM 17(4), 1–18 (2021)

    Article  Google Scholar 

  19. Pan, Y., Li, Y., Yao, T., Mei, T., Li, H., Rui, Y.: Learning deep intrinsic video representation by exploring temporal coherence and graph structure. In: IJCAI, pp. 3832–3838 (2016)

    Google Scholar 

  20. Pan, Y., Yao, T., Li, Y., Mei, T.: X-linear attention networks for image captioning. In: CVPR (2020)

    Google Scholar 

  21. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for 3d classification and segmentation. In: CVPR, pp. 652–660 (2017)

    Google Scholar 

  22. Rematas, K., Liu, A., Srinivasan, P.P., Barron, J.T., Tagliasacchi, A., Funkhouser, T.A., Ferrari, V.: Urban radiance fields. CoRR abs/2111.14643 (2021)

    Google Scholar 

  23. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2016)

    Article  Google Scholar 

  24. Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: CVPR (2016)

    Google Scholar 

  25. Schönberger, J.L., Zheng, E., Pollefeys, M., Frahm, J.M.: Pixelwise view selection for unstructured multi-view stereo. In: ECCV (2016)

    Google Scholar 

  26. Sitzmann, V., Thies, J., Heide, F., Nießner, M., Wetzstein, G., Zollhöfer, M.: Deepvoxels: Learning persistent 3d feature embeddings. In: CVPR, pp. 2437–2446 (2019)

    Google Scholar 

  27. Wang, P., Liu, L., Liu, Y., Theobalt, C., Komura, T., Wang, W.: Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. In: NeurIPS, pp. 27171–27183 (2021)

    Google Scholar 

  28. Yao, T., Li, Y., Pan, Y., Wang, Y., Zhang, X.P., Mei, T.: Dual vision transformer. arXiv preprint arXiv:2207.04976 (2022)

  29. Yao, T., Pan, Y., Li, Y., Ngo, C.W., Mei, T.: Wave-vit: unifying wavelet and transformers for visual representation learning. In: Proceedings of the European Conference on Computer Vision (ECCV) (2022)

    Google Scholar 

  30. Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yingwei Pan .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chen, J., Zhang, Y., Zhang, Z., Pan, Y., Yao, T. (2022). 3D-Producer: A Hybrid and User-Friendly 3D Reconstruction System. In: Fang, L., Povey, D., Zhai, G., Mei, T., Wang, R. (eds) Artificial Intelligence. CICAI 2022. Lecture Notes in Computer Science(), vol 13606. Springer, Cham. https://doi.org/10.1007/978-3-031-20503-3_43

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20503-3_43

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20502-6

  • Online ISBN: 978-3-031-20503-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics