Abstract
In this paper, we introduce a hybrid and user-friendly 3D reconstruction system for objects, which seamlessly integrates a deep neural network model for 3D shape reconstruction and a multi-band image blending algorithm for realistic texturing into a single encapsulated application, dubbed as 3D-Producer. Compared to expensive laser scanning devices, our 3D-Producer offers an economical and practical solution, where a textured 3D model for a target object can be easily created by general users with mobile devices, a physical turntable, and our application. An illustration video can be found here (https://github.com/winnechan/3D-Producer/blob/main/3D-Producer-Demo-480P.mp4).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Allène, C., Pons, J., Keriven, R.: Seamless image-based texture atlases using multi-band blending. In: ICPR, pp. 1–4. IEEE Computer Society (2008)
Baumberg, A.: Blending images for texturing 3d models. In: Rosin, P.L., Marshall, A.D. (eds.) BMVC, pp. 1–10 (2002)
Cernea, D.: OpenMVS: Multi-view stereo reconstruction library (2020). http://cdcseacave.github.io/openMVS
Chen, X., Ma, H., Wan, J., Li, B., Xia, T.: Multi-view 3d object detection network for autonomous driving. In: CVPR, pp. 1907–1915 (2017)
Contributors, P.: Paddleseg, end-to-end image segmentation kit based on paddlepaddle. http://github.com/PaddlePaddle/PaddleSeg (2019)
Deng, J., Pan, Y., Yao, T., Zhou, W., Li, H., Mei, T.: Single shot video object detector. IEEE Trans. Multimedia 23, 846–858 (2020)
Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F.J., Marín-Jiménez, M.J.: Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognit. 47(6), 2280–2292 (2014)
Gkioxari, G., Johnson, J., Malik, J.: Mesh R-CNN. In: ICCV. IEEE (2019)
Griwodz, C., Gasparini, S., Calvet, L., Gurdjos, P., Castan, F., Maujean, B., Lillo, G.D., Lanthony, Y.: Alicevision Meshroom: an open-source 3D reconstruction pipeline. In: Proceedings of the 12th ACM Multimedia Systems Conference - MMSys 2021. ACM Press (2021)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)
Li, Y., Yao, T., Pan, Y., Mei, T.: Contextual transformer networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. (2022)
Liu, Y., Chu, L., Chen, G., Wu, Z., Chen, Z., Lai, B., Hao, Y.: Paddleseg: a high-efficient development toolkit for image segmentation (2021)
Lombardi, S., Simon, T., Saragih, J.M., Schwartz, G., Lehrmann, A.M., Sheikh, Y.: Neural volumes: learning dynamic renderable volumes from images. ACM Trans. Graph. 38(4), 65:1–65:14 (2019)
Long, F., Qiu, Z., Pan, Y., Yao, T., Luo, J., Mei, T.: Stand-alone inter-frame attention in video models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3192–3201 (2022)
Lorensen, W.E., Cline, H.E.: Marching cubes: a high resolution 3d surface construction algorithm. In: Stone, M.C. (ed.) SIGGRAPH. ACM (1987)
Mescheder, L.M., Oechsle, M., Niemeyer, M., Nowozin, S., Geiger, A.: Occupancy networks: learning 3d reconstruction in function space. In: CVPR, pp. 4460–4470 (2019)
Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 405–421. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_24
Pan, Y., Chen, Y., Bao, Q., Zhang, N., Yao, T., Liu, J., Mei, T.: Smart director: an event-driven directing system for live broadcasting. TOMM 17(4), 1–18 (2021)
Pan, Y., Li, Y., Yao, T., Mei, T., Li, H., Rui, Y.: Learning deep intrinsic video representation by exploring temporal coherence and graph structure. In: IJCAI, pp. 3832–3838 (2016)
Pan, Y., Yao, T., Li, Y., Mei, T.: X-linear attention networks for image captioning. In: CVPR (2020)
Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for 3d classification and segmentation. In: CVPR, pp. 652–660 (2017)
Rematas, K., Liu, A., Srinivasan, P.P., Barron, J.T., Tagliasacchi, A., Funkhouser, T.A., Ferrari, V.: Urban radiance fields. CoRR abs/2111.14643 (2021)
Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2016)
Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: CVPR (2016)
Schönberger, J.L., Zheng, E., Pollefeys, M., Frahm, J.M.: Pixelwise view selection for unstructured multi-view stereo. In: ECCV (2016)
Sitzmann, V., Thies, J., Heide, F., Nießner, M., Wetzstein, G., Zollhöfer, M.: Deepvoxels: Learning persistent 3d feature embeddings. In: CVPR, pp. 2437–2446 (2019)
Wang, P., Liu, L., Liu, Y., Theobalt, C., Komura, T., Wang, W.: Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. In: NeurIPS, pp. 27171–27183 (2021)
Yao, T., Li, Y., Pan, Y., Wang, Y., Zhang, X.P., Mei, T.: Dual vision transformer. arXiv preprint arXiv:2207.04976 (2022)
Yao, T., Pan, Y., Li, Y., Ngo, C.W., Mei, T.: Wave-vit: unifying wavelet and transformers for visual representation learning. In: Proceedings of the European Conference on Computer Vision (ECCV) (2022)
Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, J., Zhang, Y., Zhang, Z., Pan, Y., Yao, T. (2022). 3D-Producer: A Hybrid and User-Friendly 3D Reconstruction System. In: Fang, L., Povey, D., Zhai, G., Mei, T., Wang, R. (eds) Artificial Intelligence. CICAI 2022. Lecture Notes in Computer Science(), vol 13606. Springer, Cham. https://doi.org/10.1007/978-3-031-20503-3_43
Download citation
DOI: https://doi.org/10.1007/978-3-031-20503-3_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20502-6
Online ISBN: 978-3-031-20503-3
eBook Packages: Computer ScienceComputer Science (R0)