Abstract
Art is an essential part of the entertainment. As 3D entertainment such as 3D games is a trend, it is an exciting topic how to create 3D artworks from 2D artworks. In this work, we investigate the 3D reconstruction problem of the artwork called “Sound of Ikebana,” which is created by shooting fluid phenomena using a high-speed camera and can create organic, sophisticated, and complex forms. Firstly, we used the Phase Only Correlation method to capture the artwork’s point cloud based on the images captured by multiple high-speed cameras. Then we create a 3D model by a deep learning-based approach from the 2D Sound of Ikebana images. Our result shows that we can apply deep learning techniques to improve the reconstruction of 3D modeling from 2D images with highly complicated forms.
M. C. Hung, M. X. Trang---Equally contributed as co-first authors
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Tosa, N., Nakatsu, R., Pang, Y.: Creation of media art utilizing fluid dynamics. In: 2017 International Conference on Culture and Computing, pp. 129–135 (2017)
Creswell, A., et al.: Generative adversarial networks: an overview. IEEE Signal Process. Mag. 35(1), 53–65 (2018)
Wu, J., et al.: Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. Advances in Neural Information Processing Systems, pp. 82–90 (2016)
Zhang, Y., et al.: Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering. ICLR 2021 (2021)
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. CVPR2019 (2019)
Tosa, N., et al.: 3D modeling and 3D materialization of fluid art that occurs in very short time. In: 19th IFIP, TC 14 International Conference, pp. 409–421 (2020)
Sakai, S. et al.: An efficient image matching method for multi-view stereo. ACCV 2012, LNCS, vol. 7727, pp. 283–296. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-37447-0_22
Shuji, S., et al.: Phase-based window matching with geometric correction for multi-view stereo. IEJCE Trans. Inf. Syst. 98(10), 1818–1828 (2015)
Edelsbrunner, H., Kirkpatrick, D.G., Seidel, R.: On the shape of a set of points in the plane. IEEE Trans. Inf. Theory 29(4), 551–559 (1983)
Chen, W. et al.: Learning to predict 3d objects with an interpolation-based differentiable renderer. In: 2017 Neural Information Processing Systems NiPS (2019)
Fuji Tsang, C., et al.: Kaolin: A Pytorch Library for Accelerating 3D Deep Learning Research (2022). https://github.com/NVIDIAGameWorks/kaolin
Zhu, J., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251 (2017)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 IFIP International Federation for Information Processing
About this paper
Cite this paper
Hung, M.C., Trang, M.X., Yamada, A., Tosa, N., Nakatsu, R. (2022). Improvement of Deep Learning Technology to Create 3D Model of Fluid Art. In: Göbl, B., van der Spek, E., Baalsrud Hauge, J., McCall, R. (eds) Entertainment Computing – ICEC 2022. ICEC 2022. Lecture Notes in Computer Science, vol 13477. Springer, Cham. https://doi.org/10.1007/978-3-031-20212-4_18
Download citation
DOI: https://doi.org/10.1007/978-3-031-20212-4_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20211-7
Online ISBN: 978-3-031-20212-4
eBook Packages: Computer ScienceComputer Science (R0)