research-article

TreeSketchNet: From Sketch to 3D Tree Parameters Generation

Authors:

Gilda Manfredi,

Monica GruossoAuthors Info & Claims

ACM Transactions on Intelligent Systems and Technology, Volume 14, Issue 3

Article No.: 41, Pages 1 - 29

https://doi.org/10.1145/3579831

Published: 24 March 2023 Publication History

Abstract

Three-dimensional (3D) modeling of non-linear objects from stylized sketches is a challenge even for computer graphics experts. The extrapolation of object parameters from a stylized sketch is a very complex and cumbersome task. In the present study, we propose a broker system that can transform a stylized sketch of a tree into a complete 3D model by mediating between a modeler and a 3D modeling software. The input sketches do not need to be accurate or detailed: They must only contain a rudimentary outline of the tree that the modeler wishes to 3D model. Our approach is based on a well-defined Deep Neural Network architecture, called TreeSketchNet (TSN), based on convolutions and capable of generating Weber and Penn [1995] parameters from a simple sketch of a tree. These parameters are then interpreted by the modeling software, which generates the 3D model of the tree pictured in the sketch. The training dataset consists of synthetically generated sketches that are associated with Weber–Penn parameters, generated by a dedicated Blender modeling software add-on. The accuracy of the proposed method is demonstrated by testing the TSN with synthetic and hand-made sketches. Finally, we provide a qualitative analysis of our results, by evaluating the coherence of the predicted parameters with several distinguishing features.

References

[1]

Christopher M. Bishop. 1995. Neural Networks for Pattern Recognition. Oxford University Press, New York, NY.

[2]

John Canny. 1986. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8, 6 (1986), 679–698. DOI:

Digital Library

[3]

Nicola Capece, Francesco Banterle, Paolo Cignoni, Fabio Ganovelli, Roberto Scopigno, and Ugo Erra. 2019. Deepflash: Turning a flash selfie into a studio portrait. Sign. Process.: Image Commun. 77 (2019), 28–39.

Digital Library

[4]

Liang Cheng, Lihua Tong, Yanming Chen, Wen Zhang, Jie Shan, Yongxue Liu, and Manchun Li. 2013. Integration of LiDAR data and optical multi-view images for 3D reconstruction of building roofs. Optics Lasers Eng. 51, 4 (2013), 493–502.

[5]

Paolo Cignoni, Claudio Rocchini, and Roberto Scopigno. 1998. Metro: Measuring error on simplified surfaces. Comput. Graph. Forum 17, 2 (1998), 167–174. DOI:

[6]

Zihang Dai, Hanxiao Liu, Quoc V. Le, and Mingxing Tan. 2021. CoAtNet: Marrying convolution and attention for all data sizes. In Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P. S. Liang, and J. Wortman Vaughan (Eds.), Vol. 34. Curran Associates, Inc., 3965–3977.

[7]

Stephanie Delalieux, Pablo J. Zarco-Tejada, Laurent Tits, Miguel Ángel Jiménez Bello, Diego S. Intrigliolo, and Ben Somers. 2014. Unmixing-based fusion of hyperspatial and hyperspectral airborne imagery for early detection of vegetation stress. IEEE J. Select. Top. Appl. Earth Observ. Remote Sens. 7, 6 (2014), 2571–2582. DOI:

[8]

Johanna Delanoy, Mathieu Aubry, Phillip Isola, Alexei A. Efros, and Adrien Bousseau. 2018. 3d sketching using multi-view deep volumetric prediction. Proc. ACM Comput. Graph. Interact. Techn. 1, 1 (2018), 1–22.

Digital Library

[9]

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei Li. 2009. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Los Alamitos, CA, 248–255. DOI:

[10]

Oliver Deussen and Bernd Lintermann. 2005. Digital Design of Nature: Computer Generated Plants and Organics. Springer Science & Business Media, Berlin.

[11]

Chao Ding and Ligang Liu. 2016. A survey of sketch based modeling systems. Front. Comput. Sci. 10, 6 (2016), 985–999.

Digital Library

[12]

David S. Ebert, F. Kenton Musgrave, Darwyn Peachey, Ken Perlin, and Steven Worley. 2002. Texturing and Modeling: A Procedural Approach (3rd ed.). Morgan Kaufmann, San Francisco, CA.

[13]

Mathias Eitz, James Hays, and Marc Alexa. 2012. How do humans sketch objects? ACM Trans. Graph. 31, 4, Article 44 (July 2012), 1--10. DOI:

Digital Library

[14]

Haoqiang Fan, Hao Su, and Leonidas J. Guibas. 2017. A point set generation network for 3D object reconstruction from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Los Alamitos, CA, 605–613.

[15]

Teodor Fredriksson, David Issa Mattos, Jan Bosch, and Helena Holmström Olsson. 2020. Data labeling: An empirical investigation into industrial challenges and mitigation strategies. In International Conference on Product-Focused Software Process Improvement. Springer, 202–216.

Digital Library

[16]

Sarah Frisken, Nazim Haouchine, Rose Du, and Alexandra J. Golby. 2022. Using temporal and structural data to reconstruct 3D cerebral vasculature from a pair of 2D digital subtraction angiography sequences. Comput. Med. Imag. Graph. 99 (2022), 102076. DOI:

[17]

Demetrios Gatziolis, Jean F. Lienard, Andre Vogs, and Nikolay S. Strigul. 2015. 3D tree dimensionality assessment using photogrammetry and small unmanned aerial vehicles. PLoS One 10, 9 (09 2015), 1–21. DOI:

[18]

Georgia Gkioxari, Jitendra Malik, and Justin Johnson. 2019. Mesh R-CNN. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV’19). IEEE, 9784–9794. DOI:

[19]

Tim Golla, Tom Kneiphof, Heiner Kuhlmann, Michael Weinmann, and Reinhard Klein. 2020. Temporal upsampling of point cloud sequences by optimal transport for plant growth visualization. Comput. Graph. Forum 39, 6 (2020), 167–179.

[20]

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press, Cambridge, MA.

Digital Library

[21]

Benoit Guillard, Edoardo Remelli, Pierre Yvernay, and Pascal Fua. 2021. Sketch2Mesh: Reconstructing and Editing 3D Shapes from Sketches.

[22]

Jianwei Guo, Haiyong Jiang, Bedrich Benes, Oliver Deussen, Xiaopeng Zhang, Dani Lischinski, and Hui Huang. 2020a. Inverse procedural modeling of branching structures by inferring L-Systems. ACM Trans. Graph. 39, 5, Article 155 (June 2020), 1--13. DOI:

Digital Library

[23]

Jianwei Guo, Shibiao Xu, Dong-Ming Yan, Zhanglin Cheng, Marc Jaeger, and Xiaopeng Zhang. 2020b. Realistic procedural plant modeling from multiple view images. IEEE Trans. Vis. Comput. Graph. 26, 2 (2020), 1372–1384. DOI:

[24]

Torsten Hädrich, Daniel T. Banuti, Wojtek Pałubicki, Sören Pirk, and Dominik L. Michels. 2021. Fire in paradise: Mesoscale simulation of wildfires. ACM Trans. Graph. 40, 4, Article 163 (July 2021), 1--15. DOI:

Digital Library

[25]

Christian Häne, Shubham Tulsiani, and Jitendra Malik. 2017. Hierarchical surface prediction for 3d object reconstruction. In Proceedings of the International Conference on 3D Vision (3DV’17). IEEE, 412–420.

[26]

Mohammadul Haque, Avishek Chatterjee, Venu Madhav Govindu, et al. 2014. High quality photometric reconstruction using a depth camera. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). IEEE.

Digital Library

[27]

Karl Haubenwallner, Hans-Peter Seidel, and Markus Steinberger. 2017. ShapeGenetics: Using genetic algorithms for procedural modeling. Comput. Graph. Forum 36, 2 (May 2017), 213–223. DOI:

Digital Library

[28]

Kaiming He and Jian Sun. 2015. Convolutional neural networks at constrained time cost. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 5353–5360.

[29]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). IEEE, 770–778. DOI:

[30]

Shaojun Hu, Zhengrong Li, Zhiyi Zhang, Dongjian He, and Michael Wimmer. 2017. Efficient tree modeling from airborne LiDAR point clouds. Comput. Graph. 67 (2017), 1–13. DOI:

Digital Library

[31]

Haibin Huang, Evangelos Kalogerakis, Ersin Yumer, and Radomir Mech. 2017. Shape synthesis from sketches via procedural models and convolutional networks. IEEE Trans. Vis. Comput. Graph. 23, 8 (2017), 2003–2013. DOI:

Digital Library

[32]

John F. Hughes, Andries van Dam, Morgan McGuire, David F. Sklar, James D. Foley, Steven K. Feiner, and Kurt Akeley. 2013. Computer Graphics: Principles and Practice (3rd ed.). Addison-Wesley Professional, Boston, MA, 1264 pages.

[33]

Kaitlyn Kingsland. 2020. Comparative analysis of digital photogrammetry software for cultural heritage. Digit. Appl. Archaeol. Cult. Herit. 18 (2020), e00157. DOI:

[34]

Shahram Izadi, David Kim, Otmar Hilliges, David Molyneaux, Richard Newcombe, Pushmeet Kohli, Jamie Shotton, Steve Hodges, Dustin Freeman, Andrew Davison, et al. 2011. KinectFusion: Real-time 3D reconstruction and interaction using a moving depth camera. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST’11). Association for Computing Machinery, New York, NY, 559–568. DOI:

Digital Library

[35]

Angjoo Kanazawa, Shubham Tulsiani, Alexei A Efros, and Jitendra Malik. 2018. Learning category-specific mesh reconstruction from image collections. In Proceedings of the European Conference on Computer Vision (ECCV’18). Springer.

Digital Library

[36]

Nick Kanopoulos, Nagesh Vasanthavada, and Robert L. Baker. 1988. Design of an image edge detection filter using the Sobel operator. IEEE J. Solid-State Circ. 23, 2 (1988), 358–367. DOI:

[37]

Kamal Kc, Zhendong Yin, Dasen Li, and Zhilu Wu. 2021. Impacts of background removal on convolutional neural networks for plant disease classification in-situ. Agriculture 11, 9 (2021), 827. DOI:

[38]

Asifullah Khan, Anabia Sohail, Umme Zahoora, and Aqsa Saeed Qureshi. 2020. A survey of the recent architectures of deep convolutional neural networks. Artif. Intell. Rev. 53, 8 (2020), 5455–5516.

Digital Library

[39]

Jaehwan Kim and Il-Kwon Jeong. 2014. Single image–based 3D tree and growth models reconstruction. Etri J. 36, 3 (2014), 450–459.

[40]

Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. https://doi.org/10.48550/ARXIV.1412.6980

[41]

John F. Kolen and Stefan C. Kremer. 2001. Gradient Flow in Recurrent Nets: The Difficulty of Learning LongTerm Dependencies. Wiley-IEEE Press, 237–243. DOI:

[42]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Proceeding of the Advances in Neural Information Processing Systems, F. Pereira, C. J. Burges, L. Bottou, and K. Q. Weinberger (Eds.). Vol. 25. Curran Associates, Inc., 1097--1105. https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf.

[43]

Jiahui Lei, Srinath Sridhar, Paul Guerrero, Minhyuk Sung, Niloy Mitra, and Leonidas J. Guibas. 2020. Pix2Surf: Learning parametric 3D surface models of objects from images. In Proceedings of European Conference on Computer Vision (ECCV’20). Springer International Publishing.

Digital Library

[44]

Bosheng Li, Jacek Kałużny, Jonathan Klein, Dominik L. Michels, Wojtek Pałubicki, Bedrich Benes, and Sören Pirk. 2021. Learning to reconstruct botanical trees from single images. ACM Trans. Graph. 40, 6 (2021), 1–15.

Digital Library

[45]

Changjian Li, Hao Pan, Yang Liu, Xin Tong, Alla Sheffer, and Wenping Wang. 2017. BendSketch: Modeling freeform surfaces through 2D sketching. ACM Trans. Graph. 36, 4, Article 125 (July 2017), 1--14. DOI:

Digital Library

[46]

Aristid Lindenmayer. 1968. Mathematical models for cellular interactions in development I. Filaments with one-sided inputs. J. Theor. Biol. 18, 3 (1968), 280–299. DOI:

[47]

Jialin Liu, Sam Snodgrass, Ahmed Khalifa, Sebastian Risi, Georgios N. Yannakakis, and Julian Togelius. 2021b. Deep learning for procedural content generation. Neural Comput. Appl. 33, 1 (2021), 19–37.

Digital Library

[48]

Jia Liu, Xiaopeng Zhang, and Hongjun Li. 2010. Sketch-based tree modeling by distribution control on planes. In Proceedings of the 9th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry (VRCAI’10). Association for Computing Machinery, New York, NY, 185–190. DOI:

Digital Library

[49]

Yanchao Liu, Jianwei Guo, Bedrich Benes, Oliver Deussen, Xiaopeng Zhang, and Hui Huang. 2021a. TreePartNet: Neural decomposition of point clouds for 3D tree reconstruction. ACM Trans. Graph. 40, 6, Article 232 (December 2021), 1--16. DOI:

Digital Library

[50]

Zhihao Liu, Kai Wu, Jianwei Guo, Yunhai Wang, Oliver Deussen, and Zhanglin Cheng. 2021c. Single image tree reconstruction via adversarial network. Graph. Models 117 (2021), 101–115. DOI:

Digital Library

[51]

Steven Longay, Adam Runions, Frédéric Boudon, and Przemyslaw Prusinkiewicz. 2012. TreeSketch: Interactive procedural modeling of trees on a tablet. In Eurographics Workshop on Sketch-Based Interfaces and Modeling. The Eurographics Association, Eindhoven, The Netherlands. DOI:

[52]

Lu-Xingchang and Liu-Xianlin. 2006. Reconstruction of 3D Model Based on Laser Scanning. Springer, Berlin, 317–332. DOI:

[53]

Miłosz Makowski, Torsten Hädrich, Jan Scheffczyk, Dominik L. Michels, Sören Pirk, and Wojtek Pałubicki. 2019. Synthetic silviculture: Multi-scale modeling of plant ecosystems. ACM Trans. Graph. 38, 4, Article 131 (July 2019), 1--14. DOI:

Digital Library

[54]

Gilda Manfredi, Nicola Capece, Ugo Erra, and Monica Gruosso. 2021. TreeSketchNet Parameters Details. Retrieved from https://github.com/Unibas3D/TreeSketchNet/blob/main/Appendix.md.

[55]

Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. 2018. Foundations of Machine Learning. MIT Press, Cambridge, MA.

Digital Library

[56]

Andrew Y. Ng. 2004. Feature selection, L1 vs. L2 regularization, and rotational invariance. In Proceedings of the 21st International Conference on Machine Learning (ICML’04). Association for Computing Machinery, New York, NY, 78. DOI:

Digital Library

[57]

Trong-Nguyen Nguyen, Huu-Hung Huynh, and Jean Meunier. 2018. 3D reconstruction with time-of-flight depth camera and multiple mirrors. IEEE Access 6 (2018), 38106–38114. DOI:

[58]

Gen Nishida, Ignacio Garcia-Dorado, Daniel G. Aliaga, Bedrich Benes, and Adrien Bousseau. 2016. Interactive sketching of urban procedural models. ACM Trans. Graph. 35, 4 (2016), 1–11. DOI:

Digital Library

[59]

Chengjie Niu, Jun Li, and Kai Xu. 2018. Im2Struct: Recovering 3D shape structure from a single RGB image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’18). IEEE Computer Society, Los Alamitos, CA, 4521–4529. DOI:

[60]

Makoto Okabe, Shigeru Owada, and Takeo Igarashi. 2007. Interactive design of botanical trees using freehand sketches and example-based editing. In ACM SIGGRAPH 2007 Courses (SIGGRAPH’07). Association for Computing Machinery, New York, NY, 26–es. DOI:

Digital Library

[61]

Wojciech Palubicki, Kipp Horel, Steven Longay, Adam Runions, Brendan Lane, Radomír Měch, and Przemyslaw Prusinkiewicz. 2009. Self-organizing tree models for image synthesis. ACM Trans. Graph. 28, 3, Article 58 (July 2009), 1--10. DOI:

Digital Library

[62]

Wojtek Pałubicki, Miłosz Makowski, Weronika Gajda, Torsten Hädrich, Dominik L. Michels, and Sören Pirk. 2022. Ecoclimates: Climate-response modeling of vegetation. ACM Trans. Graph. 41, 4, Article 155 (July 2022), 1--19. DOI:

Digital Library

[63]

Junyi Pan, Xiaoguang Han, Weikai Chen, Jiapeng Tang, and Kui Jia. 2019. Deep mesh reconstruction from single RGB images via topology modification networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV’19). IEEE Computer Society, Los Alamitos, CA, 9963–9972. DOI:

[64]

Kyungjin Park, Bradford W. Mott, Wookhee Min, Kristy Elizabeth Boyer, Eric N. Wiebe, and James C. Lester. 2019. Generating educational game levels with multistep deep convolutional generative adversarial networks. In Proceedings of the IEEE Conference on Games (CoG’19). IEEE, Los Alamitos, CA, 1–8. DOI:

Digital Library

[65]

Tomas Polasek, David Hrusa, Bedrich Benes, and Martin Čadík. 2021. ICTree: Automatic perceptual metrics for tree models. ACM Trans. Graph. 40, 6, Article 230 (December 2021), 1--15. DOI:

Digital Library

[66]

Thomas Porter and Tom Duff. 1984. Compositing digital images. SIGGRAPH Comput. Graph. 18, 3 (January 1984), 253–259. DOI:

Digital Library

[67]

Christophe Pradal, Frédéric Boudon, Christophe Nouguier, Jérôme Chopard, and Christophe Godin. 2009. PlantGL: A Python-based geometric library for 3D plant modelling at different scales. Graph. Models 71, 1 (2009), 1–21. DOI:

Digital Library

[68]

Przemyslaw Prusinkiewicz and Aristid Lindenmayer. 2012. The Algorithmic Beauty of Plants. Springer Science & Business Media, New York.

[69]

Ed Quigley, Winnie Lin, Yilin Zhu, and Ronald Fedkiw. 2021. Three dimensional reconstruction of botanical trees with simulatable geometry. Proc. ACM Comput. Graph. Interact. Tech. 4, 3, Article 37 (September 2021), 1--16. DOI:

Digital Library

[70]

Fabio Remondino, Stefano Girardi, Alessandro Rizzi, and Lorenzo Gonzo. 2009. 3D modeling of complex and detailed cultural heritage using multi-resolution data. J. Comput. Cult. Herit. 2, 1, Article 2 (July 2009), 1--20. DOI:

Digital Library

[71]

Katja Richter and Hans-Gerd Maas. 2022. Radiometric enhancement of full-waveform airborne laser scanner data for volumetric representation in environmental applications. ISPRS J. Photogram. Remote Sens. 183 (2022), 510–524. DOI:

[72]

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. 2015. ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 3 (2015), 211–252. DOI:

Digital Library

[73]

Martin Rutzinger, Arun Kumar Pratihast, S. Oude Elberink, and George Vosselman. 2010. Detection and modelling of 3D trees from mobile laser scanning data. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci 38 (2010), 520–525.

[74]

Shaeke Salman and Xiuwen Liu. 2019. Overfitting Mechanism and Avoidance in Deep Neural Networks. https://doi.org/10.48550/ARXIV.1901.06566

[75]

Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. MobileNetV2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Computer Vision and Pattern Recognition Conference (CVPR’18). IEEE, Los Alamitos, CA, 4510–4520. DOI:

[76]

Michael Schwarz and Pascal Müller. 2015. Advanced procedural modeling of architecture. ACM Trans. Graph. 34, 4, Article 107 (July 2015), 1--12. DOI:

Digital Library

[77]

Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. In Proceedings of the International Conference on Learning Representations. IEEE.

[78]

Ruben M. Smelik, Tim Tutenel, Rafael Bidarra, and Bedrich Benes. 2014. A survey on procedural modelling for virtual worlds. Comput. Graph. Forum 33, 6 (2014), 31–50. DOI:

Digital Library

[79]

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1 (2014), 1929–1958.

Digital Library

[80]

Ondrej Stava, Bedrich Beneš, Radomir Měch, Daniel G. Aliaga, and Peter Krištof. 2010. Inverse procedural modeling by automatic generation of L-systems. Comput. Graph. Forum 29, 2 (2010), 665–674. DOI:

[81]

Ondrej Stava, Sören Pirk, Julian Kratt, Baoquan Chen, Radomír Měch, Oliver Deussen, and Bedrich Benes. 2014. Inverse procedural modelling of trees. Comput. Graph. Forum 33, 6 (September 2014), 118–131. DOI:

Digital Library

[82]

Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). IEEE Computer Society, Los Alamitos, CA, 2818–2826. DOI:

[83]

Julián Tachella, Yoann Altmann, Nicolas Mellado, Aongus McCarthy, Rachael Tobin, Gerald S. Buller, Jean-Yves Tourneret, and Stephen McLaughlin. 2019. Real-time 3D reconstruction from single-photon lidar data using plug-and-play point cloud denoisers. Nat. Commun. 10, 1 (2019), 1–6.

[84]

Mingxing Tan and Quoc Le. 2019. EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). 6105–6114.

[85]

Ping Tan, Gang Zeng, Jingdong Wang, Sing Bing Kang, and Long Quan. 2007. Image-based tree modeling. ACM Trans. Graph. 26, 3 (July 2007), 87–es. DOI:

Digital Library

[86]

Liyu Tang, Xianmin Peng, Chongcheng Chen, Hongyu Huang, and Ding Lin. 2019. Three-dimensional Forest growth simulation in virtual geographic environments. Earth Sci. Inf. 12, 1 (03 2019), 31--41. DOI:

[87]

Maxim Tatarchenko, Alexey Dosovitskiy, and Thomas Brox. 2017. Octree generating networks: Efficient convolutional architectures for high-resolution 3d outputs. In Proceedings of the IEEE International Conference on Computer Vision. IEEE, 2088–2096.

[88]

Vajira Thambawita, Inga Strümke, Steven A. Hicks, Pål Halvorsen, Sravanthi Parasa, and Michael A. Riegler. 2021. Impact of image resolution on deep learning performance in endoscopy image classification: An experimental study using a large dataset of endoscopic images. Diagnostics (Basel) 11, 12 (November 2021), 2183.

[89]

Gizem Unlu, Mohamed Sayed, and Gabriel Brostow. 2022. Interactive sketching of mannequin poses. In Proceedings of the IEEE/CVF International Conference on 3D Vision (3DV’22).

[90]

Guan Wang, Hamid Laga, Ning Xie, Jinyuan Jia, and Hedi Tabia. 2018b. The shape space of 3D botanical tree models. ACM Trans. Graph. 37, 1, Article 7 (January 2018), 1--18. DOI:

Digital Library

[91]

Lingjing Wang, Cheng Qian, Jifei Wang, and Yi Fang. 2018c. Unsupervised learning of 3D model reconstruction from hand-drawn sketches. In Proceedings of the 26th ACM International Conference on Multimedia (MM’18). Association for Computing Machinery, New York, NY, 1820–1828. DOI:

Digital Library

[92]

Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. 2018d. Pixel2mesh: Generating 3D mesh models from single rgb images. In Proceedings of the European Conference on Computer Vision (ECCV’18). Springer, 52–67.

Digital Library

[93]

Tuanfeng Y. Wang, Duygu Ceylan, Jovan Popovic, and Niloy J. Mitra. 2018a. Learning a shared shape space for multimodal garment design. ACM Trans. Graph. 37, 6, Article 203 (December 2018), 1--13. DOI:

Digital Library

[94]

Jason Weber and Joseph Penn. 1995. Creation and rendering of realistic trees. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’95). Association for Computing Machinery, New York, NY, 119–128. DOI:

Digital Library

[95]

Karl Weiss, Taghi M. Khoshgoftaar, and DingDing Wang. 2016. A survey of transfer learning. J. Big Data 3, 1 (28 May 2016), 9. DOI:

[96]

Haozhe Xie, Hongxun Yao, Xiaoshuai Sun, Shangchen Zhou, and Shengping Zhang. 2019. Pix2Vox: Context-aware 3D reconstruction from single and multi-view images. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV’19). IEEE Computer Society, Los Alamitos, CA, 2690–2698. DOI:

[97]

Ke Xie, Feilong Yan, Andrei Sharf, Oliver Deussen, Hui Huang, and Baoquan Chen. 2016. Tree modeling with real tree-parts examples. IEEE Trans. Vis. Comput. Graph. 22, 12 (2016), 2608–2618. DOI:

Digital Library

[98]

Qiangeng Xu, Weiyue Wang, Duygu Ceylan, Radomir Mech, and Ulrich Neumann. 2019. DISN: Deep implicit surface network for high-quality single-view 3D reconstruction. In Proceeding of the Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Eds.). Vol. 32. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2019/file/39059724f73a9969845dfe4146c5660e-Paper.pdf.

[99]

Guandao Yang, Yin Cui, Serge Belongie, and Bharath Hariharan. 2018. Learning single-view 3D reconstruction with limited pose supervision. In Proceedings of the European Conference on Computer Vision (ECCV’18). Springer, 86–101.

Digital Library

[100]

Liuming Yang, Meng Yang, and Gang Yang. 2019. Modeling fractures and cracks on tree branches. Comput. Graph. 80 (2019), 63–72. DOI:

Digital Library

[101]

Xiangli Yang, Zixing Song, Irwin King, and Zenglin Xu. 2022. A survey on deep semi-supervised learning. IEEE Transactions on Knowledge and Data Engineering (2022), 1--20. https://doi.org/10.1109/TKDE.2022.3220219

[102]

Mehmet Ersin Yumer, Paul Asente, Radomir Mech, and Levent Burak Kara. 2015. Procedural modeling using autoencoder networks. In Proceedings of the 28th Annual ACM Symposium on User Interface Software &; Technology (UIST’15). Association for Computing Machinery, New York, NY, 109–118. DOI:

Digital Library

[103]

Yang Zhang, Zhen Liu, Tianpeng Liu, Bo Peng, and Xiang Li. 2019. RealPoint3D: An efficient generation network for 3D object reconstruction from a single image. IEEE Access 7 (2019), 57539–57549. DOI:

[104]

Qian-Yi Zhou and Vladlen Koltun. 2014. Color map optimization for 3D reconstruction with consumer depth cameras. ACM Trans. Graph. 33, 4, Article 155 (July 2014), 1--10. DOI:

Digital Library

[105]

Changqing Zou, Qian Yu, Ruofei Du, Haoran Mo, Yi-Zhe Song, Tao Xiang, Chengying Gao, Baoquan Chen, and Hao Zhang. 2018. SketchyScene: Richly-annotated scene sketches. In Proceedings of the European Conference on Computer Vision (ECCV’18), Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss (Eds.). Springer International Publishing, Cham, 438–454.

Digital Library

[106]

Chuhang Zou, Ersin Yumer, Jimei Yang, Duygu Ceylan, and Derek Hoiem. 2017. 3D-PRNN: Generating shape primitives with recurrent neural networks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’17). IEEE Computer Society, Los Alamitos, CA, 900–909. DOI:

Cited By

Stacchio LBalloni EGorgoglione LMancini AGiovanola BTiribelli SZingaretti P(2024)An ethical framework for trustworthy Neural Rendering applied in cultural heritage and creative industriesFrontiers in Computer Science10.3389/fcomp.2024.14598076Online publication date: 2-Oct-2024
https://doi.org/10.3389/fcomp.2024.1459807
Liu ZLi YTu FZhang RCheng ZYokoya N(2024)DeepTreeSketch: Neural Graph Prediction for Faithful 3D Tree Modeling from SketchesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642125(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642125
Ding WChen HWan ZXu LXu XZhang Y(2024)Dynamic simulation of branch deformation under snow pressureInternational Journal of Modeling, Simulation, and Scientific Computing10.1142/S179396232450024715:01Online publication date: 23-Feb-2024
https://doi.org/10.1142/S1793962324500247
Show More Cited By

Index Terms

TreeSketchNet: From Sketch to 3D Tree Parameters Generation
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Image and video acquisition
        3D imaging
  2. Computer graphics
    1. Shape modeling
      1. Mesh models

Recommendations

Procedural Hair Generation
SBGAMES '09: Proceedings of the 2009 VIII Brazilian Symposium on Games and Digital Entertainment

This paper presents a hair generation technique for cartoon and Anime characters. The main goal of this work is to provide a procedural generation technique, robust enough in order to create different types of hairs. We propose a parametric model that ...
Interactive example-based terrain authoring with conditional generative adversarial networks

Authoring virtual terrains presents a challenge and there is a strong need for authoring tools able to create realistic terrains with simple user-inputs and with high user control. We propose an example-based authoring pipeline that uses a set of ...
Procedural Modeling Using Autoencoder Networks
UIST '15: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology

Procedural modeling systems allow users to create high quality content through parametric, conditional or stochastic rule sets. While such approaches create an abstraction layer by freeing the user from direct geometry editing, the nonlinear nature and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Intelligent Systems and Technology

ACM Transactions on Intelligent Systems and Technology Volume 14, Issue 3

June 2023

451 pages

ISSN:2157-6904

EISSN:2157-6912

DOI:10.1145/3587032

Editor:
Huan Liu
Arizona State University, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 March 2023

Online AM: 09 January 2023

Accepted: 14 December 2022

Revised: 14 December 2022

Received: 26 July 2022

Published in TIST Volume 14, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
810
Total Downloads

Downloads (Last 12 months)379
Downloads (Last 6 weeks)35

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Stacchio LBalloni EGorgoglione LMancini AGiovanola BTiribelli SZingaretti P(2024)An ethical framework for trustworthy Neural Rendering applied in cultural heritage and creative industriesFrontiers in Computer Science10.3389/fcomp.2024.14598076Online publication date: 2-Oct-2024
https://doi.org/10.3389/fcomp.2024.1459807
Liu ZLi YTu FZhang RCheng ZYokoya N(2024)DeepTreeSketch: Neural Graph Prediction for Faithful 3D Tree Modeling from SketchesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642125(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642125
Ding WChen HWan ZXu LXu XZhang Y(2024)Dynamic simulation of branch deformation under snow pressureInternational Journal of Modeling, Simulation, and Scientific Computing10.1142/S179396232450024715:01Online publication date: 23-Feb-2024
https://doi.org/10.1142/S1793962324500247
Wu DYang MLiu ZTu FLiu FCheng Z(2024)VRTree: Example‐Based 3D Interactive Tree Modeling in Virtual RealityComputer Graphics Forum10.1111/cgf.1525443:7Online publication date: 7-Nov-2024
https://doi.org/10.1111/cgf.15254
Iuliano ALiò PManfredi GRomaniello F(2024)Denoising Probabilistic Diffusion Models for Synthetic Healthcare Image Generation2024 IEEE International Workshop on Metrology for Living Environment (MetroLivEnv)10.1109/MetroLivEnv60384.2024.10615511(415-420)Online publication date: 12-Jun-2024
https://doi.org/10.1109/MetroLivEnv60384.2024.10615511
Xu JZhang ZHu XKe T(2024)Accelerated forest modeling from tree canopy point clouds via deep learningInternational Journal of Applied Earth Observation and Geoinformation10.1016/j.jag.2024.104074132(104074)Online publication date: Aug-2024
https://doi.org/10.1016/j.jag.2024.104074
Chen CWang HWang DWang D(2024)Towards the digital twin of urban forest: 3D modeling and parameterization of large-scale urban trees from close-range laser scanningInternational Journal of Applied Earth Observation and Geoinformation10.1016/j.jag.2024.103695127(103695)Online publication date: Mar-2024
https://doi.org/10.1016/j.jag.2024.103695
Umezawa WMukai T(2024)Training Climbing Roses by Constrained Graph SearchComputer Animation and Virtual Worlds10.1002/cav.229735:6Online publication date: 15-Oct-2024
https://doi.org/10.1002/cav.2297
Notarangelo NManfredi GGilio G(2023)A Collaborative Virtual Walkthrough of Matera’s Sassi Using Photogrammetric Reconstruction and Hand Gesture NavigationJournal of Imaging10.3390/jimaging90400889:4(88)Online publication date: 21-Apr-2023
https://doi.org/10.3390/jimaging9040088
Capece NManfredi GGilio GErra UToscano FFiorentino CD'Antonio P(2023)An Educational Approach for Mixed Reality Visualization of Agro-meteorological Parameters2023 IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE)10.1109/MetroXRAINE58569.2023.10405837(46-51)Online publication date: 25-Oct-2023
https://doi.org/10.1109/MetroXRAINE58569.2023.10405837
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents