Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article
Open access

Deep Sketch Vectorization via Implicit Surface Extraction

Published: 19 July 2024 Publication History

Abstract

We introduce an algorithm for sketch vectorization with state-of-the-art accuracy and capable of handling complex sketches. We approach sketch vectorization as a surface extraction task from an unsigned distance field, which is implemented using a two-stage neural network and a dual contouring domain post processing algorithm. The first stage consists of extracting unsigned distance fields from an input raster image. The second stage consists of an improved neural dual contouring network more robust to noisy input and more sensitive to line geometry. To address the issue of under-sampling inherent in grid-based surface extraction approaches, we explicitly predict undersampling and keypoint maps. These are used in our post-processing algorithm to resolve sharp features and multi-way junctions. The keypoint and undersampling maps are naturally controllable, which we demonstrate in an interactive topology refinement interface. Our proposed approach produces far more accurate vectorizations on complex input than previous approaches with efficient running time.

Supplementary Material

ZIP File (papers_763.zip)
supplemental

References

[1]
Bin Bao and Hongbo Fu. 2023. Line Drawing Vectorization via Coarse-to-Fine Curve Network Optimization. Computer Graphics Forum n/a, n/a (March 2023).
[2]
Mikhail Bessmeltsev and Justin Solomon. 2018. Vectorization of Line Drawings via PolyVector Fields. arXiv:1801.01922 [cs] (Sept. 2018). arXiv:1801.01922 [cs]
[3]
Mikhail Bessmeltsev and Justin Solomon. 2019. Vectorization of Line Drawings via Polyvector Fields. ACM Transactions on Graphics 38, 1 (Jan. 2019), 1--12.
[4]
Alexandre Carlier, Martin Danelljan, Alexandre Alahi, and Radu Timofte. 2020. DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation. In Advances in Neural Information Processing Systems, Vol. 33. Curran Associates, Inc., 16351--16361.
[5]
Zhiqin Chen, Andrea Tagliasacchi, Thomas Funkhouser, and Hao Zhang. 2022. Neural Dual Contouring. ACM Transactions on Graphics 41, 4 (July 2022), 104:1--104:13.
[6]
Ayan Das, Yongxin Yang, Timothy Hospedales, Tao Xiang, and Yi-Zhe Song. 2021. Cloud2Curve: Generation and Vectorization of Parametric Sketches. arXiv:2103.15536 [cs]
[7]
Edoardo Alberto Dominici, Nico Schertler, Jonathan Griffin, Shayan Hoshyari, Leonid Sigal, and Alla Sheffer. 2020. PolyFit: Perception-Aligned Vectorization of Raster Clip-Art via Intermediate Polygonal Fitting. ACM Transactions on Graphics 39, 4 (Aug. 2020), 77:77:1--77:77:16.
[8]
Luca Donati, Simone Cesano, and Andrea Prati. 2018. A Complete Hand-Drawn Sketch Vectorization Framework. arXiv:1802.05902 [cs]
[9]
Zheng-Jun Du, Liang-Fu Kang, Jianchao Tan, Yotam Gingold, and Kun Xu. 2023. Image vectorization and editing via linear gradient layer decomposition. ACM Transactions on Graphics (TOG) 42, 4 (Aug. 2023).
[10]
Vage Egiazarian, Oleg Voynov, Alexey Artemov, Denis Volkhonskiy, Aleksandr Safin, Maria Taktasheva, Denis Zorin, and Evgeny Burnaev. 2020. Deep Vectorization of Technical Drawings. In Computer Vision - ECCV 2020, Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Vol. 12358. Springer International Publishing, Cham, 582--598.
[11]
Jean-Dominique Favreau, Florent Lafarge, and Adrien Bousseau. 2016. Fidelity vs. Simplicity: A Global Approach to Line Drawing Vectorization. ACM Transactions on Graphics 35, 4 (July 2016), 120:1--120:10.
[12]
Jean-Dominique Favreau, Florent Lafarge, and Adrien Bousseau. 2017. Photo2clipart: image abstraction and vectorization using layered linear gradients. ACM Transactions on Graphics 36, 6 (Dec. 2017), 1--11.
[13]
Songwei Ge, Vedanuj Goswami, C. Lawrence Zitnick, and Devi Parikh. 2021. Creative Sketch Generation. arXiv:2011.10039 [cs] (March 2021). arXiv:2011.10039 [cs]
[14]
Yulia Gryaditskaya, Felix Hähnlein, Chenxi Liu, Alla Sheffer, and Adrien Bousseau. 2020. Lifting Freehand Concept Sketches into 3D. ACM Transactions on Graphics 39, 6 (Nov. 2020), 167:1--167:16.
[15]
Yi Guo, Zhuming Zhang, Chu Han, Wenbo Hu, Chengze Li, and Tien-Tsin Wong. 2019. Deep Line Drawing Vectorization via Line Subdivision and Topology Reconstruction. Computer Graphics Forum 38, 7 (Oct. 2019), 81--90.
[16]
Olga Gutan, Shreya Hegde, Erick Jimenez Berumen, Mikhail Bessmeltsev, and Edward Chien. 2023. Singularity-Free Frame Fields for Line Drawing Vectorization. Computer Graphics Forum (CGF) (2023).
[17]
David Ha and Douglas Eck. 2017. A Neural Representation of Sketch Drawings. arXiv:1704.03477 [cs, stat]
[18]
X. Hilaire and K. Tombre. 2006. Robust and Accurate Vectorization of Line Drawings. IEEE Transactions on Pattern Analysis and Machine Intelligence 28, 6 (June 2006), 890--904.
[19]
Tao Ju, Frank Losasso, Scott Schaefer, and Joe Warren. 2002. Dual Contouring of Hermite Data. ACM Transactions on Graphics 21, 3 (July 2002), 339--346.
[20]
Matthew Kaplan and Elaine Cohen. 2006. Producing Models From Drawings of Curved Surfaces. In SBM. 51--58.
[21]
Yu-Kun Lai, Shi-Min Hu, and Ralph R. Martin. 2009. Automatic and Topology-Preserving Gradient Mesh Generation for Image Vectorization. ACM Transactions on Graphics 28, 3 (July 2009), 85:1--85:8.
[22]
Gregory Lecot and Bruno Levy. 2006. Ardeco: Automatic Region DEtection and COnversion. The Eurographics Association.
[23]
Chenxi Liu, Enrique Rosales, and Alla Sheffer. 2018. StrokeAggregator: Consolidating Raw Sketches into Artist-Intended Curve Drawings. ACM Transactions on Graphics 37, 4 (Aug. 2018), 1--15.
[24]
Hanyuan Liu, Chengze Li, Xueting Liu, and Tien-Tsin Wong. 2022. End-to-End Line Drawing Vectorization. Proceedings of the AAAI Conference on Artificial Intelligence 36, 4 (June 2022), 4559--4566. Number: 4.
[25]
Raphael Gontijo Lopes, David Ha, Douglas Eck, and Jonathon Shlens. 2019. A Learned Representation for Scalable Vector Graphics. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, Seoul, Korea (South), 7929--7938.
[26]
Haoran Mo, Edgar Simo-Serra, Chengying Gao, Changqing Zou, and Ruomei Wang. 2021. General Virtual Sketching Framework for Vector Line Art. ACM Transactions on Graphics 40, 4 (July 2021), 51:1--51:14.
[27]
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. 2022. Instant Neural Graphics Primitives with a Multiresolution Hash Encoding. ACM Trans. Graph. 41, 4, Article 102 (July 2022), 15 pages.
[28]
Gioacchino Noris, Alexander Hornung, Robert W. Sumner, Maryann Simmons, and Markus Gross. 2013. Topology-Driven Vectorization of Clean Line Drawings. ACM Transactions on Graphics 32, 1 (Jan. 2013), 1--11.
[29]
Alexandrina Orzan, Adrien Bousseau, Holger Winnemöller, Pascal Barla, Joëlle Thollot, and David Salesin. 2008. Diffusion curves: a vector representation for smooth-shaded images. ACM Transactions on Graphics 27, 3 (Aug. 2008), 1--8.
[30]
Amal Dev Parakkat, Uday Bondi Pundarikaksha, and Ramanathan Muthuganapathy. 2018. A Delaunay Triangulation Based Approach for Cleaning Rough Sketches. Computers & Graphics 74 (Aug. 2018), 171--181.
[31]
Ivan Puhachov, William Neveu, Edward Chien, and Mikhail Bessmeltsev. 2021. Keypoint-Driven Line Drawing Vectorization via PolyVector Flow. ACM Transactions on Graphics 40, 6 (Dec. 2021), 1--17.
[32]
Pradyumna Reddy, Michael Gharbi, Michal Lukac, and Niloy J. Mitra. 2021. Im2Vec: Synthesizing Vector Graphics without Vector Supervision. arXiv:2102.02798 [cs]
[33]
Philip J. Schneider. 1990. Graphics Gems. Chapter An Algorithm for Automatically Fitting Digitized Curves, 612--626.
[34]
Cloud Shao, Adrien Bousseau, Alla Sheffer, and Karan Singh. 2012. CrossShade: Shading concept sketches using cross-section curves. ACM Transactions on Graphics (TOG) 31, 4 (2012), 1--11. Publisher: ACM New York, NY, USA.
[35]
Edgar Simo-Serra, Satoshi Iizuka, and Hiroshi Ishikawa. 2018. Real-Time Data-Driven Interactive Rough Sketch Inking. ACM Transactions on Graphics 37, 4 (Aug. 2018), 1--14.
[36]
Tibor Stanko, Mikhail Bessmeltsev, David Bommes, and Adrien Bousseau. 2020. Integer-Grid Sketch Simplification and Vectorization. Computer Graphics Forum 39, 5 (Aug. 2020), 149--161.
[37]
Brian Whited, Gioacchino Noris, Maryann Simmons, Robert W Sumner, Markus Gross, and Jarek Rossignac. 2010. BetweenIt: An interactive tool for tight inbetweening. In Computer Graphics Forum, Vol. 29. Wiley Online Library, 605--614. Issue: 2.
[38]
Xiaoyu Xiang, Ding Liu, Xiao Yang, Yiheng Zhu, Xiaohui Shen, and Jan P. Allebach. 2022. Adversarial Open Domain Adaptation for Sketch-to-Photo Synthesis. In 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, Waikoloa, HI, USA, 944--954.
[39]
Chuan Yan, David Vanderhaeghe, and Yotam Gingold. 2020. A Benchmark for Rough Sketch Cleanup. ACM Transactions on Graphics 39, 6 (Nov. 2020), 163:1--163:14.
[40]
Wenwu Yang, Hock-Soon Seah, Quan Chen, Hong-Ze Liew, and Daniel Sýkora. 2018. FTP-SC: Fuzzy Topology Preserving Stroke Correspondence. Computer Graphics Forum 37, 8 (2018), 125--135.
[41]
Jerry Yin, Chenxi Liu, Rebecca Lin, Nicholas Vining, Helge Rhodin, and Alla Sheffer. 2022. Detecting Viewer-Perceived Intended Vector Sketch Connectivity. ACM Transactions on Graphics 41, 4 (July 2022), 87:1--87:11.
[42]
T. Y. Zhang and C. Y. Suen. 1984. A Fast Parallel Algorithm for Thinning Digital Patterns. Commun. ACM 27, 3 (March 1984), 236--239.
[43]
Zibo Zhang, Xueting Liu, Chengze Li, Huisi Wu, and Zhenkun Wen. 2022. Vectorizing Line Drawings of Arbitrary Thickness via Boundary-based Topology Reconstruction. Computer Graphics Forum 41, 2 (2022), 433--445.
[44]
Shuang Zhao, Fredo Durand, and Changxi Zheng. 2018. Inverse Diffusion Curves Using Shape Optimization. IEEE Transactions on Visualization and Computer Graphics 24, 7 (July 2018), 2153--2166.
[45]
Haikuan Zhu, Juan Cao, Yanyang Xiao, Zhonggui Chen, Zichun Zhong, and Yongjie Jessica Zhang. 2022. TCB-spline-based Image Vectorization. ACM Transactions on Graphics 41, 3 (June 2022), 34:1--34:17.

Index Terms

  1. Deep Sketch Vectorization via Implicit Surface Extraction

      Recommendations

      Comments

      Please enable JavaScript to view thecomments powered by Disqus.

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Graphics
      ACM Transactions on Graphics  Volume 43, Issue 4
      July 2024
      1774 pages
      EISSN:1557-7368
      DOI:10.1145/3675116
      Issue’s Table of Contents
      This work is licensed under a Creative Commons Attribution International 4.0 License.

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 19 July 2024
      Published in TOG Volume 43, Issue 4

      Check for updates

      Author Tags

      1. vectorization
      2. raster
      3. sketch
      4. drawing

      Qualifiers

      • Research-article

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • 0
        Total Citations
      • 579
        Total Downloads
      • Downloads (Last 12 months)579
      • Downloads (Last 6 weeks)159
      Reflects downloads up to 29 Nov 2024

      Other Metrics

      Citations

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Login options

      Full Access

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media