Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article
Open access

Vectorization for Fast, Analytic, and Differentiable Visibility

Published: 15 July 2021 Publication History

Abstract

In Computer Graphics, the two main approaches to rendering and visibility involve ray tracing and rasterization. However, a limitation of both approaches is that they essentially use point sampling. This is the source of noise and aliasing, and also leads to significant difficulties for differentiable rendering. In this work, we present a new rendering method, which we call vectorization, that computes 2D point-to-region integrals analytically, thus eliminating point sampling in the 2D integration domain such as for pixel footprints and area lights. Our vectorization revisits the concept of beam tracing, and handles the hidden surface removal problem robustly and accurately. That is, for each intersecting triangle inserted into the viewport of a beam in an arbitrary order, we are able to maintain all the visible regions formed by intersections and occlusions, thanks to our Visibility Bounding Volume Hierarchy structure. As a result, our vectorization produces perfectly anti-aliased visibility, accurate and analytic shading and shadows, and most important, fast and noise-free gradients with Automatic Differentiation or Finite Differences that directly enables differentiable rendering without any changes to our rendering pipeline. Our results are inherently high-quality and noise-free, and our gradients are one to two orders of magnitude faster than those computed with existing differentiable rendering methods.

References

[1]
John M. Airey, John H. Rohlf, and Frederick P. Brooks Jr. 1990. Towards image realism with interactive update rates in complex virtual building environments. Comput. Graph. 24, 2 (1990), 41–50.
[2]
John Amanatides. 1984. Ray tracing with cones. Comput. Graph. 18, 3 (1984), 129–135.
[3]
Oana Livia Apostu. 2012. Analytic Visibility in Plücker Space: From Theory to Practical Applications. Ph.D. Dissertation. Université de Limoges.
[4]
Thomas Auzinger, Michael Guthe, and Stefan Jeschke. 2012. Analytic anti-aliasing of linear functions on polytopes. Comput. Graph. Forum 31, 2pt1 (2012), 335–344.
[5]
Thomas Auzinger, Przemyslaw Musialski, Reinhold Preiner, and Michael Wimmer. 2013a. Non-sampled anti-aliasing. In Proceedings of the Conference on Vision, Modeling, and Visualization. The Eurographics Association.
[6]
Thomas Auzinger, Michael Wimmer, and Stefan Jescke. 2013b. Analytic visibility on the GPU. Comput. Graph. Forum 32, 2pt4 (2013), 409–418.
[7]
Sai Bangaru, Tzu-Mao Li, and Frédo Durand. 2020. Unbiased warped-area sampling for differentiable rendering. ACM Trans. Graph. 39, 6 (2020), 245:1–245:18.
[8]
D. R. Baum, H. E. Rushmeier, and J. M. Winget. 1989. Improving radiosity solutions through the use of analytically determined form-factors. Comput. Graph. 23, 3 (1989), 325–334.
[9]
Jiří Bittner. 2002a. Efficient construction of visibility maps using approximate occlusion sweep. In Proceedings of the Spring Conference on Computer Graphics. 167–175.
[10]
Jirı Bittner. 2002b. Hierarchical Techniques for Visibility Computations. Ph.D. Dissertation. Czech Technical University.
[11]
Jiri Bittner, Vlastimil Havran, and Pavel Slavik. 1998. Hierarchical visibility culling with occlusion trees. In Proceedings of the Computer Graphics International Conference. IEEE, 207–219.
[12]
Christoph Burnikel, Jochen Könemann, Kurt Mehlhorn, Stefan Näher, Stefan Schirra, and Christian Uhrig. 1995. Exact geometric computation in LEDA. In Proceedings of the Symposium on Computational Geometry. 418–419.
[13]
Richard H. Byrd, Samantha L. Hansen, Jorge Nocedal, and Yoram Singer. 2016. A stochastic quasi-Newton method for large-scale optimization. SIAM J. Optimiz. 26, 2 (2016), 1008–1031.
[14]
Bob Carpenter, Matthew D. Hoffman, Marcus Brubaker, Daniel Lee, Peter Li, and Michael Betancourt. 2015. The Stan math library: Reverse-mode automatic differentiation in C++. Retrieved from https://arXiv:1509.07164.
[15]
Edwin Catmull. 1978. A hidden-surface algorithm with anti-aliasing. Compute. Graph. 12, 3 (1978), 6–11.
[16]
Edwin Catmull. 1984. An analytic visible surface algorithm for independent pixel processing. Comput. Graph. 18, 3 (1984), 109–115.
[17]
Anish Chandak, Lakulish Antani, Micah Taylor, and Dinesh Manocha. 2009. FastV: From-point visibility culling on complex models. Comput. Graph. Forum 28, 4 (2009), 1237–1246.
[18]
Satyan Coorg and Seth Teller. 1997. Real-time occlusion culling for models with large occluders. In Proceedings of the Symposium on Interactive 3D Graphics and Games. 83–90.
[19]
Cyril Crassin, Fabrice Neyret, Miguel Sainz, Simon Green, and Elmar Eisemann. 2011. Interactive indirect illumination using voxel cone tracing. Comput. Graph. Forum 30, 7 (2011), 1921–1930.
[20]
Cyril Crassin, Chris Wyman, Morgan McGuire, and Aaron Lefohn. 2018. Correlation-aware semi-analytic visibility for antialiased rendering. In Proceedings of the High-Performance Graphics Conference. 1–4.
[21]
Mark de Berg, Marc Van Kreveld, Mark Overmars, and Otfried Schwarzkopf. 1997. Computational Geometry. Springer-Verlag.
[22]
Frédo Durand. 1999. 3D Visibility: Analytical Study and Applications. Ph.D. Dissertation. Université Joseph Fourier.
[23]
Bernardt Duvenhage, Kadi Bouatouch, and Derrick Kourie. 2010. Exploring the use of glossy light volumes for interactive global illumination. In Proceedings of the International Conference on Computer Graphics, Virtual Reality, Visualisation and Interaction in Africa. 139–148.
[24]
Bernardt Duvenhage, Kadi Bouatouch, and Derrick G. Kourie. 2014. Light beam tracing for multi-bounce specular and glossy transport paths. In Proceedings of the Southern African Institute for Computer Scientist and Information Technologists Annual Conference. 199–208.
[25]
Jeff Erickson. 2000. Finite-resolution hidden surface removal. In Proceedings of the11th Annual ACM-SIAM Symposium on Discrete Algorithms.
[26]
Jon Ferraiolo, Fujisawa Jun, and Dean Jackson. 2000. Scalable Vector Graphics (SVG) 1.0 Specification.
[27]
Steven Fortune. 1999. Topological beam tracing. In Proceedings of the Symposium on Computational Geometry. 59–68.
[28]
Anka Gajentaan and Mark H. Overmars. 1995. On a class of O (n2) problems in computational geometry. Comput. Geom. 5, 3 (1995), 165–185.
[29]
Michael T. Goodrich. 1992. A polygonal approach to hidden-line and hidden-surface elimination. CVGIP: Graphical Models and Image Processing 54, 1 (1992), 1–12.
[30]
Craig Gotsman, Oded Sudarsky, and Jeffrey A. Fayman. 1999. Optimized occlusion culling using five-dimensional subdivision. Comput. Graph. 23, 5 (1999), 645–654.
[31]
Gaël Guennebaud, Benoît Jacob et al. 2010. Eigen v3. Retrieved from http://eigen.tuxfamily.org.
[32]
Denis Haumont, Otso Makinen, and Shaun Nirenstein. 2005. A low dimensional framework for exact polygon-to-polygon occlusion queries. In Eurographics Symposium on Rendering. The Eurographics Association, 211--222.
[33]
Paul S. Heckbert and Pat Hanrahan. 1984. Beam tracing polygonal objects. Comput. Graph. 18, 3 (1984), 119–127.
[34]
Eric Heitz, Jonathan Dupuy, Cyril Crassin, and Carsten Dachsbacher. 2015. The SGGX microflake distribution. ACM Trans. Graph. 34, 4 (2015), 1–11.
[35]
Eric Heitz, Jonathan Dupuy, Stephen Hill, and David Neubelt. 2016. Real-time polygonal-light shading with linearly transformed cosines. ACM Trans. Graph. 35, 4 (2016), 1–8.
[36]
Tom Hudson, Dinesh Manocha, Jonathan Cohen, Ming Lin, Kenneth Hoff, and Hansong Zhang. 1997. Accelerated occlusion culling using shadow frusta. In Proceedings of the Symposium on Computational Geometry. 1–10.
[37]
Homan Igehy. 1999. Tracing ray differentials. In Proceedings of The 26th Annual Conference on Computer Graphics And Interactive Techniques. 179–186.
[38]
Wenzel Jakob. 2010. Mitsuba renderer. Retrieved from http://www.mitsuba-renderer.org.
[39]
Wenzel Jakob. 2019. Enoki: Structured vectorization and differentiation on modern processor architectures. Retrieved from https://github.com/mitsuba-renderer/enoki.
[40]
André Jalobeanu, Frank O. Kuehnel, and John C. Stutz. 2004. Modeling images of natural 3d surfaces: Overview and potential applications. In Proceedings of the Conference on Computer Vision and Pattern Recognition Workshop. IEEE, 188–188.
[41]
Matthew J. Katz, Mark H. Overmars, and Micha Sharir. 1992. Efficient hidden surface removal for objects with small union size. Comput. Geom. 2, 4 (1992), 223–234.
[42]
Lutz Kettner, Kurt Mehlhorn, Sylvain Pion, Stefan Schirra, and Chee Yap. 2008. Classroom examples of robustness problems in geometric computations. Comput. Geom. 40, 1 (2008), 61–78.
[43]
Diederick P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations (ICLR’15).
[44]
Dale Kirkland, Bill Armstrong, Michael Gold, Jon Leech, and Paula Womack. 2002. ARB multisample. OpenGL Extension Registry (2002), 83–94.
[45]
Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, and Timo Aila. 2020. Modular primitives for high-performance differentiable rendering. ACM Trans. Graph. 39, 6 (2020).
[46]
Chen Li, Sylvain Pion, and Chee-Keng Yap. 2005. Recent progress in exact geometric computation. J. Logic Algebra. Program. 64, 1 (2005), 85–111.
[47]
Tzu-Mao Li, Miika Aittala, Frédo Durand, and Jaakko Lehtinen. 2018. Differentiable Monte Carlo ray tracing through edge sampling. ACM Trans. Graph. 37, 6 (2018), 222:1–222:11.
[48]
Tzu-Mao Li, Michal Lukáč, Gharbi Michaël, and Jonathan Ragan-Kelley. 2020. Differentiable vector graphics rasterization for editing and learning. ACM Trans. Graph. 39, 6 (2020), 193:1–193:15.
[49]
Baoquan Liu, Li-Yi Wei, Xu Yang, Chongyang Ma, Ying-Qing Xu, Baining Guo, and Enhua Wu. 2011. Non-linear beam tracing on a GPU. Comput. Graph. Forum 30, 8 (2011), 2156–2169.
[50]
Dong C. Liu and Jorge Nocedal. 1989. On the limited memory BFGS method for large scale optimization. Math. Program. 45, 1–3 (1989), 503–528.
[51]
Shichen Liu, Tianye Li, Weikai Chen, and Hao Li. 2019. Soft rasterizer: A differentiable renderer for image-based 3D reasoning. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’19).
[52]
Guillaume Loubet, Nicolas Holzschuch, and Wenzel Jakob. 2019. Reparameterizing discontinuous integrands for differentiable rendering. ACM Trans. Graph. 38, 6 (2019), 1–14.
[53]
Guillaume Loubet and Fabrice Neyret. 2017. Hybrid mesh-volume LoDs for all-scale pre-filtering of complex 3D assets. Comput. Graph. Forum 36, 2 (2017), 431–442.
[54]
Josiah Manson and Scott Schaefer. 2013. Analytic rasterization of curves with polynomial filters. Comput. Graph. Forum 32, 2 (2013), 499–507.
[55]
Michael McKenna. 1987. Worst-case optimal hidden-surface removal. Comput. Graph. 6, 1 (1987), 19–28.
[56]
Kurt Mehlhorn and Stefan Naher. 1995. LEDA: A platform for combinatorial and geometric computing. Commun. ACM 38, 1 (1995), 96–103.
[57]
Philipp Moritz, Robert Nishihara, and Michael Jordan. 2016. A linearly-convergent stochastic L-BFGS algorithm. In Proceeding sof the Conference on Artificial Intelligence and Statistics. 249–258.
[58]
Ketan Mulmuley. 1989. An efficient algorithm for hidden surface removal. In Proceedings of The 16th Annual Conference on Computer Graphics and Interactive Techniques. 379–388.
[59]
Ketan Mulmuley. 1994. An efficient algorithm for hidden surface removal, II. J. Comput. Syst. Sci. 49, 3 (1994), 427–453.
[60]
Merlin Nimier-David, Sébastien Speierer, Benoît Ruiz, and Wenzel Jakob. 2020. Radiative backpropagation: An adjoint method for lightning-fast differentiable rendering. ACM Trans. Graph. 39, 4, Article 146 (2020), 15 pages.
[61]
Shaun Nirenstein and Edwin Blake. 2004. Hardware accelerated visibility preprocessing using adaptive sampling. In Eurographics Workshop on Rendering. The Eurographics Association, 207--216.
[62]
Shaun Nirenstein, Edwin Blake, and James Gain. 2002. Exact from-region visibility culling. In Proceedings of the 13th Eurographics Workshop on Rendering.
[63]
Derek Nowrouzezahrai, Ilya Baran, Kenny Mitchell, and Wojciech Jarosz. 2014. Visibility silhouettes for semi-analytic spherical integration. Comput. Graph. Forum 33, 1 (2014), 105–117.
[64]
Otto Nurmi. 1985. A fast line-sweep algorithm for hidden line elimination. BIT Numer. Math. 25, 3 (1985), 466–472.
[65]
Joseph O’Rourke et al. 1998. Computational Geometry in C. Cambridge University Press.
[66]
Alexandrina Orzan, Adrien Bousseau, Holger Winnemöller, Pascal Barla, Joëlle Thollot, and David Salesin. 2008. Diffusion curves: A vector representation for smooth-shaded images. ACM Trans. Graph. 27, 3 (2008), 1–8.
[67]
Ryan Overbeck, Ravi Ramamoorthi, and William R. Mark. 2007. A real-time beam tracer with application to exact soft shadows. In Proceedings of the Rendering Techniques. The Eurographics Association, 85–98.
[68]
Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan-Yen Lo, Justin Johnson, and Georgia Gkioxari. 2020. Accelerating 3D deep learning with PyTorch3D. Retrieved from https://arXiv:2007.08501.
[69]
Michael Ian Shamos. 1978. Computational Geometry. Ph.D. Dissertation. Yale University.
[70]
Micha Sharir and Mark H. Overmars. 1992. A simple output-sensitive algorithm for hidden surface removal. Comput. Graph. 11, 1 (1992), 1–11.
[71]
Jonathan Richard Shewchuk. 2013. Lecture notes on geometric robustness. Retreived on December 2020 from https://people.eecs.berkeley.edu/∼jrs/meshpapers/robnotes.pdf.
[72]
A. James Stewart and Tasso Karkanis. 1998. Computing the approximate visibility map, with applications to form factors and discontinuity meshing. In Proceedings of the Eurographics Workshop on Rendering Techniques (EGWR’98). 57–68.
[73]
Ivan E. Sutherland, Robert F. Sproull, and Robert A. Schumacker. 1974. A characterization of ten hidden-surface algorithms. ACM Comput. Surveys 6, 1 (1974), 1–55.
[74]
The CGAL Project. 2020. CGAL User and Reference Manual (5.0.2 ed.). CGAL Editorial Board. Retrieved from https://doc.cgal.org/5.0.2/Manual/packages.html.
[75]
Csaba D. Toth, Joseph O’Rourke, and Jacob E. Goodman. 2017. Handbook of Discrete and Computational Geometry. Chapman and Hall/CRC.
[76]
Yuxiang Wang, Chris Wyman, Yong He, and Pradeep Sen. 2015. Decoupled coverage anti-aliasing. In Proceedings of the High-Performance Graphics Conference. 33–42.
[77]
Mark Watt. 1990. Light-water interaction using backward beam tracing. Comput. Graph. 24, 4 (1990), 377–385.
[78]
Kevin Weiler and Peter Atherton. 1977. Hidden surface removal using polygon area sorting. Comput. Graph. 11, 2 (1977), 214–222.
[79]
Lifan Wu, Shuang Zhao, Ling-Qi Yan, and Ravi Ramamoorthi. 2019. Accurate appearance preserving prefiltering for rendering displacement-mapped surfaces. ACM Trans. Graph. 38, 4 (2019), 1–14.
[80]
Chris Wyman and Morgan McGuire. 2017. Hashed alpha testing. In Proceedings of the the Symposium on Interactive 3D Graphics and Games. 1–9.
[81]
Lei Yang, Shiqiu Liu, and Marco Salvi. 2020. A survey of temporal antialiasing techniques. Comput. Graph. Forum 39, 2 (2020), 607–621.
[82]
Chee-Keng Yap. 1997. Towards exact geometric computation. Comput. Geom. 7, 1–2 (1997), 3–23.
[83]
Cheng Zhang, Bailey Miller, Kai Yan, Ioannis Gkioulekas, and Shuang Zhao. 2020. Path-space differentiable rendering. ACM Trans. Graph. 39, 4 (2020), 143:1–143:19.
[84]
Cheng Zhang, Lifan Wu, Changxi Zheng, Ioannis Gkioulekas, Ravi Ramamoorthi, and Shuang Zhao. 2019. A differential theory of radiative transfer. ACM Trans. Graph. 38, 6 (2019), 227:1–227:16.
[85]
Shuang Zhao, Frédo Durand, and Changxi Zheng. 2017. Inverse diffusion curves using shape optimization. IEEE Trans. Vis. Comput. Graph. 24, 7 (2017), 2153–2166.
[86]
Shuang Zhao, Lifan Wu, Frédo Durand, and Ravi Ramamoorthi. 2016. Downsampling scattering parameters for rendering anisotropic media. ACM Trans. Graph. 35, 6 (2016).

Cited By

View all

Index Terms

  1. Vectorization for Fast, Analytic, and Differentiable Visibility

      Recommendations

      Comments

      Please enable JavaScript to view thecomments powered by Disqus.

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Graphics
      ACM Transactions on Graphics  Volume 40, Issue 3
      June 2021
      264 pages
      ISSN:0730-0301
      EISSN:1557-7368
      DOI:10.1145/3463476
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 15 July 2021
      Accepted: 01 February 2021
      Revised: 01 February 2021
      Received: 01 December 2020
      Published in TOG Volume 40, Issue 3

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Vectorization
      2. beam tracing
      3. anti-aliasing
      4. differentiable rendering

      Qualifiers

      • Research-article
      • Refereed

      Funding Sources

      • NVIDIA Fellowship
      • Ronald L. Graham Chair
      • UC San Diego Center for Visual Computing

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)500
      • Downloads (Last 6 weeks)67
      Reflects downloads up to 04 Oct 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Conditional Mixture Path Guiding for Differentiable RenderingACM Transactions on Graphics10.1145/365813343:4(1-11)Online publication date: 19-Jul-2024
      • (2024)Residual path integrals for re‐renderingComputer Graphics Forum10.1111/cgf.1515243:4Online publication date: 24-Jul-2024
      • (2024)CloVR: Fast-Startup Low-Latency Cloud VRIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337205930:5(2337-2346)Online publication date: 4-Mar-2024
      • (2024)Efficient and Robust From-Point VisibilityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.329113830:8(5313-5327)Online publication date: 1-Aug-2024
      • (2024)Temporal vectorized visibility for direct illumination of animated modelsComputational Visual Media10.1007/s41095-023-0339-3Online publication date: 29-May-2024
      • (2023)Projective Sampling for Differentiable Rendering of GeometryACM Transactions on Graphics10.1145/361838542:6(1-14)Online publication date: 5-Dec-2023
      • (2023)Differentiable Heightfield Path Tracing with Accelerated DiscontinuitiesACM SIGGRAPH 2023 Conference Proceedings10.1145/3588432.3591530(1-9)Online publication date: 23-Jul-2023
      • (2023)ELTRA: An Embedding Method based on Learning-to-Rank to Preserve Asymmetric Information in Directed GraphsProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614862(2116-2125)Online publication date: 21-Oct-2023
      • (2023)A Differentiable Image Source Model for Room Acoustics Optimization2023 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)10.1109/WASPAA58266.2023.10248140(1-5)Online publication date: 22-Oct-2023
      • (2023)Free Energy Node Embedding via Generalized Skip-Gram With Negative SamplingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.320617535:8(8024-8036)Online publication date: 1-Aug-2023
      • Show More Cited By

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Get Access

      Login options

      Full Access

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media