research-article

High-quality single-shot capture of facial geometry

Authors:

Paul Beardsley,

Markus GrossAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 29, Issue 4

Article No.: 40, Pages 1 - 9

https://doi.org/10.1145/1778765.1778777

Published: 26 July 2010 Publication History

Abstract

This paper describes a passive stereo system for capturing the 3D geometry of a face in a single-shot under standard light sources. The system is low-cost and easy to deploy. Results are submillimeter accurate and commensurate with those from state-of-the-art systems based on active lighting, and the models meet the quality requirements of a demanding domain like the movie industry. Recovered models are shown for captures from both high-end cameras in a studio setting and from a consumer binocular-stereo camera, demonstrating scalability across a spectrum of camera deployments, and showing the potential for 3D face modeling to move beyond the professional arena and into the emerging consumer market in stereoscopic photography.

Our primary technical contribution is a modification of standard stereo refinement methods to capture pore-scale geometry, using a qualitative approach that produces visually realistic results. The second technical contribution is a calibration method suited to face capture systems. The systemic contribution includes multiple demonstrations of system robustness and quality. These include capture in a studio setup, capture off a consumer binocular-stereo camera, scanning of faces of varying gender and ethnicity and age, capture of highly-transient facial expression, and scanning a physical mask to provide ground-truth validation.

Supplementary Material

Supplemental material. (040.zip)

Download
164.97 MB

References

[1]

Alexander, O., Rogers, M., Lambeth, W., Chiang, M., and Debevec, P. 2009. The Digital Emily Project: Photoreal facial modeling and animation. ACM Trans. Graph.

[2]

Beeler, T., Bickel, B., Beardsley, P., Sumner, B., and Gross, M. 2010. High-quality single shot capture of facial geometry: Implementation details. Tech. Rep. 671, ETH Zurich.

[3]

Bradley, D., Heidrich, W., Popa, T., and Sheffer, A. 2010. High resolution passive facial performance capture. ACM Trans. Graph.

Digital Library

[4]

Chen, T., Goesele, M., and Seidel, H. 2006. Mesostructure from specularity. CVPR.

Digital Library

[5]

DI3D. 2009. Dimensional imaging. http://www.di3d.com.

[6]

Donner, C., and Jensen, H. 2006. A spectral bssrdf for shading human skin. Eurographics Symposium on Rendering.

[7]

Donner, C., Weyrich, T., d'Eon, E., Ramamoorthi, R., and Rusinkiewicz, S. 2008. A layered, heterogeneous reflectance model for acquiring and rendering human skin. ACM Trans. Graph.

Digital Library

[8]

Furukawa, Y., and Ponce, J. 2007. Accurate, dense, and robust multi-view stereopsis. CVPR.

[9]

Glencross, M., Ward, G., Melendez, F., Jay, C., Liu, J., and Hubbold, R. 2008. A perceptually validated model for surface depth hallucination. ACM Trans. Graph.

Digital Library

[10]

Golovinskiy, A., Matusik, W., Pfister, H., Rusinkiewicz, S., and Funkhouser, T. 2006. A statistical model for synthesis of detailed facial geometry. ACM Trans. Graph.

Digital Library

[11]

Hartley, R., and Zisserman, A. 2000. Multiple View Geometry, second ed. Cambridge University Press.

Digital Library

[12]

Hernandez, C., Vogiatzis, G., and Cipolla, R. 2008. Shadows in three-source photometric stereo. ECCV.

Digital Library

[13]

Hiep, V., Keriven, R., Labatut, P., and Pons, J. 2009. Towards high-resolution large-scale multi-view stereo. CVPR.

[14]

Hyneman, W., Itokazu, H., Williams, L., and Zhao, X. 2005. Human face project. SIGGRAPH 2005 Courses.

Digital Library

[15]

Intel. 2001. Opencv reference manual. http://developer.intel.com.

[16]

Kazhdan, M., Bolitho, M., and Hoppe, H. 2006. Poisson surface reconstruction. In SGP.

Digital Library

[17]

Ma, W., Hawkins, T., Peers, P., Chabert, C., Weiss, M., and Debevec, P. 2007. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. Rendering Techniques.

Digital Library

[18]

Merrell, P., Akbarzadeh, A., Wang, L., Mordohai, P., Frahm, J., Yang, R., Nister, D., and Pollefeys, M. 2007. Real-time visibility-based fusion of depth maps. ICCV.

[19]

Meyer, M., Desbrun, M., Schröder, P., and Barr, A. H. 2003. Discrete differential-geometry operators for triangulated 2-manifolds. In Visualization and Mathematics III.

[20]

Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3d geometry. ACM Trans. Graph.

Digital Library

[21]

Parke, F. 1974. A parametric model for human faces. PhD Thesis, University of Utah.

Digital Library

[22]

Pighin, F., and Lewis, J. 2005. Digital face cloning. ACM Trans. Graph.

[23]

Robert, L., and Deriche, R. 1996. Dense depth map reconstruction: A minimization and regularization approach which preserves discontinuities. In ECCV.

Digital Library

[24]

Scharstein, D., and Szeliski, R. 1996. Stereo matching with non-linear diffusion. CVPR.

Digital Library

[25]

Scharstein, D., and Szeliski, R. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV.

Digital Library

[26]

Seitz, S., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. 2006. A comparison and evaluation of multi-view stereo reconstruction algorithms. CVPR.

Digital Library

[27]

Sifakis, E., Neverov, I., and Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph.

Digital Library

[28]

Svoboda, T. Multi camera self-calibration. http://cmp.felk.cvut.cz/svoboda/SelfCal/index.html.

[29]

Torralba, A., and Freeman, W. 2003. Properties and applications of shape recipes. CVPR.

[30]

Weise, T., Leibe, B., and Gool, L. V. 2007. Fast 3D scanning with automatic motion compensation. CVPR.

[31]

Weise, T., Li, H., Gool, L., and Pauly, M. 2009. Face/off: live facial puppetry. SCA.

Digital Library

[32]

Weyrich, T., Matusik, W., Pfister, H., Bickel, B., Donner, C., Tu, C., McAndless, J., Lee, J., Ngan, A., Jensen, H., and Gross, M. 2006. Analysis of human faces using a measurement-based skin reflectance model. ACM Trans. Graph.

Digital Library

[33]

Woodford, O., Torr, P., Reid, I., and Fitzgibbon, A. 2008. Global stereo reconstruction under second order smoothness priors. CVPR.

Cited By

Singh PBornstein MHsung RAjmera DLeung YGu M(2024)Frontiers in Three-Dimensional Surface Imaging Systems for 3D Face Acquisition in Craniofacial Research and Practice: An Updated Literature ReviewDiagnostics10.3390/diagnostics1404042314:4(423)Online publication date: 14-Feb-2024
https://doi.org/10.3390/diagnostics14040423
Tian ZLiang BFang HWeng D(2024)Dynamic 4D facial capture pipeline with appearance driven progressive retopology based on optical flowOptics Express10.1364/OE.52984632:18(31830)Online publication date: 19-Aug-2024
https://doi.org/10.1364/OE.529846
Bridgeman LRainer GGhosh A(2024)High-Quality Facial Geometry from Sparse Heterogeneous Devices under Active IlluminationProceedings of 21st ACM SIGGRAPH Conference on Visual Media Production10.1145/3697294.3697296(1-10)Online publication date: 18-Nov-2024
https://dl.acm.org/doi/10.1145/3697294.3697296
Show More Cited By

Index Terms

High-quality single-shot capture of facial geometry
1. Computer systems organization
  1. Architectures
    1. Other architectures
      1. Special purpose systems
2. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Image and video acquisition
  2. Computer graphics
    1. Graphics systems and interfaces
    2. Image manipulation
      1. Texturing

Recommendations

Single-shot high-quality facial geometry and skin appearance capture

We propose a new light-weight face capture system capable of reconstructing both high-quality geometry and detailed appearance maps from a single exposure. Unlike currently employed appearance acquisition systems, the proposed technology does not require ...
High-quality single-shot capture of facial geometry
SIGGRAPH '10: ACM SIGGRAPH 2010 papers

This paper describes a passive stereo system for capturing the 3D geometry of a face in a single-shot under standard light sources. The system is low-cost and easy to deploy. Results are submillimeter accurate and commensurate with those from state-of-...
Epipolar geometry for prism-based single-lens stereovision

In order to simplify the design and implementation of a stereo vision system, prism has been used to capture stereo images with a single camera. This kind of system not only provides advantages over traditional two-camera stereo, but also reduces the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 29, Issue 4

July 2010

942 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1778765

Issue’s Table of Contents

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 July 2010

Published in TOG Volume 29, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

250
Total Citations
View Citations
3,985
Total Downloads

Downloads (Last 12 months)46
Downloads (Last 6 weeks)9

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Singh PBornstein MHsung RAjmera DLeung YGu M(2024)Frontiers in Three-Dimensional Surface Imaging Systems for 3D Face Acquisition in Craniofacial Research and Practice: An Updated Literature ReviewDiagnostics10.3390/diagnostics1404042314:4(423)Online publication date: 14-Feb-2024
https://doi.org/10.3390/diagnostics14040423
Tian ZLiang BFang HWeng D(2024)Dynamic 4D facial capture pipeline with appearance driven progressive retopology based on optical flowOptics Express10.1364/OE.52984632:18(31830)Online publication date: 19-Aug-2024
https://doi.org/10.1364/OE.529846
Bridgeman LRainer GGhosh A(2024)High-Quality Facial Geometry from Sparse Heterogeneous Devices under Active IlluminationProceedings of 21st ACM SIGGRAPH Conference on Visual Media Production10.1145/3697294.3697296(1-10)Online publication date: 18-Nov-2024
https://dl.acm.org/doi/10.1145/3697294.3697296
Chen YWang LLi QXiao HZhang SYao HLiu Y(2024)MonoGaussianAvatar: Monocular Gaussian Point-based Head AvatarACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657499(1-9)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657499
Weiss SStanhope JChandran PZoss GBradley D(2024)Stylize My Wrinkles: Bridging the Gap from Simulation to RealityComputer Graphics Forum10.1111/cgf.1504843:2Online publication date: 15-May-2024
https://doi.org/10.1111/cgf.15048
Cong MLan LFedkiw R(2024)Local Geometric Indexing of High Resolution Data for Facial Reconstruction From Sparse MarkersIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.328949530:8(5289-5298)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3289495
Xu YChen BLi ZZhang HWang LZheng ZLiu Y(2024)Gaussian Head Avatar: Ultra High-Fidelity Head Avatar via Dynamic Gaussians2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00189(1931-1941)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00189
Deng BWang YWetzstein G(2024)LumiGAN: Unconditional Generation of Relightable 3D Human Faces2024 International Conference on 3D Vision (3DV)10.1109/3DV62453.2024.00081(302-312)Online publication date: 18-Mar-2024
https://doi.org/10.1109/3DV62453.2024.00081
Kabadayi BZielonka WBhatnagar BPons-Moll GThies J(2024)GAN-Avatar: Controllable Personalized GAN-based Human Head Avatar2024 International Conference on 3D Vision (3DV)10.1109/3DV62453.2024.00058(882-892)Online publication date: 18-Mar-2024
https://doi.org/10.1109/3DV62453.2024.00058
Kim JYoo T(2023)A Study on the Digital Twin Pipeline for Facial Digitizing System OptimizationJOURNAL OF BROADCAST ENGINEERING10.5909/JBE.2023.28.5.53028:5(530-544)Online publication date: 30-Sep-2023
https://doi.org/10.5909/JBE.2023.28.5.530
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents