article

Analysis of human faces using a measurement-based skin reflectance model

Authors:

Wojciech Matusik,

Hanspeter Pfister,

Janet McAndless,

Henrik Wann Jensen,

Markus GrossAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 25, Issue 3

Pages 1013 - 1024

https://doi.org/10.1145/1141911.1141987

Published: 01 July 2006 Publication History

Abstract

We have measured 3D face geometry, skin reflectance, and subsurface scattering using custom-built devices for 149 subjects of varying age, gender, and race. We developed a novel skin reflectance model whose parameters can be estimated from measurements. The model decomposes the large amount of measured skin data into a spatially-varying analytic BRDF, a diffuse albedo map, and diffuse subsurface scattering. Our model is intuitive, physically plausible, and -- since we do not use the original measured data -- easy to edit as well. High-quality renderings come close to reproducing real photographs. The analysis of the model parameters for our sample population reveals variations according to subject age, gender, skin type, and external factors (e.g., sweat, cold, or makeup). Using our statistics, a user can edit the overall appearance of a face (e.g., changing skin type and age) or change small-scale features using texture synthesis (e.g., adding moles and freckles). We are making the collected statistics publicly available to the research community for applications in face synthesis and analysis.

Supplementary Material

JPG File (p1013-weyrich-high.jpg)

Download
12.82 KB

JPG File (p1013-weyrich-low.jpg)

Download
11.12 KB

High Resolution (p1013-weyrich-high.mov)

Download
135.67 MB

Low Resolution (p1013-weyrich-low.mov)

Download
30.77 MB

References

[1]

Angelopoulou, E., Molana, R., and Daniilidis, K. 2001. Multispectral skin color modeling. In IEEE Conf. on Comp. Vision and Pattern Rec. (CVPR), 635--642.

[2]

Barsky, S., and Petrou, M. 2001. Colour photometric stereo: simultaneous reconstruction of local gradient and colour of rough textured surfaces. In Eighth IEEE International Conference on Computer Vision, vol. 2, 600--605.

[3]

Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. Computer Graphics 33, Annual Conference Series, 187--194.

Digital Library

[4]

Blinn, J. 1977. Models of light reflection for computer synthesized pictures. Computer Graphics 11, Annual Conference Series, 192--198.

Digital Library

[5]

Borshukov, G., and Lewis, J. 2003. Realistic human face rendering for the matix reloaded. In ACM SIGGRAPH 2003 Conference Abstracts and Applications (Sketch).

Digital Library

[6]

Cula, O., and Dana, K. 2002. Image-based skin analysis. In Texture 2002, The Second International Workshop on Texture Analysis and Synthesis, 35--42.

[7]

Cula, O., Dana, K., Murphy, F., and Rao, B. 2004. Bidirectional imaging and modeling of skin texture. IEEE Transactions on Biomedical Engineering 51, 12 (Dec.), 2148--2159.

[8]

Cula, O., Dana, K., Murphy, F., and Rao, B. 2005. Skin texture modeling. International Journal of Computer Vision 62, 1--2 (April--May), 97--119.

Digital Library

[9]

Dana, K. J., van Ginneken, B., Nayar, S. K., and Koen-Derink, J. J. 1999. Reflectance and texture of real world surfaces. ACM Transactions on Graphics 1, 18, 1--34.

Digital Library

[10]

Debevec, P., and Malik, J. 1997. Recovering high dynamic range radiance maps from photographs. In Computer Graphics, SIGGRAPH 97 Proceedings, 369--378.

Digital Library

[11]

Debevec, P., Taylor, C., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. In Computer Graphics, SIGGRAPH 96 Proceedings, 11--20.

Digital Library

[12]

Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. In Computer Graphics, SIGGRAPH 2000 Proceedings, 145--156.

Digital Library

[13]

Debevec, P., Wenger, A., Tchou, C., Gardner, A., Waese, J., and Hawkins, T. 2002. A lighting reproduction approach to live-action compositing. ACM Transactions on Graphics (SIGGRAPH 2002) 21, 3 (July), 547--556.

Digital Library

[14]

Donner, C., and Jensen, H. W. 2005. Light diffusion in multi-layered translucent materials. ACM Transactions on Graphics 24, 3 (July), 1032--1039.

Digital Library

[15]

Fitzpatrick, T. 1988. The validity and practicality of sun-reactive skin types i through vi. Arch. Dermatology 124, 6 (June), 869--871.

[16]

Fuchs, C., Goesele, M., Chen, T., and Seidel, H.-P. 2005. An empirical model for heterogeneous translucent objects. Tech. Rep. MPI-I-2005-4-006, Max-Planck Institute für Informatik.

[17]

Fuchs, M., Blanz, V., Lensch, H., and Seidel, H.-P. 2005. Reflectance from images: A model-based approach for human faces. Research Report MPI-I-2005-4-001, Max-Planck-Institut für Informatik, Stuhlsatzenhausweg 85, 66123 Saarbrücken, Germany. Accepted for publication in IEEE TVCG.

[18]

Georghiades, A., Belhumeur, P., and Kriegman, D. 1999. Illumination-based image synthesis: Creating novel images of human faces under differing pose and lighting. In IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes, 47--54.

Digital Library

[19]

Georghiades, A. 2003. Recovering 3-D shape and reflectance from a small number of photographs. In Rendering Techniques, 230--240.

Digital Library

[20]

Goesele, M., Lensch, H., Lang, J., Fuchs, C., and Seidel, H. 2004. Disco -- acquisition of translucent objects. ACM Transactions on Graphics 24, 3, 835--844.

Digital Library

[21]

Green, P., Halbert, M., and Robinson, P. 1966. Canonical analysis: An exposition and illustrative application. Marketing Research 3 (Feb.), 32--39.

[22]

Hanrahan, P., and Krueger, W. 1993. Reflection from layered surfaces due to subsurface scattering. In Computer Graphics, SIGGRAPH 93 Proceedings, 165--174.

Digital Library

[23]

Hawkins, T., Wenger, A., Tchou, C., Goransson, F., and Debevec, P. 2004. Animatable facial reflectance fields. In Rendering Techniques '04 (Proceedings of the Second Eurographics Symposium on Rendering).

Digital Library

[24]

Heeger, D., and Bergen, J. 1995. Pyramid-based texture analysis/synthesis. In Proceedings of SIGGRAPH 95, Computer Graphics Proceedings, Annual Conference Series, 229--238.

Digital Library

[25]

Hery, C., 2005. Face cloning at ILM. Digital Face Cloning, Siggraph Course Notes.

[26]

Igarashi, T., Nishino, K., and Nayar, S. 2005. The appearance of human skin. Tech. Rep. CUCS-024-05, Department of Computer Science, Columbia University, June.

[27]

Jensen, H. W., and Buhler, J. 2002. A rapid hierarchical rendering technique for translucent materials. In Computer Graphics, SIGGRAPH 2002 Proceedings, 576--581.

Digital Library

[28]

Jensen, H. W., Marschner, S. R., Levoy, M., and Hanrahan, P. 2001. A practical model for subsurface light transport. In Computer Graphics, SIGGRAPH 2001 Proceedings, 511--518.

Digital Library

[29]

Jones, A., Gardner, A., Bolas, M., McDowall, I., and Debevec, P. 2005. Performance geometry capture for spatially varying relighting. In ACM SIGGRAPH 2005 Conference Abstracts and Applications (Sketch).

Digital Library

[30]

Koenderink, J., and Pont, S. 2003. The secret of velvety skin. Machine Vision and Application, 14, 260--268. Special Issue on Human Modeling, Analysis, and Synthesis.

Digital Library

[31]

Krishnaswamy, A., and Baranoski, G. 2004. A biophysically-based spectral model of light interaction with human skin. Computer Graphics Forum 23, 3 (Sept.), 331--340.

[32]

Lafortune, E., Foo, S.-C., Torrance, K., and Greenberg, D. 1997. Non-linear approximation of reflectance functions. Computer Graphics 31, Annual Conference Series, 117--126.

Digital Library

[33]

Larsson, M., Nilsson, H., and Strömberg, T. 2003. In vivo determination of local skin optical properties and photon path length by use of spatially resolved diffuse reflectance with applications in laser doppler flowmetry. Applied Optics 42, 1 (Jan.), 124--134.

[34]

Lawrence, J., Rusinkiewicz, S., and Ramamoorthi, R. 2004. Efficient brdf importance sampling using a factored representation. ACM Transactions on Graphics 23, 3, 496--505.

Digital Library

[35]

Lensch, H., Kautz, J., Goesele, M., Heidrich, W., and Seidel, H.-P. 2001. Image-based reconstruction of spatially varying materials. In Proceedings of the 12th Eurographics Workshop on Rendering, 104--115.

Digital Library

[36]

Loop, C. 1987. Smooth Subdivision Surfaces based on Triangles. Master's thesis, Department of Mathematics, University of Utah.

[37]

Marschner, S., Westin, S., Lafortune, E., Torrance, K., and Greenberg, D. 1999. Image-based BRDF measurement including human skin. In Proceedings of the 10th Eurographics Workshop on Rendering, 139--152.

Digital Library

[38]

Marschner, S., Guneter, B., and Raghupathy, S. 2000. Modeling and rendering for realistic facial animation. In 11th Eurographics Workshop on Rendering, 231--242.

Digital Library

[39]

Marschner, S., Westin, S., Lafortune, E., and Torrance, K. 2000. Image-based measurement of the Bidirectional Reflectance Distribution Function. Applied Optics 39, 16 (June), 2592--2600.

[40]

Matusik, W., Zwicker, M., and Durand, F. 2005. Texture design using a simplicial complex of morphable textures. ACM Transactions on Graphics 24, 3, 787--794.

Digital Library

[41]

Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3d geometry. ACM Transactions on Graphics 24, 3, 536--543.

Digital Library

[42]

Ng, C., and Li, L. 2001. A multi-layered reflection model of natural human skin. In Computer Graphics International, 249256.

Digital Library

[43]

Ngan, A., Durand, F., and Matusik, W. 2005. Experimental analysis of brdf models. In Eurographics Symposium on Rendering, 117--126.

Digital Library

[44]

Nickell, S., Hermann, M., Essenpreis, M., Farrell, T. J., Kramer, U., and Patterson, M. S. 2000. Anisotropy of light propagation in human skin. Phys. Med. Biol. 45, 2873--2886.

[45]

Nicodemus, F., Richmond, J., Hsia, J., Ginsberg, I., and Limperis, T. 1977. Geometric considerations and nomenclature for reflectance. Monograph 160, National Bureau of Standards (US), October.

[46]

Paris, S., Sillion, F., and Quan, L. 2003. Lightweight face relighting. In Proceedings of Pacific Graphics, 41--50.

Digital Library

[47]

Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., and Salesin, D. 1998. Synthesizing realistic facial expressions from photographs. In Computer Graphics, vol. 32 of SIGGRAPH 98 Proceedings, 75--84.

Digital Library

[48]

Sander, P., Gosselin, D., and Mitchell, J. 2004. Real-time skin rendering on graphics hardware. SIGGRAPH 2004 Sketch.

Digital Library

[49]

Sheffer, A., Levy, B., Mogilnitsky, M., and Bogomyakov, A. 2005. ABF++: Fast and robust angle based flattening. ACM Transactions on Graphics 24, 2, 311--330.

Digital Library

[50]

Stam, J. 2001. An illumination model for a skin layer bounded by rough surfaces. In Proceedings of the 12th Eurographics Workshop on Rendering Techniques, Springer, Wien, Vienna, Austria, 39--52.

Digital Library

[51]

Tong, X., Wang, J., Lin, S., Guo, B., and Shum, H.-Y. 2005. Modeling and rendering of quasi-homogeneous materials. ACM Transactions on Graphics 24, 3, 1054--1061.

Digital Library

[52]

Torrance, K. E., and Sparrow, E. M. 1967. Theory for off-specular reflection from roughened surfaces. Journal of the Optical Society of America 57, 1104--1114.

[53]

Tsumura, N., Ojima, N., Sato, K., Shiraishi, M., Shimizu, H., Nabeshima, H., Akazaki, S., Hori, K., and Miyake, Y. 2003. Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin. ACM Transactions on Graphics 22, 3, 770--779.

Digital Library

[54]

Tuchin, V. 2000. Tissue Optics: Light Scattering Methods and Instruments for Medical Diagnosis. SPIE Press.

[55]

Wenger, A., Gardner, A., Tchou, C., Unger, J., Hawkins, T., and Debevec, P. 2005. Performance relighting and reflectance transformation with time-multiplexed illumination. ACM Transactions on Graphics 24, 3, 756--764.

Digital Library

[56]

Weyrich, T., Matusik, W., Pfister, H., Ngan, A., and Gross, M. 2005. Measuring skin reflectance and subsurface scattering. Tech. Rep. TR2005-046, Mitsubishi Electric Research Laboratories (MERL).

[57]

Williams, L. 2005. Case study: The gemini man. In SIGGRAPH 2005 Course 'Digital Face Cloning'.

Cited By

Lin HLuo C(2025)Demonstration of computational ghost imaging through fogOptics & Laser Technology10.1016/j.optlastec.2024.112075182(112075)Online publication date: Apr-2025
https://doi.org/10.1016/j.optlastec.2024.112075
Pranovich AHannemose MJensen JTran DAanæs HGooran SNyström DFrisvad J(2024)Digitizing the Appearance of 3D Printing Materials Using a SpectrophotometerSensors10.3390/s2421702524:21(7025)Online publication date: 31-Oct-2024
https://doi.org/10.3390/s24217025
Ha HHwang IMonzon NCho JKim DBaek SMuñoz AGutierrez DKim M(2024)Polarimetric BSSRDF Acquisition of Dynamic FacesACM Transactions on Graphics10.1145/368776743:6(1-11)Online publication date: 19-Dec-2024
https://dl.acm.org/doi/10.1145/3687767
Show More Cited By

Index Terms

Analysis of human faces using a measurement-based skin reflectance model
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding

Recommendations

A practical model for subsurface light transport
SIGGRAPH '01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques

This paper introduces a simple model for subsurface light transport in translucent materials. The model enables efficient simulation of effects that BRDF models cannot capture, such as color bleeding within materials and diffusion of light across shadow ...
Acquiring the reflectance field of a human face
SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques

We present a method to acquire the reflectance field of a human face and use these measurements to render the face under arbitrary changes in lighting and viewpoint. We first acquire images of the face from a small set of viewpoints under a dense ...
A layered, heterogeneous reflectance model for acquiring and rendering human skin

We introduce a layered, heterogeneous spectral reflectance model for human skin. The model captures the inter-scattering of light among layers, each of which may have an independent set of spatially-varying absorption and scattering parameters. For ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 25, Issue 3

July 2006

742 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1141911

Issue’s Table of Contents

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2006

Published in TOG Volume 25, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

250
Total Citations
View Citations
3,424
Total Downloads

Downloads (Last 12 months)114
Downloads (Last 6 weeks)17

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lin HLuo C(2025)Demonstration of computational ghost imaging through fogOptics & Laser Technology10.1016/j.optlastec.2024.112075182(112075)Online publication date: Apr-2025
https://doi.org/10.1016/j.optlastec.2024.112075
Pranovich AHannemose MJensen JTran DAanæs HGooran SNyström DFrisvad J(2024)Digitizing the Appearance of 3D Printing Materials Using a SpectrophotometerSensors10.3390/s2421702524:21(7025)Online publication date: 31-Oct-2024
https://doi.org/10.3390/s24217025
Ha HHwang IMonzon NCho JKim DBaek SMuñoz AGutierrez DKim M(2024)Polarimetric BSSRDF Acquisition of Dynamic FacesACM Transactions on Graphics10.1145/368776743:6(1-11)Online publication date: 19-Dec-2024
https://dl.acm.org/doi/10.1145/3687767
Kavoosighafi BHajisharif SMiandji EBaravdish GCao WUnger J(2024)Deep SVBRDF Acquisition and Modelling: A SurveyComputer Graphics Forum10.1111/cgf.1519943:6Online publication date: 16-Sep-2024
https://doi.org/10.1111/cgf.15199
Choi SYoon SNam GLee SBaek S(2024)Differentiable Display Photometric Stereo2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01124(11831-11840)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01124
Han YLyu JXu F(2024)High-Quality Facial Geometry and Appearance Capture at Home2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00073(697-707)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00073
Saito SSchwartz GSimon TLi JNam G(2024)Relightable Gaussian Codec Avatars2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00021(130-141)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00021
Chen ZMoon GGuo KCao CPidhorskyi SSimon TJoshi RDong YXu YPires BWen HEvans LPeng BBuffalini JTrimble AMcPhail KSchoeller MYu SRomero JZollhöfer MSheikh YLiu ZSaito S(2024)URHand: Universal Relightable Hands2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00020(119-129)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00020
Rao PMallikarjun BFox GWeyrich TBickel BPfister HMatusik WZhan FTewari ATheobalt CElgharib M(2024)A Deeper Analysis of Volumetric Relightable FacesInternational Journal of Computer Vision10.1007/s11263-023-01899-3132:4(1148-1166)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s11263-023-01899-3
Kim JYoo T(2023)A Study on the Digital Twin Pipeline for Facial Digitizing System OptimizationJOURNAL OF BROADCAST ENGINEERING10.5909/JBE.2023.28.5.53028:5(530-544)Online publication date: 30-Sep-2023
https://doi.org/10.5909/JBE.2023.28.5.530
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents