research-article

Physical face cloning

Authors:

Peter Kaufmann,

Mélina Skouras,

Bernhard Thomaszewski,

Steve Marschner,

Wojciech Matusik,

Markus GrossAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 31, Issue 4

Article No.: 118, Pages 1 - 10

https://doi.org/10.1145/2185520.2185614

Published: 01 July 2012 Publication History

Abstract

We propose a complete process for designing, simulating, and fabricating synthetic skin for an animatronics character that mimics the face of a given subject and its expressions. The process starts with measuring the elastic properties of a material used to manufacture synthetic soft tissue. Given these measurements we use physics-based simulation to predict the behavior of a face when it is driven by the underlying robotic actuation. Next, we capture 3D facial expressions for a given target subject. As the key component of our process, we present a novel optimization scheme that determines the shape of the synthetic skin as well as the actuation parameters that provide the best match to the target expressions. We demonstrate this computational skin design by physically cloning a real human face onto an animatronics figure.

References

[1]

Becker, M., and Teschner, M. 2007. Robust and efficient estimation of elasticity parameters using the linear finite element method. In SimVis, 15--28.

[2]

Beeler, T., Bickel, B., Beardsley, P., Sumner, B., and Gross, M. 2010. High-quality single-shot capture of facial geometry. ACM Trans. Graph. 29, 4 (July), 40:1--40:9.

Digital Library

[3]

Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Sumner, R. W., and Gross, M. 2011. High-quality passive facial performance capture using anchor frames. ACM Trans. Graph. 30, 4 (August), 75:1--75:10.

Digital Library

[4]

Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., and Gross, M. 2007. Multi-scale capture of facial geometry and motion. ACM Trans. Graph. 26, 3 (July), 33:1--33:10.

Digital Library

[5]

Bickel, B., Bächer, M., Otaduy, M. A., Matusik, W., Pfister, H., and Gross, M. 2009. Capture and modeling of non-linear heterogeneous soft tissue. ACM Trans. Graph. 28, 3 (July), 89:1--89:9.

Digital Library

[6]

Bickel, B., Bächer, M., Otaduy, M. A., Lee, H. R., Pfister, H., Gross, M., and Matusik, W. 2010. Design and fabrication of materials with desired deformation behavior. ACM Trans. Graph. 29, 4 (July), 63:1--63:10.

Digital Library

[7]

Bonet, J., and Wood, R. D. 1997. Nonlinear Continuum Mechanics for Finite Element Analysis. Cambridge Uni. Press.

[8]

Bradley, D., Boubekeur, T., and Heidrich, W. 2008. Accurate multi-view reconstruction using robust binocular stereo and surface meshing. In Proc. CVPR.

[9]

Bradley, D., Heidrich, W., Popa, T., and Sheffer, A. 2010. High resolution passive facial performance capture. ACM Trans. Graph. 29, 4 (July), 41:1--41:10.

Digital Library

[10]

Bucur, D., and Buttazzo, G. 2005. Variational Methods in Shape Optimization Problems. Birkhuser Mathematics.

[11]

Chentanez, N., Alterovitz, R., Ritchie, D., Cho, L., Hauser, K. K., Goldberg, K., Shewchuk, J. R., and O'Brien, J. F. 2009. Interactive simulation of surgical needle insertion and steering. ACM Trans. Graph. 28, 3 (July), 88:1--88:10.

Digital Library

[12]

Clarberg, P., Jarosz, W., Akenine-Möller, T., and Jensen, H. W. 2005. Wavelet importance sampling: efficiently evaluating products of complex functions. ACM Trans. Graph. 24, 3 (Aug.), 1166--1175.

Digital Library

[13]

Dong, Y., Wang, J., Pellacini, F., Tong, X., and Guo, B. 2010. Fabricating spatially-varying subsurface scattering. ACM Trans. Graph. 29 (July), 62:1--62:10.

Digital Library

[14]

Gourret, J.-P., Thalmann, N. M., and Thalmann, D. 1989. Simulation of object and human skin deformations in a grasping task. In Comp. Graph. (Proc. SIGGRAPH), 21--30.

Digital Library

[15]

Hara, F., Akazawa, H., and Kobayashi, H. 2001. Realistic facial expressions by sma driven face robot. In Proc. of IEEE International Workshop on Robot and Human Interactive Communication, 504--511.

[16]

Hašan, M., Fuchs, M., Matusik, W., Pfister, H., and Rusinkiewicz, S. 2010. Physical reproduction of materials with specified subsurface scattering. ACM Trans. Graph. 29 (July), 61:1--61:10.

Digital Library

[17]

Irving, G., Teran, J., and Fedkiw, R. 2004. Invertible finite elements for robust simulation of large deformation. In 2004 ACM SIGGRAPH/Eurographics SCA, 131--140.

Digital Library

[18]

Kauer, M., Vuskovic, V., Dual, J., Szekely, G., and Bajka, M. 2002. Inverse finite element characterization of soft tissues. Medical Image Analysis 6, 3, 257--287.

[19]

Kharevych, L., Mullen, P., Owhadi, H., and Desbrun, M. 2009. Numerical coarsening of inhomogeneous elastic materials. ACM Trans. Graph. 28 (July), 51:1--51:8.

Digital Library

[20]

Koch, R. M., Gross, M. H., Carls, F. R., von Büren, D. F., Fankhauser, G., and Parish, Y. I. H. 1996. Simulating facial surgery using finite element models. In Proc. of Comp. graph. and int. tech., ACM, SIGGRAPH '96, 421--428.

Digital Library

[21]

Lee, S.-H., and Terzopoulos, D. 2006. Heads up!: biomechanical modeling and neuromuscular control of the neck. In ACM SIGGRAPH 2006 Papers, ACM, 1188--1198.

Digital Library

[22]

Lee, S.-H., Sifakis, E., and Terzopoulos, D. 2009. Comprehensive biomechanical modeling and simulation of the upper body. ACM Trans. Graph. 28 (September), 99:1--99:17.

Digital Library

[23]

Levin, D. 1998. The approximation power of moving least-squares. Math. Comput. 67 (October), 1517--1531.

Digital Library

[24]

Li, X.-Y., Shen, C.-H., Huang, S.-S., Ju, T., and Hu, S.-M. 2010. Popup: automatic paper architectures from 3d models. ACM Trans. Graph. 29 (July), 111:1--111:9.

Digital Library

[25]

Mori, Y., and Igarashi, T. 2007. Plushie: an interactive design system for plush toys. In ACM SIGGRAPH 2007 papers, ACM, SIGGRAPH '07.

Digital Library

[26]

Nesme, M., Kry, P. G., Jeřábková, L., and Faure, F. 2009. Preserving topology and elasticity for embedded deformable models. In ACM SIGGRAPH 2009 papers, ACM, 52:1--52:9.

Digital Library

[27]

Nishio, S., Ishiguro, H., and Hagita, N. 2007. Humanoid Robots: New Developments. I-Tech, ch. Geminoid: Teleoperated Android of an Existing Person.

[28]

Nocedal, J., and Wright, S. J. 2000. Numerical Optimization. Springer, August.

[29]

Oh, J.-H., Hanson, D., Kim, W.-S., Han, I. Y., Kim, J.-Y., and Park, I.-W. 2006. Design of android type humanoid robot albert hubo. In Proc. of IEEE/RSJ Int. Conference on Intelligent Robots and Systems, 1428--1433.

[30]

Pai, D. K., Doel, K. v. d., James, D. L., Lang, J., Lloyd, J. E., Richmond, J. L., and Yau, S. H. 2001. Scanning physical interaction behavior of 3d objects. In Proc. of Comp. graph. and int. tech., ACM, SIGGRAPH '01, 87--96.

Digital Library

[31]

Schenk, O., and Gärtner, K. 2006. On fast factorization pivoting methods for symmetric indefinite systems. In Elec. Trans. Numer., no. 23, 158--179.

[32]

Sifakis, E., Neverov, I., and Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. In ACM SIGGRAPH 2005, 417--425.

Digital Library

[33]

Sueda, S., Kaufman, A., and Pai, D. K. 2008. Musculotendon simulation for hand animation. ACM Trans. Graph. (Proc. SIGGRAPH) 27, 3.

Digital Library

[34]

Teran, J., Sifakis, E., Blemker, S. S., Ng-Thow-Hing, V., Lau, C., and Fedkiw, R. 2005. Creating and simulating skeletal muscle from the visible human data set. IEEE Trans. on Vis. and Comp. Graph. 11, 3, 317--328.

Digital Library

[35]

Terzopoulos, D., and Waters, K. 1993. Analysis and synthesis of facial image sequences using physical and anatomical models. IEEE Trans. Pattern Anal. Mach. Intell. 15 (June), 569--579.

Digital Library

[36]

Terzopoulos, D., Platt, J., Barr, A., and Fleischer, K. 1987. Elastically deformable models. In Proc. Comp. graph. and int. tech., ACM, SIGGRAPH '87, 205--214.

Digital Library

[37]

Thoutireddy, P., and Ortiz, M. 2004. A variational r-adaption and shape-optimization method for finite-deformation elasticity. Int. J. Numer. Meth. Engng. 61, 1--21.

[38]

Van Gelder, A. 1998. Approximate simulation of elastic membranes by triangulated spring meshes. J. Graph. Tools 3, 2, 21--42.

Digital Library

[39]

Wang, Y., Huang, X., Lee, C.-S., Zhang, S., Li, Z., Samaras, D., Metaxas, D., Elgammal, A., and Huang, P. 2004. High resolution acquisition, learning and transfer of dyanmic 3-d facial expressions. In Comp. Graph. Forum, 677--686.

[40]

Weyrich, T., Peers, P., Matusik, W., and Rusinkiewicz, S. 2009. Fabricating microgeometry for custom surface reflectance. ACM Trans. Graph. 28 (July), 32:1--32:6.

Digital Library

[41]

Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: high resolution capture for modeling and animation. ACM Transactions on Graphics 23, 3 (Aug.), 548--558.

Digital Library

Cited By

Su MZhang YChen HGuan YXiang C(2023)Modeling, Analysis, and Computational Design of Muscle-driven Soft RobotsSoft Robotics10.1089/soro.2022.012210:4(808-824)Online publication date: 1-Aug-2023
https://doi.org/10.1089/soro.2022.0122
Wu XXu JZhu ZBao HHuang QTompkin JXu W(2022)Scalable neural indoor scene renderingACM Transactions on Graphics10.1145/3528223.353015341:4(1-16)Online publication date: 22-Jul-2022
https://dl.acm.org/doi/10.1145/3528223.3530153
She QHu RXu JLiu MXu KHuang H(2022)Learning high-DOF reaching-and-grasping via dynamic representation of gripper-object interactionACM Transactions on Graphics10.1145/3528223.353009141:4(1-14)Online publication date: 22-Jul-2022
https://dl.acm.org/doi/10.1145/3528223.3530091
Show More Cited By

Index Terms

Physical face cloning
1. Mathematics of computing
2. Social and professional topics
  1. Professional topics
    1. History of computing

Index terms have been assigned to the content through auto-classification.

Recommendations

Facial motion cloning

We propose a method for automatically copying facial motion from one 3D face model to another, while preserving the compliance of the motion to the MPEG-4 Face and Body Animation (FBA) standard. Despite the enormous progress in the field of Facial ...
Modeling Expressive Wrinkles of Face For Animation
ICIG '07: Proceedings of the Fourth International Conference on Image and Graphics

Vivid facial expressions contribute greatly to the visual realism of 3D face models. However, it is often tedious and expensive to model facial details which are authentic and real-time. In this paper, a simple and functional approach for modeling ...
SparseSoftDECA — Efficient high-resolution physics-based facial animation from sparse landmarks
Abstract
Facial animation on computationally limited systems still heavily relies on linear blendshape models. Nonetheless, these models exhibit common issues like volume loss, self-collisions, and inaccuracies in soft tissue elasticity. Furthermore, ...
Highlights
- Computationally efficient physics-based facial animations.
- Localized material adaptions.
- Anatomically plausible personalized facial animations from tracked landmarks.
- Data-driven procedure for fitting the biomechanics anatomy ...
Graphical abstract

Display Omitted

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 31, Issue 4

July 2012

935 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2185520

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2012

Published in TOG Volume 31, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

57
Total Citations
View Citations
1,248
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)5

Reflects downloads up to 20 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Su MZhang YChen HGuan YXiang C(2023)Modeling, Analysis, and Computational Design of Muscle-driven Soft RobotsSoft Robotics10.1089/soro.2022.012210:4(808-824)Online publication date: 1-Aug-2023
https://doi.org/10.1089/soro.2022.0122
Wu XXu JZhu ZBao HHuang QTompkin JXu W(2022)Scalable neural indoor scene renderingACM Transactions on Graphics10.1145/3528223.353015341:4(1-16)Online publication date: 22-Jul-2022
https://dl.acm.org/doi/10.1145/3528223.3530153
She QHu RXu JLiu MXu KHuang H(2022)Learning high-DOF reaching-and-grasping via dynamic representation of gripper-object interactionACM Transactions on Graphics10.1145/3528223.353009141:4(1-14)Online publication date: 22-Jul-2022
https://dl.acm.org/doi/10.1145/3528223.3530091
Arnavaz KNielsen MKry PMacklin MErleben K(2022)Differentiable Depth for Real2Sim Calibration of Soft Body SimulationsComputer Graphics Forum10.1111/cgf.1472042:1(277-289)Online publication date: 23-Nov-2022
https://doi.org/10.1111/cgf.14720
Huber SPoranne RCoros S(2021)Designing actuation systems for animatronic figures via globally optimal discrete searchACM Transactions on Graphics10.1145/3450626.345986740:4(1-10)Online publication date: 19-Jul-2021
https://dl.acm.org/doi/10.1145/3450626.3459867
Ma PDu TZhang JWu KSpielberg AKatzschmann RMatusik W(2021)DiffAquaACM Transactions on Graphics10.1145/3450626.345983240:4(1-14)Online publication date: 19-Jul-2021
https://dl.acm.org/doi/10.1145/3450626.3459832
Vicini DJakob WKaplanyan A(2021)A non-exponential transmittance model for volumetric scene representationsACM Transactions on Graphics10.1145/3450626.345981540:4(1-16)Online publication date: 19-Jul-2021
https://dl.acm.org/doi/10.1145/3450626.3459815
Schissler CMückl GCalamia P(2021)Fast diffraction pathfinding for dynamic sound propagationACM Transactions on Graphics10.1145/3450626.345975140:4(1-13)Online publication date: 19-Jul-2021
https://dl.acm.org/doi/10.1145/3450626.3459751
Mandad MCampen M(2021)Guaranteed-quality higher-order triangular meshing of 2D domainsACM Transactions on Graphics10.1145/3450626.345967340:4(1-14)Online publication date: 19-Jul-2021
https://dl.acm.org/doi/10.1145/3450626.3459673
Feng XLiu JWang HYang YBao HBickel BXu W(2021)Computational Design of Skinned Quad-RobotsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2019.295721827:6(2881-2895)Online publication date: 1-Jun-2021
https://doi.org/10.1109/TVCG.2019.2957218
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents