research-article

Artist friendly facial animation retargeting

Authors:

Paul Hyunjin Kim,

Junyong NohAuthors Info & Claims

SA '11: Proceedings of the 2011 SIGGRAPH Asia Conference

Article No.: 162, Pages 1 - 10

https://doi.org/10.1145/2024156.2024196

Published: 12 December 2011 Publication History

Abstract

This paper presents a novel facial animation retargeting system that is carefully designed to support the animator's workflow. Observation and analysis of the animators' often preferred process of key-frame animation with blendshape models informed our research. Our retargeting system generates a similar set of blendshape weights to those that would have been produced by an animator. This is achieved by rearranging the group of blendshapes into several sequential retargeting groups and solving using a matching pursuit-like scheme inspired by a traditional key-framing approach. Meanwhile, animators typically spend a tremendous amount of time simplifying the dense weight graphs created by the retargeting. Our graph simplification technique effectively produces editable weight graphs while preserving the visual characteristics of the original retargeting. Finally, we automatically create GUI controllers to help artists perform key-framing and editing very efficiently. The set of proposed techniques greatly reduce the time and effort required by animators to achieve high quality retargeted facial animations.

Supplementary Material

AVI File (a162-seol.avi)

Download
15.61 MB

References

[1]

Alexander, O., Rogers, M., Lambeth, W., Chiang, M., and Debevec, P. 2009. Creating a photoreal digital actor: The Digital Emily Project. Conference for Visual Media Production., 176--187.

Digital Library

[2]

Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., and Gross, M. 2007. Multi-scale capture of facial geometry and motion. ACM Trans. Graph. 26 (July).

Digital Library

[3]

Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3d faces. In Proceedings of SIGGRAPH 99, ACM Press/Addison-Wesley Publishing Co., SIGGRAPH 99, 187--194.

Digital Library

[4]

Choe, B., Lee, H., and seok Ko, H. 2001. Performance-driven muscle-based facial animation. The Journal of Visualization and Computer Animation 12, 67--79.

[5]

Chuang, E., and Bregler, C. 2002. Performance driven facial animation using blendshape interpolation. Tech. rep., Department of Computer Science, Stanford University.

[6]

Chuang, E., and Bregler, C. 2005. Mood swings: expressive speech animation. ACM Trans. Graph. 24, 2, 331--347.

Digital Library

[7]

Deng, Z., Chiang, P.-Y., Fox, P., and Neumann, U. 2006. Animating blendshape faces by cross-mapping motion capture data. In Proceedings of I3D 2006, ACM, New York, NY, USA, 43--48.

Digital Library

[8]

Digital-Tutors. 2006. Facial Animation and Lip Sync in Maya. Digital-Tutors.

[9]

Douglas, D., and Peucker, T. 1973. Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartographica: The International Journal for Geographic Information and Geovisualization 10, 112--122.

[10]

Ekman, P., and Friesen, W. 1977. Manual for the facial action coding system. Consulting Psychologists Press, Palo Alto.

[11]

Havaldar, P., 2006. Performance driven facial animation. ACM SIGGRAPH 2006 Course #30 Notes, August.

Digital Library

[12]

Hoppe, H. 1996. Progressive meshes. In Proceedings of SIGGRAPH 96, ACM, SIGGRAPH 96, 99--108.

Digital Library

[13]

Huang, H., Chai, J., Tong, X., and Wu, H. 2011. Leveraging motion capture and 3d scanning for high-fidelity facial performance acquisition. ACM Trans. Graph. 30 (August), 74:1--74:10.

Digital Library

[14]

Joshi, P., Tien, W. C., Desbrun, M., and Pighin, F. 2003. Learning controls for blend shape based realistic facial animation. In Proceedings of the SCA 2003, Eurographics Association, 187--192.

Digital Library

[15]

Lewis, J., and Anjyo, K. 2010. Direct manipulation blend-shapes. Computer Graphics and Applications special issue: Digital Human Faces 30, 4, 42--50.

Digital Library

[16]

Li, H., Weise, T., and Pauly, M. 2010. Example-based facial rigging. ACM Trans. Graph. 29 (July), 32:1--32:6.

Digital Library

[17]

Luamanuvae, J., 2011. Personal communication (Weta Digital).

[18]

Ma, W., Jones, A., Chiang, J.-Y., Hawkins, T., Frederiksen, S., Peers, P., Vukovic, M., Ouhyoung, M., and Debevec, P. 2008. Facial performance synthesis using deformation-driven polynomial displacement maps. ACM Trans. Graph. 27 (December), 121:1--121:10.

Digital Library

[19]

Ma, X., Le, B. H., and Deng, Z. 2009. Style learning and transferring for facial animation editing. In Proceedings of SCA 2009, ACM, 123--132.

Digital Library

[20]

Mallat, S., and Zhang, Z. 1993. Matching pursuits with time-frequency dictionaries. Signal Processing, IEEE Transactions on 41, 12 (Dec.), 3397--3415.

Digital Library

[21]

Noh, J., and Neumann, U. 2001. Expression cloning. In Proceedings of SIGGRAPH 2001, ACM, SIGGRAPH 2001, 277--288.

Digital Library

[22]

Orvalho, V. C., Zacur, E., and Susin, A. 2008. Transferring the rig and animations from a character to different face models. Computer Graphics Forum 27, 8, 1997--2012.

[23]

Pyun, H., Kim, Y., Chae, W., Kang, H. W., and Shin, S. Y. 2003. An example-based approach for facial expression cloning. In Proceedings of SCA 2003, Eurographics Association, 167--176.

Digital Library

[24]

Seo, J., Irving, G., Lewis, J. P., and Noh, J. 2011. Compression and direct manipulation of complex blendshape models. ACM Trans. Graph. (Proceedings of SIGGRAPH ASIA 2011) 30, 6.

Digital Library

[25]

Seol, Y., Lewis, J., Seo, J., Anjyo, K., and Noh, J. 2011. Spacetime expression cloning for blendshapes. ACM Trans. Graph. (Conditionally Accepted).

Digital Library

[26]

Skocaj, D., and Leonardis, A. 2002. Weighted incremental subspace learning. In Proceedings of Workshop on Cognitive Vision.

[27]

Sumner, R. W., and Popović, J. 2004. Deformation transfer for triangle meshes. ACM Trans. Graph. 23 (August), 399--405.

Digital Library

[28]

Weise, T., Li, H., Van Gool, L., and Pauly, M. 2009. Face/off: live facial puppetry. In Proceedings of SCA 2009, ACM, 7--16.

Digital Library

[29]

Williams, L. 1990. Performance-driven facial animation. In SIGGRAPH '90: Proceedings of the 17th annual conference on Computer graphics and interactive techniques, ACM, New York, NY, USA, 235--242.

Digital Library

[30]

Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: high resolution capture for modeling and animation. ACM Trans. Graph. 23 (August), 548--558.

Digital Library

Cited By

Racković SJakovetić DSoares C(2024)Refined Inverse Rigging: A Balanced Approach to High-fidelity Blendshape AnimationSIGGRAPH Asia 2024 Conference Papers10.1145/3680528.3687670(1-9)Online publication date: 3-Dec-2024
https://dl.acm.org/doi/10.1145/3680528.3687670
Racković SSoares CJakovetić D(2023)Distributed Solution of the Blendshape Rig Inversion ProblemSIGGRAPH Asia 2023 Technical Communications10.1145/3610543.3626166(1-4)Online publication date: 28-Nov-2023
https://dl.acm.org/doi/10.1145/3610543.3626166
Racković SSoares CJakovetić DDesnica Z(2023)A majorization–minimization-based method for nonconvex inverse rig problems in facial animation: algorithm derivationOptimization Letters10.1007/s11590-023-02012-w18:2(545-559)Online publication date: 20-May-2023
https://doi.org/10.1007/s11590-023-02012-w
Show More Cited By

Index Terms

Artist friendly facial animation retargeting
1. Computing methodologies
  1. Computer graphics
    1. Animation

Recommendations

Creating an actor-specific facial rig from performance capture
DigiPro '16: Proceedings of the 2016 Symposium on Digital Production

Creating a high quality blendshape rig usually involves a large amount of effort from skilled artists. Although current 3D reconstruction technologies are able to capture accurate facial geometry of the actor, it is still very difficult to build a ...
Facial retargeting with automatic range of motion alignment

While facial capturing focuses on accurate reconstruction of an actor's performance, facial animation retargeting has the goal to transfer the animation to another character, such that the semantic meaning of the animation remains. Because of the ...
Spacetime expression cloning for blendshapes

The goal of a practical facial animation retargeting system is to reproduce the character of a source animation on a target face while providing room for additional creative control by the animator. This article presents a novel spacetime facial ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SA '11: Proceedings of the 2011 SIGGRAPH Asia Conference

December 2011

730 pages

ISBN:9781450308076

DOI:10.1145/2024156

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

In-Cooperation

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 December 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

KOCCA/MCST

Conference

SA '11

Sponsor:

SIGGRAPH

SA '11: SIGGRAPH Asia 2011

December 12 - 15, 2011

Hong Kong, China

Acceptance Rates

Overall Acceptance Rate 178 of 869 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
1,418
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Racković SJakovetić DSoares C(2024)Refined Inverse Rigging: A Balanced Approach to High-fidelity Blendshape AnimationSIGGRAPH Asia 2024 Conference Papers10.1145/3680528.3687670(1-9)Online publication date: 3-Dec-2024
https://dl.acm.org/doi/10.1145/3680528.3687670
Racković SSoares CJakovetić D(2023)Distributed Solution of the Blendshape Rig Inversion ProblemSIGGRAPH Asia 2023 Technical Communications10.1145/3610543.3626166(1-4)Online publication date: 28-Nov-2023
https://dl.acm.org/doi/10.1145/3610543.3626166
Racković SSoares CJakovetić DDesnica Z(2023)A majorization–minimization-based method for nonconvex inverse rig problems in facial animation: algorithm derivationOptimization Letters10.1007/s11590-023-02012-w18:2(545-559)Online publication date: 20-May-2023
https://doi.org/10.1007/s11590-023-02012-w
Menzel TBotsch MLatoschik M(2022)Automated Blendshape Personalization for Faithful Face Animations Using Commodity SmartphonesProceedings of the 28th ACM Symposium on Virtual Reality Software and Technology10.1145/3562939.3565622(1-9)Online publication date: 29-Nov-2022
https://dl.acm.org/doi/10.1145/3562939.3565622
Seol YCozens M(2019)Interactive editing of performance-based facial animationSIGGRAPH Asia 2019 Technical Briefs10.1145/3355088.3365147(61-64)Online publication date: 17-Nov-2019
https://dl.acm.org/doi/10.1145/3355088.3365147
Onizuka HThomas DUchiyama HTaniguchi R(2019)Landmark-Guided Deformation Transfer of Template Facial Expressions for Automatic Generation of Avatar Blendshapes2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)10.1109/ICCVW.2019.00265(2100-2108)Online publication date: Oct-2019
https://doi.org/10.1109/ICCVW.2019.00265
Gunanto SHariadi MYuniarno E(2016)Computer facial animation with synthesize marker on 3D faces surface2016 2nd International Conference of Industrial, Mechanical, Electrical, and Chemical Engineering (ICIMECE)10.1109/ICIMECE.2016.7910452(260-263)Online publication date: 2016
https://doi.org/10.1109/ICIMECE.2016.7910452
Jungjin Lee Younghui Kim Sangwoo Lee Bumki Kim Junyong Noh (2015)High-Quality Depth Estimation Using an Exemplar 3D Model for Stereo ConversionIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2015.239844021:7(835-847)Online publication date: 1-Jul-2015
https://dl.acm.org/doi/10.1109/TVCG.2015.2398440
Tseng J(2014)Surface Simplification of 3D Animation Models Using Robust Homogeneous Coordinate TransformationJournal of Applied Mathematics10.1155/2014/1892412014(1-14)Online publication date: 2014
https://doi.org/10.1155/2014/189241
Xu FChai JLiu YTong X(2014)Controllable high-fidelity facial performance transferACM Transactions on Graphics10.1145/2601097.260121033:4(1-11)Online publication date: 27-Jul-2014
https://dl.acm.org/doi/10.1145/2601097.2601210
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten