research-article

Creating an actor-specific facial rig from performance capture

Authors:

J. P. LewisAuthors Info & Claims

DigiPro '16: Proceedings of the 2016 Symposium on Digital Production

Pages 13 - 17

https://doi.org/10.1145/2947688.2947693

Published: 23 July 2016 Publication History

Abstract

Creating a high quality blendshape rig usually involves a large amount of effort from skilled artists. Although current 3D reconstruction technologies are able to capture accurate facial geometry of the actor, it is still very difficult to build a production-ready blendshape rig from unorganized scans. Removing rigid head motion and separating mixed expressions from the captures are two of the major challenges in this process. We present a technique that creates a facial blendshape rig based on performance capture and a generic face rig. The customized rig accurately captures actor-specific face details while producing a semantically meaningful FACS basis. The resulting rig faithfully serves both artist friendly keyframe animation and high quality facial motion retargeting in production.

References

[1]

Beeler, T., and Bradley, D. 2014. Rigid stabilization of facial expressions. ACM Trans. Graph. 33, 4, 44:1--44:9.

Digital Library

[2]

Beeler, T., Bickel, B., Beardsley, P., Sumner, B., and Gross, M. 2010. High-quality single-shot capture of facial geometry. ACM Trans. Graph. 29, 3, 40:1--40:9.

Digital Library

[3]

Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3d faces. SIGGRAPH 1999, 187--194.

Digital Library

[4]

Bouaziz, S., Wang, Y., And Pauly, M. 2013. Online modeling for realtime facial animation. ACM Trans. Graph. 32, 4, 40:1--40:10.

Digital Library

[5]

Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. 2004. High accuracy optical flow estimation based on a theory for warping. In ECCV, vol. 3024 of LNCS, 25--36.

[6]

Cao, C., Hou, Q., And Zhou, K. 2014. Displaced dynamic expression regression for real-time facial tracking and animation. ACM Trans. Graph. 33, 4, 43:1--43:10.

Digital Library

[7]

Chuang, E., and Bregler, C. 2005. Mood swings: Expressive speech animation. ACM Trans. Graph. 24, 2, 331--347.

Digital Library

[8]

Ekman, P., And Friesen, W. 1978. Facial Action Coding System: Manual. No. v. 1-2. Consulting Psychologists Press.

[9]

Garrido, P., Valgaerts, L., Wu, C., and Theobalt, C. 2013. Reconstructing detailed dynamic face geometry from monocular video. In ACM Trans. Graph., vol. 32, 158:1--158:10.

Digital Library

[10]

Golub, G. H., and Van Loan, C. F. 1996. Matrix Computations (3rd Ed.). Johns Hopkins University Press.

Digital Library

[11]

Havaldar, P. 2006. Performance driven facial animation. In ACM SIGGRAPH 2006 Courses.

[12]

Hyneman, W., Itokazu, H., Williams, L., and Zhao, X. 2005. Human face project. In ACM SIGGRAPH 2005 Courses.

Digital Library

[13]

Ichim, A. E., Bouaziz, S., And Pauly, M. 2015. Dynamic 3d avatar creation from hand-held video input. ACM Trans. Graph. 34, 4, 45:1--45:14.

Digital Library

[14]

Li, H., Weise, T., and Pauly, M. 2010. Example-based facial rigging. ACM Trans. Graph. 29, 3.

Digital Library

[15]

Li, H., Yu, J., Ye, Y., and Bregler, C. 2013. Realtime facial animation with on-the-fly correctives. ACM Trans. Graph. 32, 4.

Digital Library

[16]

Sagar, M. 2006. Facial performance capture and expressive translation for king kong. In ACM SIGGRAPH 2006 Sketches.

Digital Library

[17]

Seol, Y., Lewis, J., Seo, J., Choi, B., Anjyo, K., and Noh, J. 2012. Spacetime expression cloning for blendshapes. ACM Trans. Graph. 31, 2, 14:1--14:12.

Digital Library

[18]

Sumner, R. W., and Popović, J. 2004. Deformation transfer for triangle meshes. ACM Trans. Graph. 23, 3, 399--405.

Digital Library

[19]

Sumner, R. W. 2005. Mesh Modification Using Deformation Gradients. Ph.D. Dissertation, MIT.

Digital Library

[20]

Weise, T., Li, H., Van Gool, L., and Pauly, M. 2009. Face/off: Live facial puppetry. SCA 2009, 7--16.

Digital Library

Cited By

Choi YLee ICha SKim SJung SNoh J(2024)Deep‐Learning‐Based Facial Retargeting Using Local PatchesComputer Graphics Forum10.1111/cgf.15263Online publication date: 25-Oct-2024
https://doi.org/10.1111/cgf.15263
Li YWeng DChen J(2023)Non-Rigid Point Cloud Matching Based on Invariant Structure for Face DeformationElectronics10.3390/electronics1204082812:4(828)Online publication date: 6-Feb-2023
https://doi.org/10.3390/electronics12040828
Zhao ZWeng DGuo HHou JZhou JEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Facial Auto Rigging from 4D Expressions via Skinning DecompositionProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612934(6101-6109)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612934
Show More Cited By

Index Terms

Creating an actor-specific facial rig from performance capture
1. Computing methodologies
  1. Computer graphics
    1. Shape modeling

Recommendations

Artist friendly facial animation retargeting

This paper presents a novel facial animation retargeting system that is carefully designed to support the animator's workflow. Observation and analysis of the animators' often preferred process of key-frame animation with blendshape models informed our ...
Artist friendly facial animation retargeting
SA '11: Proceedings of the 2011 SIGGRAPH Asia Conference

This paper presents a novel facial animation retargeting system that is carefully designed to support the animator's workflow. Observation and analysis of the animators' often preferred process of key-frame animation with blendshape models informed our ...
ILM facial performance capture
SIGGRAPH '16: ACM SIGGRAPH 2016 Talks

Industrial Light & Magic utilized facial capture technology on a massive scale to bring more than two dozen computer-generated characters to life in the new Warcraft, Teenage Mutant Ninja Turtles, and Star Wars films. Here we present our flexible ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

DigiPro '16: Proceedings of the 2016 Symposium on Digital Production

July 2016

70 pages

ISBN:9781450344296

DOI:10.1145/2947688

Conference Chairs:
Andrew Pearce
DreamWorks Animation
,
Eric Enderton
NVIDIA
,
Program Chairs:
Chris Horvath
Oculus
,
Jonathan "Jono" Gibbs

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 July 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

DigiPro '16

Sponsor:

SIGGRAPH

DigiPro '16: The Digital Production Symposium

July 23, 2016

California, Anaheim

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
791
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Choi YLee ICha SKim SJung SNoh J(2024)Deep‐Learning‐Based Facial Retargeting Using Local PatchesComputer Graphics Forum10.1111/cgf.15263Online publication date: 25-Oct-2024
https://doi.org/10.1111/cgf.15263
Li YWeng DChen J(2023)Non-Rigid Point Cloud Matching Based on Invariant Structure for Face DeformationElectronics10.3390/electronics1204082812:4(828)Online publication date: 6-Feb-2023
https://doi.org/10.3390/electronics12040828
Zhao ZWeng DGuo HHou JZhou JEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Facial Auto Rigging from 4D Expressions via Skinning DecompositionProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612934(6101-6109)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612934
Vahdati DDuc Nguyen TStamm M(2023)Defending Low-Bandwidth Talking Head Videoconferencing Systems From Real-Time Puppeteering Attacks2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW59228.2023.00105(983-992)Online publication date: Jun-2023
https://doi.org/10.1109/CVPRW59228.2023.00105
Menzel TBotsch MLatoschik M(2022)Automated Blendshape Personalization for Faithful Face Animations Using Commodity SmartphonesProceedings of the 28th ACM Symposium on Virtual Reality Software and Technology10.1145/3562939.3565622(1-9)Online publication date: 29-Nov-2022
https://dl.acm.org/doi/10.1145/3562939.3565622
Ma WGhifary MLewis JChoi BEom H(2022)FDLS: A Deep Learning Approach to Production Quality, Controllable, and Retargetable Facial Performances.Proceedings of the 2022 Digital Production Symposium10.1145/3543664.3543672(1-9)Online publication date: 7-Aug-2022
https://dl.acm.org/doi/10.1145/3543664.3543672
Haapaoja RKlintberg J(2022)Advances for Digital Humans in VFX Production at Goodbye Kansas StudiosACM SIGGRAPH 2022 Talks10.1145/3532836.3536270(1-2)Online publication date: 27-Jul-2022
https://dl.acm.org/doi/10.1145/3532836.3536270
Zell EMcDonnell R(2022)Compact Facial Landmark Layouts for Performance CaptureComputer Graphics Forum10.1111/cgf.1446341:2(121-133)Online publication date: 24-May-2022
https://doi.org/10.1111/cgf.14463
Zhang JChen KZheng J(2022)Facial Expression Retargeting From Human to Avatar Made EasyIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.301387628:2(1274-1287)Online publication date: 1-Feb-2022
https://doi.org/10.1109/TVCG.2020.3013876
Han JKim JSuh JKim H(2022)Customizing blendshapes to capture facial detailsThe Journal of Supercomputing10.1007/s11227-022-04885-779:6(6347-6372)Online publication date: 3-Nov-2022
https://doi.org/10.1007/s11227-022-04885-7
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten