research-article

Lightweight wrinkle synthesis for 3D facial modeling and animation

Authors:

Reinhard KleinAuthors Info & Claims

Computer-Aided Design, Volume 58, Issue C

Pages 117 - 122

https://doi.org/10.1016/j.cad.2014.08.016

Published: 01 January 2015 Publication History

Abstract

We present a lightweight non-parametric method to generate wrinkles for 3D facial modeling and animation. The key lightweight feature of the method is that it can generate plausible wrinkles using a single low-cost Kinect camera and one high quality 3D face model with details as the example. Our method works in two stages: (1) offline personalized wrinkled blendshape construction. User-specific expressions are recorded using the RGB-Depth camera, and the wrinkles are generated through example-based synthesis of geometric details. (2) Online 3D facial performance capturing. These reconstructed expressions are used as blendshapes to capture facial animations in real-time. Experiments on a variety of facial performance videos show that our method can produce plausible results, approximating the wrinkles in an accurate way. Furthermore, our technique is low-cost and convenient for common users. We present a lightweight non-parametric approach to generate wrinkles for 3D facial modeling and animation.Our method represents a convenient approach for generating plausible facial wrinkles with low-cost.Our method enables the reconstruction of captured expressions with wrinkles in real-time.

References

[1]

B. Bickel, M. Botsch, R. Angst, W. Matusik, M. Otaduy, H. Pfister, Multi-scale capture of facial geometry and motion, ACM Trans Graph (SIGGRAPH), 26 (2007).

Digital Library

[2]

P. Garrido, L. Valgaerts, C. Wu, C. Theobalt, Reconstructing detailed dynamic face geometry from monocular video, ACM Trans Graph (SIGGRAPH ASIA), 32 (2013).

Digital Library

[3]

T. Beeler, B. Bickel, P. Beardsley, B. Sumner, M. Gross, High-quality single-shot capture of facial geometry, ACM Trans Graph (SIGGRAPH), 29 (2010).

Digital Library

[4]

T. Beeler, F. Hahn, D. Bradley, B. Bickel, P. Beardsley, C. Gotsman, High-quality passive facial performance capture using anchor frames, ACM Trans Graph (SIGGRAPH), 30 (2011).

Digital Library

[5]

D. Bradley, W. Heidrich, T. Popa, A. Sheffer, High resolution passive facial performance capture, ACM Trans Graph (SIGGRAPH), 29 (2010).

Digital Library

[6]

XYZRGB. XYZ RGB system, 2012. http://www.xyzrgb.com.

[7]

W.-C. Ma, A. Jones, J.-Y. Chiang, T. Hawkins, S. Frederiksen, P. Peers, Facial performance synthesis using deformation-driven polynomial displacement maps, ACM Trans Graph (SIGGRAPH ASIA), 27 (2008).

Digital Library

[8]

L. Zhang, N. Snavely, B. Curless, S.M. Seitz, Spacetime faces: high resolution capture for modeling and animation, ACM Trans Graph (SIGGRAPH), 23 (2004) 548-558.

Digital Library

[9]

L. Valgaerts, C. Wu, A. Bruhn, H.-P. Seidel, C. Theobalt, Lightweight binocular facial performance capture under uncontrolled lighting, ACM Trans Graph (SIGGRAPH ASIA), 31 (2012).

Digital Library

[10]

Vicon. Vicon homepage, 2012. http://www.vicon.com/.

[11]

C. Cao, Y. Weng, S. Lin, K. Zhou, 3D shape regression for real-time facial animation, ACM Trans Graph (SIGGRAPH), 32 (2013).

Digital Library

[12]

H. Li, J. Yu, Y. Ye, C. Bregler, Realtime facial animation with on-the-fly correctives, ACM Trans Graph, 32 (2013).

Digital Library

[13]

Weise T, Li H, Van Gool L, Pauly M. Face/off: live facial puppetry.

[14]

D. Vlasic, M. Brand, H. Pfister, J. Popović, Face transfer with multilinear models, ACM Trans Graph, 24 (2005) 426-433.

Digital Library

[15]

H. Huang, J. Chai, X. Tong, H.-T. Wu, Leveraging motion capture and 3D scanning for high-fidelity facial performance acquisition, ACM Trans Graph (SIGGRAPH), 30 (2011).

Digital Library

[16]

B. Bickel, M. Lang, M. Botsch, M.A. Otaduy, M. Gross, Pose-space animation and transfer of facial details, in: SCA'08, 2008, pp. 57-66.

Digital Library

[17]

A.A. Efros, T.K. Leung, Texture synthesis by non-parametric sampling, in: ICCV, vol. 2, 1999, pp. 1033-1038.

Digital Library

[18]

Wei L-Y, Levoy M. Fast texture synthesis using tree-structured vector quantization. In: Proceedings of the 27th annual conference on computer graphics and interactive techniques.

Digital Library

[19]

A.A. Efros, W.T. Freeman, Image quilting for texture synthesis and transfer, in: SIGGRAPH'01, ACM Press, 2001, pp. 341-346.

Digital Library

[20]

L. Liang, C. Liu, Y.-Q. Xu, B. Guo, H.-Y. Shum, Real-time texture synthesis by patch-based sampling, ACM Trans Graph, 20 (2001) 127-150.

Digital Library

[21]

V. Kwatra, I. Essa, A. Bobick, N. Kwatra, Texture optimization for example-based synthesis, ACM Trans Graph, 24 (2005) 795-802.

Digital Library

[22]

Q. Huang, B. Adams, M. Wicke, J.G. Leonidas, Non-rigid registration under isometric deformations, in: Proceedings of the symposium on geometry processing, Eurographics Association, 2008, pp. 1449-1457.

Digital Library

[23]

S.J. Saragih, J. Cohn, Real-time avatar animation from a single image, in: IEEE International conference on automatic face and gesture recognition, IEEE Computer Society, Santa Barbara (CA, USA), 2011, pp. 117-124.

[24]

P. Ekman, W. Friesen, Facial action coding system: a technique for the measurement of facial movement, Consulting Psychologists Press, 1978.

[25]

T. Weise, S. Bouaziz, H. Li, M. Pauly, Realtime performance-based facial animation, ACM Trans Graph (SIGGRAPH), 30 (2011).

Digital Library

[26]

Blanz V, Vetter T. A morphable model for the synthesis of 3D faces. In: Proceedings of SIGGRAPH'00.

Digital Library

[27]

S. Rusinkiewicz, M. Levoy, Efficient variants of the ICP algorithm, in: International conference on 3D digital imaging and modeling, IEEE Computer Society, 2001, pp. 145-152.

Cited By

Ling JWang ZLu MWang QQian CXu F(2022)Structure-Aware Editable Morphable Model for 3D Facial Detail Animation and ManipulationComputer Vision – ECCV 202210.1007/978-3-031-20062-5_15(249-267)Online publication date: 23-Oct-2022
https://dl.acm.org/doi/10.1007/978-3-031-20062-5_15
Li TBolkart TBlack MLi HRomero J(2017)Learning a model of facial shape and expression from 4D scansACM Transactions on Graphics10.1145/3130800.313081336:6(1-17)Online publication date: 20-Nov-2017
https://dl.acm.org/doi/10.1145/3130800.3130813
Casas DFeng AAlexander OFyffe GDebevec PIchikari RLi HOlszewski KSuma EShapiro AThalmann NGavrilova MKomura TCordier FThalmann D(2016)Rapid Photorealistic Blendshape Modeling from RGB-D SensorsProceedings of the 29th International Conference on Computer Animation and Social Agents10.1145/2915926.2915936(121-129)Online publication date: 23-May-2016
https://dl.acm.org/doi/10.1145/2915926.2915936
Show More Cited By

Lightweight wrinkle synthesis for 3D facial modeling and animation
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
  2. Computer graphics

Recommendations

A review of motion retargeting techniques for 3D character facial animation
Abstract
3D face animation has been a critical component of character animation in a wide range of media since the early 90’s. The conventional process for animating a 3D face is usually keyframe-based, which is labor-intensive. Therefore, the film and ...
Graphical abstract

Display Omitted
Highlights
- A review of 3D facial motion retargeting.
- Provides a comprehensive taxonomy of retargeting methods and other relevant topics.
- Topics include 3D face parameterization, motion analysis, and retargeting methods.
- Also relates to ...
Expressive Facial Animation Synthesis by Learning Speech Coarticulation and Expression Spaces

Synthesizing expressive facial animation is a very challenging topic within the graphics community. In this paper, we present an expressive facial animation synthesis system enabled by automated learning from facial motion capture data. Accurate 3D ...
Versatile Face Animator: Driving Arbitrary 3D Facial Avatar in RGBD Space
MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Creating realistic 3D facial animation is crucial for various applications in the movie production and gaming industry, especially with the burgeoning demand in the metaverse. However, prevalent methods such as blendshape-based approaches and facial ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computer-Aided Design

Computer-Aided Design Volume 58, Issue C

January 2015

258 pages

ISSN:0010-4485

Issue’s Table of Contents

Copyright © Elsevier Ltd.

Publisher

Butterworth-Heinemann

United States

Publication History

Published: 01 January 2015

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ling JWang ZLu MWang QQian CXu F(2022)Structure-Aware Editable Morphable Model for 3D Facial Detail Animation and ManipulationComputer Vision – ECCV 202210.1007/978-3-031-20062-5_15(249-267)Online publication date: 23-Oct-2022
https://dl.acm.org/doi/10.1007/978-3-031-20062-5_15
Li TBolkart TBlack MLi HRomero J(2017)Learning a model of facial shape and expression from 4D scansACM Transactions on Graphics10.1145/3130800.313081336:6(1-17)Online publication date: 20-Nov-2017
https://dl.acm.org/doi/10.1145/3130800.3130813
Casas DFeng AAlexander OFyffe GDebevec PIchikari RLi HOlszewski KSuma EShapiro AThalmann NGavrilova MKomura TCordier FThalmann D(2016)Rapid Photorealistic Blendshape Modeling from RGB-D SensorsProceedings of the 29th International Conference on Computer Animation and Social Agents10.1145/2915926.2915936(121-129)Online publication date: 23-May-2016
https://dl.acm.org/doi/10.1145/2915926.2915936
Bérard PBradley DGross MBeeler T(2016)Lightweight eye capture using a parametric modelACM Transactions on Graphics10.1145/2897824.292596235:4(1-12)Online publication date: 11-Jul-2016
https://dl.acm.org/doi/10.1145/2897824.2925962
Garrido PZollhöfer MCasas DValgaerts LVaranasi KPérez PTheobalt C(2016)Reconstruction of Personalized 3D Face Rigs from Monocular VideoACM Transactions on Graphics10.1145/289049335:3(1-15)Online publication date: 18-May-2016
https://dl.acm.org/doi/10.1145/2890493
Ichim ABouaziz SPauly M(2015)Dynamic 3D avatar creation from hand-held video inputACM Transactions on Graphics10.1145/276697434:4(1-14)Online publication date: 27-Jul-2015
https://dl.acm.org/doi/10.1145/2766974
Cao CBradley DZhou KBeeler T(2015)Real-time high-fidelity facial performance captureACM Transactions on Graphics10.1145/276694334:4(1-9)Online publication date: 27-Jul-2015
https://dl.acm.org/doi/10.1145/2766943
Bermano ABeeler TKozlov YBradley DBickel BGross M(2015)Detailed spatio-temporal reconstruction of eyelidsACM Transactions on Graphics10.1145/276692434:4(1-11)Online publication date: 27-Jul-2015
https://dl.acm.org/doi/10.1145/2766924

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents