research-article

The impact of stylization on face recognition

Authors:

Nicolas Olivier,

Fabien Danieau,

Ferran Argelaguet,

Anatole Lecuyer,

Philippe Guillotel,

Franck MultonAuthors Info & Claims

SAP '20: ACM Symposium on Applied Perception 2020

Article No.: 15, Pages 1 - 9

https://doi.org/10.1145/3385955.3407930

Published: 12 September 2020 Publication History

Abstract

While digital humans are key aspects of the rapidly evolving areas of virtual reality, gaming, and online communications, many applications would benefit from using digital personalized (stylized) representations of users, as they were shown to highly increase immersion, presence and emotional response. In particular, depending on the target application, one may want to look like a dwarf or an elf in a heroic fantasy world, or like an alien on another planet, in accordance with the style of the narrative. While creating such virtual replicas requires stylization of the user’s features onto the virtual character, no formal study has however been conducted to assess the ability to recognize stylized characters. In this paper, we present a perceptual study investigating the effect of the degree of stylization on the ability to recognize an actor, and the subjective acceptability of stylizations. Results show that recognition rates decrease when the degree of stylization increases, while acceptability of the stylization increases. These results provide recommendations to achieve good compromises between stylization and recognition, and pave the way to new stylization methods providing a tradeoff between stylization and recognition of the actor.

References

[1]

Vicki Bruce, Mike A. Burton, and Neal Dench. 1994. What’s Distinctive about a Distinctive Face?The Quarterly Journal of Experimental Psychology Section A 47, 1 (Feb. 1994), 119–141.

[2]

Vicki Bruce and Andy Young. 1986. Understanding face recognition. British Journal of Psychology 77, 3 (1986), 305–327.

[3]

Alex J. Champandard. 2016. Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artworks. ArXiv (2016).

[4]

Patrick Chiroro and Tim Valentine. 1995. An Investigation of the Contact Hypothesis of the Own-race Bias in Face Recognition. The Quarterly Journal of Experimental Psychology Section A 48, 4 (Nov. 1995), 879–894.

[5]

Fabien Danieau, Ilja Gubins, Nicolas Olivier, Olivier Dumas, Bernard Denis, Thomas Lopez, Nicolas Mollet, Brian Frager, and Quentin Avril. 2019. Automatic Generation and Stylization of 3D Facial Rigs. IEEE VR (March 2019), 784–792.

[6]

Kenneth A Deffenbacher, Thomas Vetter, John Johanson, and Alice J O’Toole. 1998. Facial Aging, Attractiveness, and Distinctiveness. Perception 27, 10 (Oct. 1998), 1233–1243.

[7]

Reuben Fleming, Betty J. Mohler, Javier Romero, Michael J. Black, and Martin Breidt. 2016. Appealing Female Avatars from 3D Body Scans: Perceptual Effects of Stylization:. In Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. SCITEPRESS - Science and and Technology Publications, Rome, Italy, 333–343.

Digital Library

[8]

H. D. Ellis G. M. Davies and J. W. Shepherd. 1981. Studies of cue saliency. Perceiving and remembering faces, vol 96, University of Illinois Press (1981).

[9]

Sir Francis Galton. 1879. Composite portraits, made by combining those of many different persons into a single, resultant figure. Journal of the Anthropological Institute(1879).

[10]

Epic Games. 2018. Paragon Assets. https://www.unrealengine.com/en-US/paragon

[11]

Lin Gao, Jie Yang, Yi-Ling Qiao, Yu-Kun Lai, Paul L. Rosin, Weiwei Xu, and Shihong Xia. 2018. Automatic unpaired shape deformation transfer. ACM Transactions on Graphics 37, 6 (Dec. 2018), 1–15.

Digital Library

[12]

Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. 2015. A Neural Algorithm of Artistic Style. arXiv:1508.06576 [cs, q-bio] (Sept. 2015).

[13]

Harold Hill and Vicki Bruce. 1996. The effects of lighting on the perception of facial surfaces.Journal of Experimental Psychology: Human Perception and Performance 22, 4(1996), 986–1004.

[14]

Ludovic Hoyet, Kenneth Ryall, Katja Zibrek, Hwangpil Park, Jehee Lee, Jessica Hodgins, and Carol O’Sullivan. 2013. Evaluating the distinctiveness and attractiveness of human motions on realistic virtual bodies. ACM Transactions on Graphics 32, 6 (Nov. 2013), 1–11.

Digital Library

[15]

Xun Huang, Ming-Yu Liu, Serge Belongie, and Jan Kautz. 2018. Multimodal Unsupervised Image-to-Image Translation. arXiv:1804.04732 [cs, stat] (April 2018).

[16]

J. M. Pickett I. Pollack and W. H. Sumby. 1954. On the Identification of Speakers by Voice. The Journal of the Acoustical Society of America 26, 403 (1954).

[17]

G. Johansson. 1973. Visual perception of biological motion and a model for its analysis. Perception & Psychophysics(1973).

[18]

Alan Johnston, Harold Hill, and Nicole Carman. 1992. Recognising Faces: Effects of Lighting Direction, Inversion, and Brightness Reversal. Perception 21, 3 (June 1992), 365–375.

[19]

Parneet Kaur, Hang Zhang, and Kristin J. Dana. 2017. Photo-realistic Facial Texture Transfer. arXiv:1706.04306 [cs] (June 2017).

[20]

Junho Kim, Minjae Kim, Hyeonwoo Kang, and Kwanghee Lee. 2019. U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation. arXiv:1907.10830 [cs, eess] (July 2019).

[21]

Davis E. King. 2009. Dlib-ml: A Machine Learning Toolkit. Journal of Machine Learning Research 10 (2009), 1755–1758.

Digital Library

[22]

Helmut Leder and Vicki Bruce. 1998. Local and Relational Aspects of Face Distinctiveness. The Quarterly Journal of Experimental Psychology Section A 51, 3 (Aug. 1998), 449–473.

[23]

Daniel T. Levin. 2000. Race as a visual feature: Using visual search and perceptual discrimination tasks to understand face categories and the cross-race recognition deficit.Journal of Experimental Psychology: General 129, 4 (2000), 559–574.

[24]

Chuan Li and Michael Wand. 2016. Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis. arXiv:1601.04589 [cs] (Jan. 2016).

[25]

Ming-Yu Liu, Thomas Breuel, and Jan Kautz. 2017. Unsupervised image-to-image translation networks. In Proceedings of the 31st International Conference on Neural Information Processing Systems(NIPS’17). Curran Associates Inc., Long Beach, California, USA, 700–708.

Digital Library

[26]

Ming-Yu Liu, Xun Huang, Arun Mallya, Tero Karras, Timo Aila, Jaakko Lehtinen, and Jan Kautz. 2019. Few-Shot Unsupervised Image-to-Image Translation. arXiv:1905.01723 [cs, stat] (May 2019).

[27]

Zhaoliang Lun, Evangelos Kalogerakis, Rui Wang, and Alla Sheffer. 2016. Functionality preserving shape style transfer. ACM Transactions on Graphics 35, 6 (Nov. 2016), 1–14.

Digital Library

[28]

Chongyang Ma, Haibin Huang, Alla Sheffer, Evangelos Kalogerakis, and Rui Wang. 2014. Analogy-driven 3D style transfer: Analogy-driven 3D style transfer. Computer Graphics Forum 33, 2 (May 2014), 175–184.

Digital Library

[29]

Rachel McDonnell, Martin Breidt, and Heinrich H. Bülthoff. 2012. Render me real?: investigating the effect of render style on the perception of animated virtual humans. ACM Transactions on Graphics 31, 4 (July 2012), 1–11.

Digital Library

[30]

Christian A. Meissner and John C. Brigham. 2001. Thirty years of investigating the own-race bias in memory for faces: A meta-analytic review.Psychology, Public Policy, and Law 7, 1 (2001), 3–35.

[31]

Fiona N. Newell Melissa Peskin. 2002. Familiarity breeds attraction: Effects of exposure on the attractiveness of typical and distinctive faces. Perception (2002).

[32]

Wei-Jen Ng and R. C.L. Lindsay. 1994. Cross-Race Facial Recognition: Failure of the Contact Hypothesis. Journal of Cross-Cultural Psychology 25, 2 (June 1994), 217–232.

[33]

Ahmed Selim, Mohamed Elgharib, and Linda Doyle. 2016. Painting style transfer for head portraits using convolutional neural networks. ACM Transactions on Graphics 35, 4 (July 2016), 1–18.

Digital Library

[34]

James W. Tanaka and Martha J. Farah. 1993. Parts and Wholes in Face Recognition. The Quarterly Journal of Experimental Psychology Section A 46, 2 (May 1993), 225–245.

[35]

The MakeHuman team. 2000–2019. MakeHuman. www.makehumancommunity.org

[36]

Bruce V.1988. Recognizing faces. Lawrence Erlbaum Associates.

[37]

Tim Valentine, Stephen Darling, and Mary Donnelly. 2004. Why are average faces attractive? The effect of view and averageness on the attractiveness of female faces. Psychonomic Bulletin & Review 11, 3 (June 2004), 482–487.

[38]

Thomas Vetter Heather A. Wild Volker Blanz, Alice J. O’Toole. 2000. On The Other Side of the Mean: The Perception of Dissimilarity in Human Faces. Perception (2000).

[39]

Kangxue Yin, Zhiqin Chen, Hui Huang, Daniel Cohen-Or, and Hao Zhang. 2019. LOGAN: Unpaired Shape Transform in Latent Overcomplete Space. arXiv:1903.10170 [cs] (March 2019).

[40]

Kangxue Yin, Hui Huang, Daniel Cohen-Or, and Hao Zhang. 2018. P2P-NET: bidirectional point displacement net for shape transform. ACM Transactions on Graphics (TOG) 37, 4 (July 2018), 152:1–152:13.

Digital Library

[41]

Robert K. Yin. 1969. Looking at upside-down faces.Journal of Experimental Psychology 81, 1 (1969), 141–145.

[42]

Eduard Zell, Carlos Aliaga, Adrian Jarabo, Katja Zibrek, Diego Gutierrez, Rachel McDonnell, and Mario Botsch. 2015. To stylize or not to stylize?: the effect of shape and material stylization on the perception of computer-generated faces. ACM Transactions on Graphics 34, 6 (Oct. 2015), 1–12.

Digital Library

[43]

W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld. 2003. Face recognition: A literature survey. Comput. Surveys 35, 4 (Dec. 2003), 399–458.

Digital Library

Cited By

Olivier NBaert KDanieau FMulton FAvril Q(2023)FaceTuneGANComputers and Graphics10.1016/j.cag.2022.12.004110:C(69-85)Online publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1016/j.cag.2022.12.004
Olivier NKerbiriou GArguelaguet FAvril QDanieau FGuillotel PHoyet LMulton F(2022)Study on Automatic 3D Facial Caricaturization: From Rules to Deep LearningFrontiers in Virtual Reality10.3389/frvir.2021.7851042Online publication date: 19-Jan-2022
https://doi.org/10.3389/frvir.2021.785104
Basset JOuannas BHoyet LMulton FWuhrer S(2022)Impact of Self-Contacts on Perceived Pose EquivalencesProceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games10.1145/3561975.3562946(1-10)Online publication date: 3-Nov-2022
https://dl.acm.org/doi/10.1145/3561975.3562946

Index Terms

The impact of stylization on face recognition

Index terms have been assigned to the content through auto-classification.

Recommendations

Data-driven face cartoon stylization
SA '14: SIGGRAPH Asia 2014 Technical Briefs

This paper presents a data-driven framework for generating cartoon-like facial representations from a given portrait image. We solve our problem by an optimization that simultaneously considers a desired artistic style, image-cartoon relationships of ...
Heuristics for continuity editing of cinematic computer graphics scenes
Sandbox '09: Proceedings of the 2009 ACM SIGGRAPH Symposium on Video Games

We present a set of heuristics for editing footage of 3D computer graphics cinematic sequences into a coherent movie clip which obeys the conventions of continuity editing. Our approach mimics the decision processes of an editor assembling a clip out of ...
Toonsynth: example-based synthesis of hand-colored cartoon animations

We present a new example-based approach for synthesizing hand-colored cartoon animations. Our method produces results that preserve the specific visual appearance and stylized motion of manually authored animations without requiring artists to draw ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SAP '20: ACM Symposium on Applied Perception 2020

September 2020

137 pages

ISBN:9781450376181

DOI:10.1145/3385955

Editors:
Cindy Grimm
Oregon State University
,
Mar Gonzalez Franco
Microsoft Research
,
Elham Ebrahimi
University of North Carolina, Wilmington
,
Mar Gonzalez Franco
Microsoft Research

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 September 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SAP '20

Sponsor:

SIGGRAPH

SAP '20: ACM Symposium on Applied Perception 2020

September 12 - 13, 2020

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 43 of 94 submissions, 46%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
148
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)4

Reflects downloads up to 09 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Olivier NBaert KDanieau FMulton FAvril Q(2023)FaceTuneGANComputers and Graphics10.1016/j.cag.2022.12.004110:C(69-85)Online publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1016/j.cag.2022.12.004
Olivier NKerbiriou GArguelaguet FAvril QDanieau FGuillotel PHoyet LMulton F(2022)Study on Automatic 3D Facial Caricaturization: From Rules to Deep LearningFrontiers in Virtual Reality10.3389/frvir.2021.7851042Online publication date: 19-Jan-2022
https://doi.org/10.3389/frvir.2021.785104
Basset JOuannas BHoyet LMulton FWuhrer S(2022)Impact of Self-Contacts on Perceived Pose EquivalencesProceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games10.1145/3561975.3562946(1-10)Online publication date: 3-Nov-2022
https://dl.acm.org/doi/10.1145/3561975.3562946

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents