research-article

"Wow! you are so beautiful today!"

Authors:

Shuicheng YanAuthors Info & Claims

MM '13: Proceedings of the 21st ACM international conference on Multimedia

Pages 3 - 12

https://doi.org/10.1145/2502081.2502126

Published: 21 October 2013 Publication History

Abstract

Beauty e-Experts, a fully automatic system for hairstyle and facial makeup recommendation and synthesis, is developed in this work. Given a user-provided frontal face image with short/bound hair and no/light makeup, the Beauty e-Experts system can not only recommend the most suitable hairdo and makeup, but also show the synthetic effects. To obtain enough knowledge for beauty modeling, we build the Beauty e-Experts Database, which contains 1,505 attractive female photos with a variety of beauty attributes and beauty-related attributes annotated. Based on this Beauty e-Experts Dataset, two problems are considered for the Beauty e-Experts system: what to recommend and how to wear, which describe a similar process of selecting hairstyle and cosmetics in our daily life. For the what-to-recommend problem, we propose a multiple tree-structured super-graphs model to explore the complex relationships among the high-level beauty attributes, mid-level beauty-related attributes and low-level image features, and then based on this model, the most compatible beauty attributes for a given facial image can be efficiently inferred. For the how-to-wear problem, an effective and efficient facial image synthesis module is designed to seamlessly synthesize the recommended hairstyle and makeup into the user facial image. Extensive experimental evaluations and analysis on testing images of various conditions well demonstrate the effectiveness of the proposed system.

References

[1]

T. Ahonen, A. Hadid, and M. Pietikainen. Face description with local binary patterns: application to face recognition. TPAMI, 2006.

Digital Library

[2]

S. Belongie, J. Malik, and J. Puzicha. Shape matching and object recognition using shape contexts. TPAMI, 2002.

Digital Library

[3]

F. Bookstein. Principal warps: thin-plate splines and the decomposition of deformations. TPAMI, 1989.

Digital Library

[4]

Y. Boykov and O. Veksler. Fast approximate energy minimization via graph cuts. TPAMI, 2001.

Digital Library

[5]

C. Chang and C. Lin. Libsvm: A library for support vector machines. In TIST, 2011.

Digital Library

[6]

F. Chen and D. Zhang. A benchmark for geometric facial beauty study. In Int. Conf. Medical Biometrics, 2010.

Digital Library

[7]

C. Chow and C. Liu. Approximating discrete probability distributions with dependence trees. TIT, 1968.

Digital Library

[8]

T. Cootes, C. Taylor, D. Cooper, and J. Graham. Active shape models-their training and application. CVIU, 1995.

Digital Library

[9]

A. Criminisi, P. Perez, and K. Toyama. Region filling and object removal by exemplar-based image inpainting. TIP, 2004.

Digital Library

[10]

N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In CVPR, 2005.

Digital Library

[11]

M. de Berg, O. Cheong, M. van Kreveld, and M. Overmars. Computational Geometry: Algorithms and Applications. Springer-Verlag, third edition, 2008.

Digital Library

[12]

P. Felzenszwalb, R. Girshick, D. McAllester, and D. Ramanan. Object detection with discriminatively trained part-based models. TPAMI, 2010.

Digital Library

[13]

R. Feris and L. Davis. Image ranking and retrieval based on multi-attribute queries. In CVPR, 2011.

[14]

A. Goshtasby. Piecewise linear mapping functions for image registration. PR, 1986.

[15]

D. Guo and T. Sim. Digital face makeup by example. In CVPR, 2009.

[16]

S. Haykin. Neural Networks. Prentice Hall, 1999.

Digital Library

[17]

K. He, J. Sun, and X. Tang. Guided image filtering. In ECCV, 2010.

Digital Library

[18]

G. Huang, M. Ramesh, T. Berg, and E. Learned. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical report, University of Massachusetts, 2007.

[19]

T. Joachims. Optimizing search engines using clickthrough data. In ACM KDD, 2002.

Digital Library

[20]

I. Jolliffe. Principal component analysis. Encyclopedia of Statistics in Behavioral Science, 2002.

[21]

D. Koller and N. Friedman. Probabilistic Graphical Models: Principles and Techniques. MIT Press, 2009.

Digital Library

[22]

A. Levin, D. Lischinski, and Y. Weiss. A closed-form solution to natural image matting. TPAMI, 2008.

Digital Library

[23]

S. Liu, J. Feng, Z. Song, T. Zhang, H. Lu, C. Xu, and S. Yan. "Hi, magic closet, tell me what to wear". In ACM MM, 2012.

Digital Library

[24]

T. Mensink, J. Verbeek, and G. Csurka. Tree-structured crf models for interactive image labeling. TPAMI, 2013.

Digital Library

[25]

Y. Nagai, K. Ushiro, Y. Matsunami, T. Hashimoto, and Y. Kojima. Hairstyle suggesting system, hairstyle suggesting method, and computer program product. US Patent US20050251463 A1, 2005.

[26]

C. Rother, V. Kolmogorov, and A. Blake. Grabcut: Interactive foreground extraction using iterated graph cuts. TOG, 2004.

Digital Library

[27]

K. Scherbaum, T. Ritschel, M. Hullin, T. Thormahlen, V. Blanz, and H. Seidel. Computer-suggested facial makeup. CGF, 2011.

[28]

W. Tong, C. Tang, M. Brown, and Y. Xu. Example-based cosmetic transfer. In FG, 2007.

[29]

N. Wang, H. Ai, and F. Tang. What are good parts for hair shape modeling? In CVPR, 2012.

Digital Library

[30]

Y. Wang and G. Mori. A discriminative latent model of object classes and attributes. In ECCV, 2010.

Digital Library

[31]

Y. Yang and D. Ramanan. Articulated pose estimation with exible mixtures-of-parts. In CVPR, 2011.

Digital Library

Cited By

Deldjoo YNazary FRamisa AMcAuley JPellegrini GBellogin ANoia T(2023)A Review of Modern Fashion Recommender SystemsACM Computing Surveys10.1145/362473356:4(1-37)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3624733
Wang HXie HMiyata K(2022)ILoveEye: Eyeliner Makeup Guidance System with Eye Shape FeaturesAdaptive Instructional Systems10.1007/978-3-031-05887-5_17(238-254)Online publication date: 26-Jun-2022
https://dl.acm.org/doi/10.1007/978-3-031-05887-5_17
Ponlawan PKaewsaitiam NPhimoltares SPanthuwadeethorn S(2021)Guideline of Personalized Facial Makeup Using a Hierarchical Cascade Classifier2021 18th International Joint Conference on Computer Science and Software Engineering (JCSSE)10.1109/JCSSE53117.2021.9493828(1-6)Online publication date: 30-Jun-2021
https://doi.org/10.1109/JCSSE53117.2021.9493828
Show More Cited By

Index Terms

"Wow! you are so beautiful today!"
1. Computing methodologies
  1. Machine learning
    1. Learning settings
2. Information systems
  1. Information retrieval
    1. Retrieval models and ranking

Recommendations

"Wow! you are so beautiful today!"
MM '13: Proceedings of the 21st ACM international conference on Multimedia

In this demo, we present Beauty e-Experts, a fully automatic system for hairstyle and facial makeup recommendation and synthesis. Given a user-provided frontal facial image with short/bound hair and no/light makeup, the Beauty e-Experts system can not ...
“Wow! You Are So Beautiful Today!”
Special Issue on Multiple Sensorial (MulSeMedia) Multimodal Media : Advances and Applications

Beauty e-Experts, a fully automatic system for makeover recommendation and synthesis, is developed in this work. The makeover recommendation and synthesis system simultaneously considers many kinds of makeover items on hairstyle and makeup. Given a user-...
The Painter X Wow! Book

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '13: Proceedings of the 21st ACM international conference on Multimedia

October 2013

1166 pages

ISBN:9781450324045

DOI:10.1145/2502081

General Chairs:
Alejandro (Alex) Jaimes
Yahoo!, Spain
,
Nicu Sebe
University of Trento, Italy
,
Nozha Boujemaa
INRIA, France
,
Program Chairs:
Daniel Gatica-Perez
IDIAP & EPFL, Switzerland
,
David A. Shamma
Yahoo!, USA
,
Marcel Worring
University of Amsterdam, The Netherlands
,
Roger Zimmermann
National University of Singapore, Singapore

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 October 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM '13

Sponsor:

SIGMM

MM '13: ACM Multimedia Conference

October 21 - 25, 2013

Barcelona, Spain

Acceptance Rates

MM '13 Paper Acceptance Rate 47 of 235 submissions, 20%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
4,458
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)3

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Deldjoo YNazary FRamisa AMcAuley JPellegrini GBellogin ANoia T(2023)A Review of Modern Fashion Recommender SystemsACM Computing Surveys10.1145/362473356:4(1-37)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3624733
Wang HXie HMiyata K(2022)ILoveEye: Eyeliner Makeup Guidance System with Eye Shape FeaturesAdaptive Instructional Systems10.1007/978-3-031-05887-5_17(238-254)Online publication date: 26-Jun-2022
https://dl.acm.org/doi/10.1007/978-3-031-05887-5_17
Ponlawan PKaewsaitiam NPhimoltares SPanthuwadeethorn S(2021)Guideline of Personalized Facial Makeup Using a Hierarchical Cascade Classifier2021 18th International Joint Conference on Computer Science and Software Engineering (JCSSE)10.1109/JCSSE53117.2021.9493828(1-6)Online publication date: 30-Jun-2021
https://doi.org/10.1109/JCSSE53117.2021.9493828
Gu QWang GChiu MTai YTang C(2019)LADN: Local Adversarial Disentangling Network for Facial Makeup and De-Makeup2019 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV.2019.01058(10480-10489)Online publication date: Oct-2019
https://doi.org/10.1109/ICCV.2019.01058
Kim GKo H(2018)A Practical Approach to Physically‐Based Reproduction of Diffusive CosmeticsComputer Graphics Forum10.1111/cgf.1356237:7(223-232)Online publication date: 24-Oct-2018
https://doi.org/10.1111/cgf.13562
Wiwatwattana NChareonnivassakul SMaleerutmongkol NCharoenvitvorakul C(2018)Enhancing Lipstick Try-On with Augmented Reality and Color Prediction ModelInformation Technology – New Generations10.1007/978-3-319-77028-4_48(359-367)Online publication date: 2018
https://doi.org/10.1007/978-3-319-77028-4_48
Alashkar TJiang SWang SFu YSingh SMarkovitch S(2017)Examples-rules guided deep neural network for makeup recommendationProceedings of the Thirty-First AAAI Conference on Artificial Intelligence10.5555/3298239.3298377(941-947)Online publication date: 4-Feb-2017
https://dl.acm.org/doi/10.5555/3298239.3298377
Alashkar TJiang SFu Y(2017)Rule-Based Facial Makeup Recommendation System2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017)10.1109/FG.2017.47(325-330)Online publication date: May-2017
https://doi.org/10.1109/FG.2017.47
Feris RLampert CParikh D(2017)Introduction to Visual AttributesVisual Attributes10.1007/978-3-319-50077-5_1(1-7)Online publication date: 22-Mar-2017
https://doi.org/10.1007/978-3-319-50077-5_1
Javornik ARogers YMoutinho AFreeman RFoth MJu WSchroeter RViller S(2016)Revealing the Shopper Experience of Using a "Magic Mirror" Augmented Reality Make-Up ApplicationProceedings of the 2016 ACM Conference on Designing Interactive Systems10.1145/2901790.2901881(871-882)Online publication date: 4-Jun-2016
https://dl.acm.org/doi/10.1145/2901790.2901881
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents