article

Discriminative Deep Face Shape Model for Facial Point Detection

Authors:

Qiang JiAuthors Info & Claims

International Journal of Computer Vision, Volume 113, Issue 1

Pages 37 - 53

https://doi.org/10.1007/s11263-014-0775-8

Published: 01 May 2015 Publication History

Abstract

Facial point detection is an active area in computer vision due to its relevance to many applications. It is a nontrivial task, since facial shapes vary significantly with facial expressions, poses or occlusion. In this paper, we address this problem by proposing a discriminative deep face shape model that is constructed based on an augmented factorized three-way Restricted Boltzmann Machines model. Specifically, the discriminative deep model combines the top-down information from the embedded face shape patterns and the bottom up measurements from local point detectors in a unified framework. In addition, along with the model, effective algorithms are proposed to perform model learning and to infer the true facial point locations from their measurements. Based on the discriminative deep face shape model, 68 facial points are detected on facial images in both controlled and "in-the-wild" conditions. Experiments on benchmark data sets show the effectiveness of the proposed facial point detection algorithm against state-of-the-art methods.

References

[1]

Baker, S., Gross, R., & Matthews, I. (2002). Lucas-kanade 20 years on: A unifying framework: Part 3. International Journal of Computer Vision, 56, 221-255.

Digital Library

[2]

Belhumeur, P., Jacobs, D., Kriegman, D., & Kumar, N. (2013). Localizing parts of faces using a consensus of exemplars. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(12), 2930-2940.

Digital Library

[3]

Belhumeur, P. N., Jacobs, D. W., Kriegman, D. J., & Kumar, N. (2011). Localizing parts of faces using a consensus of exemplars. In IEEE International Conference on Computer Vision and Pattern Recognition.

[4]

Cootes, T. F., Taylor, C. J., Cooper, D. H., & Graham, J. (1995). Active shape models their training and application. Computer Vision and Image Understanding, 61(1), 38-59.

Digital Library

[5]

Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001). Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 681-685.

Digital Library

[6]

Cristinacce, D., & Cootes, T. (2008). Automatic feature localisation with constrained local models. Pattern Recognition, 41(10), 3054-3067.

Digital Library

[7]

Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. International Conference on Computer Vision and Pattern Recognition, 2, 886-893.

[8]

Eslami, S., Heess, N., & Winn, J. (2012). The shape boltzmann machine: A strong model of object shape. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 406-413).

[9]

Fan, R. E., Chang, K. W., Hsieh, C. J., Wang, X. R., & Lin, C. J. (2008). LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research, 9, 1871-1874.

Digital Library

[10]

Gross, R., Matthews, I., Cohn, J., Kanade, T., & Baker, S. (2010). Multipie. Image and Vision Computing, 28(5), 807-813.

Digital Library

[11]

Hinton, G. E. (2002). Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8), 1771-1800.

Digital Library

[12]

Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science, 313(5786), 504-507.

[13]

Kae, A., Sohn, K., Lee, H., & Learned-Miller, E. G. (2013). Augmenting crfs with boltzmann machine shape priors for image labeling. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 2019-2026).

[14]

Le, V., Brandt, J., Lin, Z., Bourdev, L., & Huang, T. S. (2012). Interactive facial feature localization. In European Conference on Computer Vision, Part III (ECCV'12, pp. 679-692).

[15]

Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal on Computer Vision, 60(2), 91-110.

Digital Library

[16]

Martinez, B., Valstar, M. F., Binefa, X., & Pantic, M. (2013). Local evidence aggregation for regression-based facial point detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(5), 1149-1163.

[17]

Matthews, I., & Baker, S. (2004). Active appearance models revisited. International Journal of Computer Vision, 60(2), 135-164.

Digital Library

[18]

Memisevic, R., & Hinton, G. E. (2010). Learning to represent spatial transformations with factored higher-order boltzmann machines. Neural Computation, 22(6), 1473-1492.

Digital Library

[19]

Mohamed, A., Dahl, G., & Hinton, G. (2011). Acoustic modeling using deep belief networks. IEEE Transactions on Audio, Speech, and Language Processing, PP(99), 1.

[20]

Ranzato, M., Krizhevsky, A., & Hinton, G. E. (2010). Factored 3-way restricted boltzmann machines for modeling natural images. In International Conference on Artificial Intelligence and Statistics (pp. 621-628).

[21]

Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M. (2013). 300 faces in-the-wild challenge: The first facial landmark localization challenge. In Proceedings of IEEE International Conference on Computer Vision (ICCV-W 2013), Sydney.

[22]

Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., & Pantic, M. (2013). A semi-automatic methodology for facial landmark annotation. In Computer Vision and Pattern Recognition Workshops (CVPRW, pp. 896-903).

[23]

Salakhutdinov, R., & Hinton, G. (2009). Deep boltzmann machines. Proceedings of the International Conference on Artificial Intelligence and Statistics, 5, 448-455.

[24]

Saragih, J. M., Lucey, S., & Cohn, J. F. (2011). Deformable model fitting by regularized landmark mean-shift. International Journal of Computer Vision, 91(2), 200-215.

Digital Library

[25]

Smola, A. J., & Schölkopf, B. (2004). A tutorial on support vector regression. Statistics and Computing, 14(3), 199-222.

Digital Library

[26]

Sun, Y., Wang, X., & Tang, X. (2013a). Deep convolutional network cascade for facial point detection. In IEEE International Conference on Computer Vision and Pattern Recognition (pp. 3476-3483).

[27]

Sun, Y., Wang, X., & Tang, X. (2013b). Hybrid deep learning for face verification. In 2013 IEEE International Conference on Computer Vision (ICCV), pp. 1489-1496.

[28]

Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014). DeepFace: Closing the gap to human-level performance in face verification. In 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1701-1708.

[29]

Taylor, G., Sigal, L., Fleet, D., & Hinton, G. (2010). Dynamical binary latent variable models for 3d human pose tracking. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR, pp. 631-638).

[30]

Tieleman, T. (2008). Training restricted boltzmann machines using approximations to the likelihood gradient. In Proceedings of the 25th International Conference on Machine Learning (pp. 1064-1071).

[31]

Tzimiropoulos, G., & Pantic, M. (2013). Optimization problems for fast aam fitting in-the-wild. In International conference on Computer Vision (pp. 593-600).

[32]

Valstar, M., Martinez, B., Binefa, V., & Pantic, M. (2010). Facial point detection using boosted regression and graph models. In IEEE International Conference on Computer Vision and Pattern Recognition (pp. 13-18).

[33]

Welling, M., & Hinton, G. E. (2002). A new learning algorithm for mean field boltzmann machines. In Proceedings of the International Conference on Artificial Neural Networks (ICANN '02, pp 351-357). London: Springer.

[34]

Wu, Y., Wang, Z., & Ji, Q. (2013). Facial feature tracking under varying facial expressions and face poses based on restricted boltzmann machines. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 3452-3459).

[35]

Xiong, X., & De la Torre Frade, F. (2013). Supervised descent method and its applications to face alignment. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR).

[36]

Zhou, E., Fan, H., Cao, Z., Jiang, Y., & Yin, Q. (2013). Extensive facial landmark localization with coarse-to-fine convolutional network cascade. In IEEE International Conference on Computer Vision Workshops (pp. 386-391).

[37]

Zhu, X., & Ramanan, D. (2012). Face detection, pose estimation, and landmark localization in the wild. In IEEE International Conference on Computer Vision and Pattern Recognition (pp. 2879-2886).

Cited By

Huang BChen XHuang GLi QLiang GZhong QLi YKe MChen HXie D(2024)Research of Facial Landmark Detection Algorithm based on Deep LearningProceedings of the 5th International Conference on Computer Information and Big Data Applications10.1145/3671151.3671251(561-569)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3671151.3671251
Zhong YPei YLi PGuo YMa GLiu MBai WWu WZha H(2020)Face Denoising and 3D Reconstruction from A Single Depth Image2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020)10.1109/FG47880.2020.00005(117-124)Online publication date: 16-Nov-2020
https://dl.acm.org/doi/10.1109/FG47880.2020.00005
Wu YJi Q(2019)Facial Landmark Detection: A Literature SurveyInternational Journal of Computer Vision10.1007/s11263-018-1097-z127:2(115-142)Online publication date: 15-Feb-2019
https://dl.acm.org/doi/10.1007/s11263-018-1097-z
Show More Cited By

Discriminative Deep Face Shape Model for Facial Point Detection
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems

Recommendations

Adaptive facial point detection and emotion recognition for a humanoid robot

We propose a robust landmark detector to deal with pose variation and occlusions.SVRs and NNs are respectively used to estimate intensities of 18 selected AUs.Fuzzy c-means clustering is employed to detect seven basic and compound emotions.Our ...
Deep face recognition using imperfect facial data
Abstract
Today, computer based face recognition is a mature and reliable mechanism which is being practically utilised for many access control scenarios. As such, face recognition or authentication is predominantly performed using ‘perfect’ ...
Highlights
- We show the performance of machine learning for face recognition using partial faces and other manipulations of the face such as rotation and zooming which ...
Deep Convolutional Network Cascade for Facial Point Detection
CVPR '13: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition

We propose a new approach for estimation of the positions of facial key points with three-level carefully designed convolutional networks. At each level, the outputs of multiple networks are fused for robust and accurate estimation. Thanks to the deep ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image International Journal of Computer Vision

International Journal of Computer Vision Volume 113, Issue 1

May 2015

79 pages

ISSN:0920-5691

Issue’s Table of Contents

Copyright © Copyright © 2015 Springer Science+Business Media New York.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 May 2015

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Huang BChen XHuang GLi QLiang GZhong QLi YKe MChen HXie D(2024)Research of Facial Landmark Detection Algorithm based on Deep LearningProceedings of the 5th International Conference on Computer Information and Big Data Applications10.1145/3671151.3671251(561-569)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3671151.3671251
Zhong YPei YLi PGuo YMa GLiu MBai WWu WZha H(2020)Face Denoising and 3D Reconstruction from A Single Depth Image2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020)10.1109/FG47880.2020.00005(117-124)Online publication date: 16-Nov-2020
https://dl.acm.org/doi/10.1109/FG47880.2020.00005
Wu YJi Q(2019)Facial Landmark Detection: A Literature SurveyInternational Journal of Computer Vision10.1007/s11263-018-1097-z127:2(115-142)Online publication date: 15-Feb-2019
https://dl.acm.org/doi/10.1007/s11263-018-1097-z
Liliana DBasaruddin TOriza I(2018)The Indonesian Mixed Emotion Dataset (IMED)Proceedings of the 2018 International Conference on Artificial Intelligence and Virtual Reality10.1145/3293663.3293671(56-60)Online publication date: 23-Nov-2018
https://dl.acm.org/doi/10.1145/3293663.3293671
Liu HLu JFeng JZhou J(2018)Two-Stream Transformer Networks for Video-Based Face AlignmentIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2017.273477940:11(2546-2554)Online publication date: 1-Nov-2018
https://dl.acm.org/doi/10.1109/TPAMI.2017.2734779
Liliana DBasaruddin CWidyanto M(2017)Mix Emotion Recognition from Facial Expression using SVM-CRF Sequence ClassifierProceedings of the 1st International Conference on Algorithms, Computing and Systems10.1145/3127942.3127958(27-31)Online publication date: 10-Aug-2017
https://dl.acm.org/doi/10.1145/3127942.3127958
Liu HLu JFeng JZhou J(2017)Learning Deep Sharable and Structural Detectors for Face AlignmentIEEE Transactions on Image Processing10.1109/TIP.2017.265711826:4(1666-1678)Online publication date: 1-Apr-2017
https://dl.acm.org/doi/10.1109/TIP.2017.2657118
Jin XTan X(2017)Face alignment in-the-wildComputer Vision and Image Understanding10.1016/j.cviu.2017.08.008162:C(1-22)Online publication date: 1-Sep-2017
https://dl.acm.org/doi/10.1016/j.cviu.2017.08.008
Wu YKender JSmith JLuo JBoll SHsu W(2016)Facial Landmark Detection and Tracking for Facial Behavior AnalysisProceedings of the 2016 ACM on International Conference on Multimedia Retrieval10.1145/2911996.2912034(431-434)Online publication date: 6-Jun-2016
https://dl.acm.org/doi/10.1145/2911996.2912034
Zhang LTjondronegoro DChandran VEggink J(2016)Towards robust automatic affective classification of images using facial expressions for practical applicationsMultimedia Tools and Applications10.1007/s11042-015-2497-575:8(4669-4695)Online publication date: 1-Apr-2016
https://dl.acm.org/doi/10.1007/s11042-015-2497-5
Show More Cited By

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents