Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Iterative Self-distillation for Precise Facial Landmark Localization

  • Conference paper
  • First Online:
Computer Vision – ACCV 2020 Workshops (ACCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12628))

Included in the following conference series:

  • 454 Accesses

Abstract

In this paper, we propose a novel training method to improve the precision of facial landmark localization. When a facial landmark localization method is applied to a facial video, the detected landmarks occasionally jitter, whereas the face apparently does not move. We hypothesize that there are two causes that induce the unstable detection: (1) small changes in input images and (2) inconsistent annotations. Corresponding to the causes, we propose (1) two loss terms to make a model robust to changes in the input images and (2) self-distillation training to reduce the effect of the annotation noise. We show that our method can improve the precision of facial landmark localization by reducing the variance using public facial landmark datasets, 300-W and 300-VW. We also show that our method can reduce jitter of predicted landmarks when applied to a video.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Koestinger, M., Wohlhart, P., Roth, P.M., Bischof, H.:Annotated facial landmarks in the wild: a large-scale, real-world database for facial landmark localization. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 2144–2151. IEEE (2011)

    Google Scholar 

  2. Gilani, S.Z., Mian, A., Eastwood, P.: Deep, dense and accurate 3D face correspondence for generating population specific deformable models. Pattern Recogn. 69, 238–250 (2017)

    Article  Google Scholar 

  3. Wang, K., Zhao, R., Ji, Q.: A hierarchical generative model for eye image synthesis and eye gaze estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 440–448 (2018)

    Google Scholar 

  4. Wang, Z., Yang, X., Cheng, K.T.: Accurate face alignment and adaptive patch selection for heart rate estimation from videos under realistic scenarios. PLoS ONE 13, e0197275 (2018)

    Article  Google Scholar 

  5. Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23, 681–685 (2001)

    Article  Google Scholar 

  6. Cao, X., Wei, Y., Wen, F., Sun, J.: Face alignment by explicit shape regression. Int. J. Comput. Vision 107, 177–190 (2014). https://doi.org/10.1007/s11263-013-0667-3

    Article  MathSciNet  Google Scholar 

  7. Ren, S., Cao, X., Wei, Y., Sun, J.: Face alignment at 3000 FPS via regressing local binary features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1685–1692 (2014)

    Google Scholar 

  8. Sun, Y., Wang, X., Tang, X.: Deep convolutional network cascade for facial point detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3476–3483 (2013)

    Google Scholar 

  9. Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Facial landmark detection by deep multi-task learning. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 94–108. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10599-4_7

    Chapter  Google Scholar 

  10. Merget, D., Rock, M., Rigoll, G.: Robust facial landmark detection via a fully-convolutional local-global context network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 781–790 (2018)

    Google Scholar 

  11. Liu, H., Lu, J., Feng, J., Zhou, J.: Two-stream transformer networks for video-based face alignment. IEEE Trans. Pattern Anal. Mach. Intell. 40, 2546–2554 (2017)

    Article  Google Scholar 

  12. Sánchez-Lozano, E., Tzimiropoulos, G., Martinez, B., De la Torre, F., Valstar, M.: A functional regression approach to facial landmark tracking. IEEE Trans. Pattern Anal. Mach. Intell. 40, 2037–2050 (2017)

    Article  Google Scholar 

  13. Belmonte, R., Ihaddadene, N., Tirilly, P., Bilasco, I.M., Djeraba, C.: Video-based face alignment with local motion modeling. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 2106–2115. IEEE (2019)

    Google Scholar 

  14. Guo, M., Lu, J., Zhou, J.: Dual-agent deep reinforcement learning for deformable face tracking. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11214, pp. 783–799. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01249-6_47

    Chapter  Google Scholar 

  15. Dong, X., Yu, S.I., Weng, X., Wei, S.E., Yang, Y., Sheikh, Y.: Supervision-by-registration: An unsupervised approach to improve the precision of facial landmark detectors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 360–368 (2018)

    Google Scholar 

  16. Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)

  17. Gao, P., Lu, K., Xue, J.: EfficientFAN: deep knowledge transfer for face alignment. In: Proceedings of the 2020 International Conference on Multimedia Retrieval, pp. 215–223 (2020)

    Google Scholar 

  18. Furlanello, T., Lipton, Z., Tschannen, M., Itti, L., Anandkumar, A.: Born-again neural networks. In: International Conference on Machine Learning, pp. 1602–1611 (2018)

    Google Scholar 

  19. Bagherinezhad, H., Horton, M., Rastegari, M., Farhadi, A.: Label refinery: improving imagenet classification through label progression. arXiv preprint arXiv:1805.02641 (2018)

  20. Kato, N., Li, T., Nishino, K., Uchida, Y.: Improving multi-person pose estimation using label correction. arXiv preprint arXiv:1811.03331 (2018)

  21. Honari, S., Molchanov, P., Tyree, S., Vincent, P., Pal, C., Kautz, J.: Improving landmark localization with semi-supervised learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1546–1555 (2018)

    Google Scholar 

  22. Sagonas, C., Antonakos, E., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: database and results. Image Vis. Comput. 47, 3–18 (2016)

    Article  Google Scholar 

  23. Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: the first facial landmark localization challenge. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 397–403 (2013)

    Google Scholar 

  24. Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: A semi-automatic methodology for facial landmark annotation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 896–903 (2013)

    Google Scholar 

  25. Belhumeur, P.N., Jacobs, D.W., Kriegman, D.J., Kumar, N.: Localizing parts of faces using a consensus of exemplars. In: CVPR 2011, pp. 545–552. IEEE (2011)

    Google Scholar 

  26. Ramanan, D., Zhu, X.:Face detection, pose estimation, and landmark localization in the wild. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2879–2886. IEEE (2012)

    Google Scholar 

  27. Le, V., Brandt, J., Lin, Z., Bourdev, L., Huang, T.S.: Interactive facial feature localization. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7574, pp. 679–692. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33712-3_49

    Chapter  Google Scholar 

  28. Messer, K., Matas, J., Kittler, J., Luettin, J., Maitre, G.: XM2VTSDB: the extended m2vts database. In: Second International Conference on Audio and Video-Based Biometric Person Authentication, vol. 964, pp. 965–966 (1999)

    Google Scholar 

  29. Zhu, S., Li, C., Change Loy, C., Tang, X.: Face alignment by coarse-to-fine shape searching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4998–5006 (2015)

    Google Scholar 

  30. Chrysos, G.G., Antonakos, E., Zafeiriou, S., Snape, P.: Offline deformable face tracking in arbitrary videos. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1–9 (2015)

    Google Scholar 

  31. Shen, J., Zafeiriou, S., Chrysos, G.G., Kossaifi, J., Tzimiropoulos, G., Pantic, M.: The first facial landmark tracking in-the-wild challenge: benchmark and results. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 50–58 (2015)

    Google Scholar 

  32. Tzimiropoulos, G.: Project-out cascaded regression with an application to face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3659–3667 (2015)

    Google Scholar 

  33. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  34. Bulat, A., Tzimiropoulos, G.: How far are we from solving the 2D & 3D face alignment problem? (and a dataset of 230,000 3d facial landmarks). In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1021–1030 (2017)

    Google Scholar 

  35. Tokui, S., et al.: Chainer: a deep learning framework for accelerating the research cycle. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2002–2011. ACM (2019)

    Google Scholar 

  36. Tokui, S., Oono, K., Hido, S., Clayton, J.: Chainer: a next-generation open source framework for deep learning. In: Proceedings of Workshop on Machine Learning Systems (LearningSys) in The Twenty-ninth Annual Conference on Neural Information Processing Systems (NIPS) (2015)

    Google Scholar 

  37. Akiba, T., Fukuda, K., Suzuki, S.: ChainerMN: scalable distributed deep learning framework. In: Proceedings of Workshop on ML Systems in The Thirty-first Annual Conference on Neural Information Processing Systems (NIPS) (2017)

    Google Scholar 

  38. Dong, X., Yan, Y., Ouyang, W., Yang, Y.: Style aggregated network for facial landmark detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 379–388 (2018)

    Google Scholar 

  39. Wang, J., et al.: Deep high-resolution representation learning for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. OMRON Corporation, 9-1 Kizugawadai, Kizugawa-City, Kyoto, 619-0283, Japan

    Shigenori Nagae & Yamato Takeuchi

Authors
  1. Shigenori Nagae
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yamato Takeuchi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shigenori Nagae .

Editor information

Editors and Affiliations

  1. National Institute of Informatics, Tokyo, Japan

    Imari Sato

  2. Seoul National University, Seoul, Korea (Republic of)

    Bohyung Han

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 1001 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nagae, S., Takeuchi, Y. (2021). Iterative Self-distillation for Precise Facial Landmark Localization. In: Sato, I., Han, B. (eds) Computer Vision – ACCV 2020 Workshops. ACCV 2020. Lecture Notes in Computer Science(), vol 12628. Springer, Cham. https://doi.org/10.1007/978-3-030-69756-3_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-69756-3_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-69755-6

  • Online ISBN: 978-3-030-69756-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation