Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Deep metric learning with dynamic margin hard sampling loss for face verification

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Metric learning with triplet loss is one of the most effective methods for face verification, which aims to minimize the distance of positive pairs while maximizing the distance of negative pairs in feature embedding space. The arduous hard triplets mining and insufficient inter-class and intra-class variations are the two limitations of the previous methods. In this paper, we propose an improved triplet loss based on deep neural network for end-to-end metric learning, which can effectively cut down the number of the possible triplets and increase the proportion of hard triplets. It considers not only the relative distance between positive and negative pairs with the same probe images, but also the absolute distance between positive and negative pairs with different probe images, resulting in a smaller intra-class variation and a larger inter-class variation by adding a new constraint to push away negative pairs from positive pairs. In particular, a dynamic margin is proposed based on the distribution of positive and negative pairs in a batch, which can avoid the under-sampling or over-sampling problems. Our method is evaluated on LFW and YTF datasets, which are the most widely used benchmarks. The experimental results indicate that the proposed method can greatly outperform the standard triplet loss, and obtain the state-of-the-art performance with less time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Yu, S., Guiquan, L.: Face recognition method based on HOG and LBP feature. Comput. Eng. 41(9), 205–208 (2015)

    Google Scholar 

  2. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: International Conference on Computer Vision & Pattern Recognition (CVPR’05), vol. 1, pp. 886–893. IEEE Computer Society (2005)

  3. Zhang, Y., Li, S.: Gabor-LBP based region covariance descriptor for person re-identification. In: 2011 Sixth International Conference on Image and Graphics, pp. 368–371. IEEE (2011)

  4. Sanket, P., Shaurya, N., Rohit, K., et al.: A performance study of SIFT SIFT-PCA and SIFT-LDA for face recognition. Int. J. Soft Comput. Eng. 7, 66–72 (2015)

    Google Scholar 

  5. Kaya, M., BİLGE, H.: Deep metric learning: a survey. Symmetry 11(9), 1066 (2019)

    Article  Google Scholar 

  6. Hu, C.-H., Lu, X.-B., Liu, P., Jing, X.-Y., Yue, D.: Single sample face recognition under varying illumination via QRCP decomposition. IEEE Trans. Image Process. 28(5), 2624–2638 (2019)

    Article  MathSciNet  Google Scholar 

  7. Jutten, C., Herault, J.: Blind separation of sources, part I: an adaptive algorithm based on neuromimetic architecture. Signal Process. 24(1), 1–10 (1991)

    Article  Google Scholar 

  8. Wu, Y.-H., Liang, W.-L.: Research on ICA face recognition algorithm based on Gabor wavelet transform. Microelectron. Comput. 7 (2013)

  9. Hu, C., Lu, X., Ye, M., Zeng, W.: Singular value decomposition and local near neighbors for face recognition under varying illumination. Pattern Recognit. 64, 60–83 (2017)

    Article  Google Scholar 

  10. Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)

  11. Shi, H., Yang, Y., Zhu, X., Liao, S., Lei, Z., Zheng, W., Li, S.Z.: Embedding deep metric for person re-identification: a study against large variations. In: European Conference on Computer Vision, pp. 732–748. Springer (2016)

  12. Oh Song, H., Xiang, Y., Jegelka, S., Savarese, S.: Deep metric learning via lifted structured feature embedding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4004–4012 (2016)

  13. Chen, W., Chen, X., Zhang, J., Huang, K.: Beyond triplet loss: a deep quadruplet network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 403–412 (2017)

  14. Kim, S., Seo, M., Laptev, I., Cho, M., Kwak, S.: Deep metric learning beyond binary supervision. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

  15. Hu, C., Ye, M., Ji, S., Zeng, W., Lu, X.: A new face recognition method based on image decomposition for single sample per person problem. Neurocomputing 160, 287–299 (2015)

    Article  Google Scholar 

  16. Hermans, A., Beyer, L., Leibe, B.: In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737 (2017)

  17. Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)

  18. Huang, G.B., Mattar, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. In: Workshop on Faces in ’Real-Life’ Images: Detection, Alignment, and Recognition (2008)

  19. Ming, Z., Chazalon, J., Luqman, M.M., Visani, M., Burie, J.-C.: Simple triplet loss based on intra/inter-class metric learning for face verification. In: 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 1656–1664. IEEE (2017)

  20. Sun, Y., Wang, X., Tang, X.: Hybrid deep learning for face verification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1489–1496 (2013)

  21. Zhu, X., Lei, Z., Yan, J., Yi, D., Li, S.Z.: High-fidelity pose and expression normalization for face recognition in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 787–796 (2015)

  22. Simonyan, K., Parkhi, O.M., Vedaldi, A., Zisserman, A.: Fisher vector faces in the wild. In: BMVC, vol. 2, no. 3, p. 4 (2013)

  23. Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: Closing the gap to human-level performance in face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1701–1708 (2014)

  24. Sun, Y., Chen, Y., Wang, X., Tang, X.: Deep learning face representation by joint identification-verification. In: Advances in Neural Information Processing Systems, pp. 1988–1996 (2014)

  25. Sun, Y., Liang, D., Wang, X., Tang, X.: Deepid3: face recognition with very deep neural networks. arXiv preprint arXiv:1502.00873 (2015)

  26. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  27. Wen, Y., Zhang, K., Li, Z., Qiao, Y.: A discriminative feature learning approach for deep face recognition. In: European Conference on Computer Vision, pp. 499–515. Springer (2016)

  28. Li, H., Hua, G., Lin, Z., Brandt, J., Yang, J.: Probabilistic elastic matching for pose variant face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3499–3506 (2013)

  29. Kan, M., Xu, D., Shan, S., Li, W., Chen, X.: Learning prototype hyperplanes for face verification in the wild. IEEE Trans. Image Process. 22(8), 3310–3316 (2013)

    Article  Google Scholar 

  30. Liu, W., Wen, Y., Yu, Z., Yang, M.: Large-margin softmax loss for convolutional neural networks. In: ICML, vol. 2, no. 3, p. 7 (2016)

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant 61802203, Natural Science Foundation of Jiangsu Province under Grant BK20180761, China Postdoctoral Science Foundation under Grant 2019M651653, Postdoctoral Research Funding Program of Jiangsu Province under Grant 2019K124, and NUPTSF under Grant NY218119.

Author information

Authors and Affiliations

  1. College of Automation and College of Artificial Intelligence, Nanjing University of Posts and Telecommunications, Nanjing, 210023, China

    Jian Yu, Chang-Hui Hu, Xiao-Yuan Jing & Yu-Jian Feng

Authors
  1. Jian Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chang-Hui Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiao-Yuan Jing
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu-Jian Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chang-Hui Hu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, J., Hu, CH., Jing, XY. et al. Deep metric learning with dynamic margin hard sampling loss for face verification. SIViP 14, 791–798 (2020). https://doi.org/10.1007/s11760-019-01612-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-019-01612-3

Keywords

Search

Navigation