research-article

Free access

Residual compensation networks for heterogeneous face recognition

AUTHORs:

Zhongying Deng,

Xiaojiang Peng,

Yu QiaoAuthors Info & Claims

AAAI'19/IAAI'19/EAAI'19: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence

Article No.: 1010, Pages 8239 - 8246

https://doi.org/10.1609/aaai.v33i01.33018239

Published: 27 January 2019 Publication History

PDF eReader Publisher Site

Abstract

Heterogeneous Face Recognition (HFR) is a challenging task due to large modality discrepancy as well as insufficient training images in certain modalities. In this paper, we propose a new two-branch network architecture, termed as Residual Compensation Networks (RCN), to learn separated features for different modalities in HFR. The RCN incorporates a residual compensation (RC) module and a modality discrepancy loss (MD loss) into traditional convolutional neural networks. The RC module reduces modal discrepancy by adding compensation to one of the modalities so that its representation can be close to the other modality. The MD loss alleviates modal discrepancy by minimizing the cosine distance between different modalities. In addition, we explore different architectures and positions for the RC module, and evaluate different transfer learning strategies for HFR. Extensive experiments on IIIT-D Viewed Sketch, Forensic Sketch, CASIA NIR-VIS 2.0 and CUHK NIR-VIS show that our RCN outperforms other state-of-the-art methods significantly.

References

[1]

Bhatt, H. S.; Bharadwaj, S.; Singh, R.; and Vatsa, M. 2012a. Memetic approach for matching sketches with digital face images.

[2]

Bhatt, H. S.; Bharadwaj, S.; Singh, R.; and Vatsa, M. 2012b. Memetically optimized mcwld for matching sketches with digital face images. IEEE T-IFS 7(5):1522–1535.

[3]

Cao, K.; Rong, Y.; Li, C.; Tang, X.; and Loy, C. C. 2018. Pose-robust face recognition via deep residual equivariant mapping. CoRR abs/1803.00839.

[4]

Chen, B. C.; Chen, C. S.; and Hsu, W. H. 2015. Face recognition and retrieval using cross-age reference coding with cross-age celebrity dataset. IEEE T-MM 17(6):804–815.

Digital Library

[5]

Gong, D.; Li, Z.; Huang, W.; Li, X.; and Tao, D. 2017. Heterogeneous face recognition: A common encoding feature discriminant approach. IEEE TIP 26(5):2079–2089.

[6]

Guo, Y.; Zhang, L.; Hu, Y.; He, X.; and Gao, J. 2016. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. In ECCV, 87–102.

[7]

He, K.; Zhang, X.; Ren, S.; and Sun, J. 2015. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In ICCV, volume 00, 1026–1034.

[8]

He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep residual learning for image recognition. In CVPR.

[9]

He, R.; Wu, X.; Sun, Z.; and Tan, T. 2017. Wasserstein cnn: Learning invariant features for nir-vis face recognition. IEEE T-PAMI PP(99):1–1.

[10]

Hu, G.; Hua, Y.; Yuan, Y.; Zhang, Z.; Lu, Z.; Mukherjee, S. S.; Hospedales, T. M.; Robertson, N. M.; and Yang, Y. 2017. Attribute-enhanced face recognition with neural tensor fusion networks. In ICCV, 3764–3773.

[11]

Hu, G.; Peng, X.; Yang, Y.; Hospedales, T. M.; and Verbeek, J. 2018. Frankenstein: Learning deep face representations using small data. IEEE TIP 27(1):293–303.

[12]

Huang, G. B.; Ramesh, M.; Berg, T.; and Learned-Miller, E. 2007. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst.

[13]

Jia, Y.; Shelhamer, E.; Donahue, J.; Karayev, S.; Long, J.; Girshick, R.; Guadarrama, S.; and Darrell, T. 2014. Caffe: Convolutional architecture for fast feature embedding.

Digital Library

[14]

Kemelmacher-Shlizerman, I.; Seitz, S. M.; Miller, D.; and Brossard, E. 2016. The megaface benchmark: 1 million faces for recognition at scale. In CVPR, 4873–4882.

[15]

Klare, B. F., and Jain, A. K. 2013. Heterogeneous face recognition using kernel prototype similarities. IEEE T-PAMI 35(6):1410–1422.

Digital Library

[16]

Klare, B. F.; Li, Z.; and Jain, A. K. 2011. Matching forensic sketches to mug shot photos. IEEE T-PAMI 33(3):639–646.

Digital Library

[17]

Li, S. Z.; Yi, D.; Lei, Z.; and Liao, S. 2013. The CASIA NIRVIS 2.0 face database. In CVPR Workshops, 348–353.

[18]

Li, Z. F.; Gong, D.; Qiao, Y.; and Tao, D. 2014. Common feature discriminant analysis for matching infrared face images to optical face images. IEEE TIP 23(6):2436–2445.

[19]

Li, Z.; Gong, D.; Li, Q.; Tao, D.; and Li, X. 2016. Mutual component analysis for heterogeneous face recognition. ACM T-IST 7(3):28.

[20]

Liao, S.; Yi, D.; Lei, Z.; Qin, R.; and Li, S. Z. 2009. Heterogeneous face recognition from local structures of normalized appearance. In International Conference on Advances in Biometrics, 209–218.

[21]

Liu, Z.; Luo, P.; Wang, X.; and Tang, X. 2015. Deep learning face attributes in the wild. In ICCV, 3730–3738.

[22]

Liu, X.; Song, L.; Wu, X.; and Tan, T. 2016. Transferring deep representation for nir-vis heterogeneous face recognition. In International Conference on Biometrics (ICB), 1–8.

[23]

Parkhi, O. M.; Vedaldi, A.; and Zisserman, A. 2015. Deep face recognition. In BMVC, 41.1–41.12.

[24]

Peng, C.; Gao, X.; Wang, N.; and Li, J. 2016. Sparse graphical representation based discriminant analysis for heterogeneous face recognition. CoRR abs/1607.00137.

[25]

Peng, C.; Gao, X.; Wang, N.; and Li, J. 2017. Graphical representation for heterogeneous face recognition. IEEE T-PAMI 39(2):301–312.

Digital Library

[26]

Rebuffi, S. A.; Bilen, H.; and Vedaldi, A. 2018. Efficient parametrization of multi-domain deep neural networks.

[27]

Sarfraz, M. S., and Stiefelhagen, R. 2015. Deep perceptual mapping for thermal to visible face recognition. CoRR.

[28]

Saxena, S., and Verbeek, J. 2016. Heterogeneous face recognition with CNNs. In ECCV, 483–491.

[29]

Schroff, F.; Kalenichenko, D.; and Philbin, J. 2015. FaceNet: A unified embedding for face recognition and clustering. In CVPR, 815–823.

[30]

Shi, H.; Wang, X.; Yi, D.; Lei, Z.; Zhu, X.; and Li, S. Z. 2017. Cross-modality face recognition via heterogeneous joint bayesian. IEEE SPL 24(1):81–85.

[31]

Song, L.; Zhang, M.; Wu, X.; and He, R. 2017. Adversarial discriminative heterogeneous face recognition.

[32]

Sun, Y.; Wang, X.; and Tang, X. 2014a. Deep learning face representation by joint identification-verification. In NIPS.

[33]

Sun, Y.; Wang, X.; and Tang, X. 2014b. Deep learning face representation from predicting 10,000 classes. In CVPR.

[34]

Wen, Y.; Zhang, K.; Li, Z.; and Qiao, Y. 2016. A discriminative feature learning approach for deep face recognition. In ECCV, 499–515.

[35]

Wu, X.; He, R.; Sun, Z.; and Tan, T. 2015. A light cnn for deep face representation with noisy labels. Computer Science.

[36]

Wu, X.; Song, L.; He, R.; and Tan, T. 2017. Coupled deep learning for heterogeneous face recognition. CoRR.

[37]

Yi, D.; Lei, Z.; Liao, S.; and Li, S. Z. 2014. Learning face representation from scratch. Computer Science.

[38]

Zhang, X.; Zhang, L.; Wang, X. J.; and Shum, H. Y. 2012. Finding celebrities in billions of web images. IEEE T-MM 14(4):995–1007.

Digital Library

[39]

Zhang, K.; Zhang, Z.; Li, Z.; and Qiao, Y. 2016. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE SPL 23(10):1499–1503.

[40]

Zhang, W.; Wang, X.; and Tang, X. 2011. Coupled information-theoretic encoding for face photo-sketch recognition. In CVPR, 513–520.

Cited By

Lin Ffu KLuo HZhan ZWang ZLiu ZCavallaro LRen KEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Cross-Modal and Multi-Attribute Face Recognition: A BenchmarkProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612329(271-279)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612329
Miao YLattas ADeng JHan JZafeiriou SKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)Physically-based face rendering for NIR-VIS face recognitionProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3601923(22752-22764)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3601923

Index Terms

Residual compensation networks for heterogeneous face recognition
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Heterogeneous face recognition
Illumination compensation and normalization in eigenspace-based face recognition: A comparative study of different pre-processing approaches

The aim of this work is to investigate illumination compensation and normalization in eigenspace-based face recognition by carrying out an independent comparative study among several pre-processing algorithms. This research is motivated by the lack of ...
Age-Invariant Face Recognition

One of the challenges in automatic face recognition is to achieve temporal invariance. In other words, the goal is to come up with a representation and matching scheme that is robust to changes due to facial aging. Facial aging is a complex process that ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

AAAI'19/IAAI'19/EAAI'19: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence

January 2019

10088 pages

ISBN:978-1-57735-809-1

Copyright © 2019 Association for the Advancement of Artificial Intelligence.

Sponsors

Association for the Advancement of Artificial Intelligence

Publisher

AAAI Press

Publication History

Published: 27 January 2019

Qualifiers

Research-article
Research
Refereed limited

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
100
Total Downloads

Downloads (Last 12 months)88
Downloads (Last 6 weeks)18

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lin Ffu KLuo HZhan ZWang ZLiu ZCavallaro LRen KEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Cross-Modal and Multi-Attribute Face Recognition: A BenchmarkProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612329(271-279)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612329
Miao YLattas ADeng JHan JZafeiriou SKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)Physically-based face rendering for NIR-VIS face recognitionProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3601923(22752-22764)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3601923

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten