research-article

Multi-resolution feature fusion for face recognition

Authors:

Kuong-Hon Pong,

Kin-Man LamAuthors Info & Claims

Pattern Recognition, Volume 47, Issue 2

Pages 556 - 567

https://doi.org/10.1016/j.patcog.2013.08.023

Published: 01 February 2014 Publication History

Abstract

For face recognition, image features are first extracted and then matched to those features in a gallery set. The amount of information and the effectiveness of the features used will determine the recognition performance. In this paper, we propose a novel face recognition approach using information about face images at higher and lower resolutions so as to enhance the information content of the features that are extracted and combined at different resolutions. As the features from different resolutions should closely correlate with each other, we employ the cascaded generalized canonical correlation analysis (GCCA) to fuse the information to form a single feature vector for face recognition. To improve the performance and efficiency, we also employ "Gabor-feature hallucination", which predicts the high-resolution (HR) Gabor features from the Gabor features of a face image directly by local linear regression. We also extend the algorithm to low-resolution (LR) face recognition, in which the medium-resolution (MR) and HR Gabor features of a LR input image are estimated directly. The LR Gabor features and the predicted MR and HR Gabor features are then fused using GCCA for LR face recognition. Our algorithm can avoid having to perform the interpolation/super-resolution of face images and having to extract HR Gabor features. Experimental results show that the proposed methods have a superior recognition rate and are more efficient than traditional methods. A face recognition approach which combines images at different resolutions is proposed.A low-resolution face recognition algorithm based on fusing images at different resolutions is proposed.A method for feature hallucination is proposed.

References

[1]

Q.S. Sun, S.G. Zeng, Y. Liu, P.A. Heng, D.S. Xia, A new method of feature fusion and its application in image recognition, Pattern Recognition, 38 (2005) 2437-2448.

Digital Library

[2]

C.J. Liu, H. Wechsler, A shape and texture based enhanced fisher classifier for face recognition, IEEE Transactions on Image Processing, 10 (2001) 598-608.

Digital Library

[3]

J. Yang, J.-Y. Yang, Generalized KL transform based combined feature extraction, Pattern Recognition, 35 (2002) 295-297.

[4]

J. Yang, J.Y. Yang, D. Zhang, J.F. Lu, Feature fusion: parallel strategy vs. serial strategy, Pattern Recognition, 36 (2003) 1369-1381.

[5]

Z. Liu, C. Liu, Fusion of color, local spatial and global frequency information for face recognition, Pattern Recognition, 43 (2010) 2882-2890.

Digital Library

[6]

Y. Yuan, Q. Sun, Q. Zhou, D. Xia, A novel multiset integrated canonical correlation analysis framework and its application in feature fusion, Pattern Recognition, 44 (2011) 1031-1040.

Digital Library

[7]

G. Feng, K. Dong, D. Hu, D. Zhang, When faces are combined with palmprints: a novel biometric fusion strategy, in: International Conference on Biometric Authentication, 2004, pp. 701-707.

[8]

A.A. Ross, R. Govindarajan, Feature level fusion of hand and face biometrics, in: Proceedings of SPIE Conference on Biometric Technology for Human Identification II 5779, 2005, pp. 196-204.

[9]

Y. Gao, M. Maggs, Feature-level fusion in personal identification, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 1, 2005, pp. 468-473.

[10]

A. Kong, D. Zhang, M. Kamel, Palmprint identification using feature level fusion, Pattern Recognition, 39 (2006) 478-487.

Digital Library

[11]

Q. Li, Z. Qiu, Handmetric verification based on feature-level fusion, International Journal of Computer Science and Network Security, 6 (2006) 164-168.

[12]

Z.Q. Wang, X.H. Sun, L. Guo, J.Y. Yang, Multifeature fusion based on Fisher discriminant criterion, Computer Engineering, 28 (2002) 41-42.

[13]

Arun Ross, Anil Jain, Information fusion in biometrics, Pattern Recognition Letters, 24 (2003) 2115-2125.

Digital Library

[14]

P.N. Belhumeur, J.P. Hespanha, D.J. Kriegman, Eigenfaces vs. fisherfaces: recognition using class specific linear projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 19 (1997) 711-720.

Digital Library

[15]

H. Ekenel, B Sankur, Multiresolution face recognition, Image and Vision Computing, 23 (2005) 469-477.

Digital Library

[16]

Y.Gao, Y. Wang, X. Feng, X. Zhou, Face recognition using most discriminative local and global features, in: 18th International Conference on Pattern Recognition, 2006, pp. 351-354.

[17]

S. Arivazhagan, J. Mumtaj, L.Ganesan, Face recognition using multi-resolution transform, in: International Conference on Computational Intelligence and Multimedia Applications, 2007, pp. 301-306.

[18]

L. Wang, Y. Li, C. Wang, H. Zhang, 2D Gaborface representation method for face recognition with ensemble and multichannel model, Image and Vision Computing, 26 (2008) 820-828.

Digital Library

[19]

Y. Xu, Z. Li, J. Pan, J Yang, Face recognition based on fusion of multi-resolution Gabor features, Neural Computing & Applications (July 2012).

[20]

D.S. Huang, X.P. Zhang, G.B. Huang, Face recognition based on generalized canonical correlation analysis, advances in intelligent computing, Lecture Notes in Computer Science, 3645 (2005) 958-967.

Digital Library

[21]

Q.S. Sun, Z.D. Liu, P.A. Heng, D.S. Xia, A theorem on the generalized canonical projective vectors, Pattern Recognition, 38 (2005) 449-452.

Digital Library

[22]

Q.S. Sun, M.L. Yang, P.A. Heng, D.S. Xia, Improvements on CCA model with application to face recognition, in: Proceedings of International Conference on Intelligent Information Processing, 2004, pp. 125-134.

[23]

Y. Fu, L. Cao, G. Guo, T.S. Huang, Multiple feature fusion by subspace learning, in: ACM International Conference on Image and Video Retrieval (ACM CIVR), 2008, pp. 127-134.

[24]

T. Sun, S. Chen, Locality preserving CCA with applications to data visualization and pose estimation, Image and Vision Computing, 25 (2007) 531-543.

Digital Library

[25]

J.R. Kettenring, Canonical analysis of several sets of variables, Biometrika, 58 (1971) 433-451.

[26]

A.A. Nielsen, Multiset canonical correlations analysis and multispectral, truly multitemporal remote sensing data, IEEE Transactions on Image Processing, 11 (2002) 293-305.

Digital Library

[27]

B. Thompson, J. Cartmill, M.R. Azimi-Sadjadi, S.G. Schock, A multichannel canonical correlation analysis feature extraction with application to buried underwater target classification, in: International Joint Conference on Neural Networks, 2006, pp. 4413-4420.

[28]

K.H. Pong, K.M. Lam, Gabor-feature hallucination based on generalized canonical correlation analysis for face recognition, in: International Symposium on Intelligent Signal Processing and Communications Systems (ISPACS), 2011, pp. 1-6.

[29]

T.S. Lee, Image representation using 2D Gabor wavelets, IEEE Transactions on Pattern Analysis Machine Intelligence, 18 (1996) 97-959.

Digital Library

[30]

J.K. Kamarainen, V. Kyrki, H. Kalviainen, Invariance properties of Gabor filter-based features-overview and applications, IEEE Transactions on Image Processing, 15 (2006) 1088-1099.

Digital Library

[31]

J.G. Daugman, Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters, Journal of the Optical Society of America A, 2 (1985) 1160-1169.

[32]

J.G. Daugman, Two-dimensional spectral analysis of cortical receptive field profiles, Vision Research, 20 (1980) 847-856.

[33]

C. Liu, H. Wechsler, Independent component analysis of Gabor features for face recognition, IEEE Transactions on Neural Networks, 14 (2003) 919-928.

Digital Library

[34]

D. Liu, K.M. Lam, L.S. Shen, Optimal sampling of Gabor features for face recognition, Pattern Recognition Letters, 25 (2004) 267-276.

Digital Library

[35]

Wiskott, J.M. Fellous, N. Krüger, C. Malsburg, Face recognition by elastic bunch graph matching, IEEE Transactions on Pattern Analysis Machine Intelligence, 19 (1997) 775-779.

Digital Library

[36]

C. Liu, Gabor-based kernel PCA with fractional power polynomial models for face recognition, IEEE Transactions on Pattern Analysis Machine Intelligence, 26 (2004) 572-781.

Digital Library

[37]

C. Liu, Capitalize on dimensionality increasing techniques for improving face recognition grand challenge performance, IEEE Transactions on Pattern Analysis Machine Intelligence, 28 (2006) 725-727.

Digital Library

[38]

X. Xie, K.M. Lam, Gabor-based kernel PCA with doubly nonlinear mapping for face recognition with a single face image, IEEE Transactions on Image Processing, 15 (2006) 2481-2492.

Digital Library

[39]

L. Shen, L. Bai, M. Fairhurst, Gabor wavelets and general discriminant analysis for face identification and verification, Image Vision Computing, 25 (2007) 553-563.

Digital Library

[40]

H. Hotelling, Relations between two sets of variates, Biometrika, 28 (1936) 321-377.

[41]

M. Borga, Canonical Correlation Tutorial. Available from {http://people.imt.liu.se/~magnus/cca}, 2001.

[42]

M. Turk, A. Pentland, Eigenfaces for recognition, Journal of Cognitive Neuroscience, 3 (1991) 71-86.

Digital Library

[43]

P.S. Penev, L. Sirovich, The global dimensionality of face space, in: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, 2000, pp. 264-270.

[44]

X. Chai, S. Shan, X. Chen, W. Gao, Local Linear Regression (LLR) for pose invariant face recognition, in: Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition, 2006, pp. 631-636.

[45]

Ben-Israel Adi, Thomas N.E. Greville, Generalized Inverses Theory and Applications, Springer-Verlag, 2003.

[46]

X. He, S. Yan, Y. Hu, P. Niyogi, H. Zhang, Face recognition using Laplacianfaces, IEEE Transactions on Pattern Analysis Machine Intelligence, 27 (2005) 328-340.

Digital Library

[47]

F. Samaria, A.C. Harter, Parameterisation of a stochastic model for human face identification, in: Proceedings of the Second IEEE Workshop on Applications of Computer Vision, 1994, pp. 138-142.

[48]

P. Philips, H. Moon, P. Pauss, S. Rivzvi, The FERET evaluation methodology for face-recognition algorithms, in: Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition, 1997, pp. 137-143.

Cited By

Ma FSun BLi S(2023)Facial Expression Recognition With Visual Transformers and Attentional Selective FusionIEEE Transactions on Affective Computing10.1109/TAFFC.2021.312214614:2(1236-1248)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TAFFC.2021.3122146
Mliki HFendri EChebil A(2022)Face Recognition Through Multi-Resolution ImagesInternational Journal of Software Innovation10.4018/IJSI.29201810:1(1-15)Online publication date: 4-Mar-2022
https://dl.acm.org/doi/10.4018/IJSI.292018
Haghighat MAbdel-Mottaleb MAlhalabi W(2021)Discriminant correlation analysis for feature level fusion with application to multimodal biometrics2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP.2016.7472000(1866-1870)Online publication date: 11-Mar-2021
https://dl.acm.org/doi/10.1109/ICASSP.2016.7472000
Show More Cited By

Multi-resolution feature fusion for face recognition
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems

Recommendations

Low-resolution face recognition based on feature-mapping face hallucination
Abstract
The image recognition approaches based on Convolutional Neural Network (CNN) have already achieved tremendous performance on super-resolution face images. However, there exist several challenges in face recognition field. For example, ...
Large margin coupled mapping for low resolution face recognition
PRICAI'16: Proceedings of the 14th Pacific Rim International Conference on Trends in Artificial Intelligence

Traditional face recognition algorithms can achieve significant performance under well-controlled environments. However, these algorithms perform poorly when the resolution of the face images varies. A two-step framework is proposed to solve the ...
A robust face super-resolution algorithm and its application in low-resolution face recognition system
Abstract
In real-world surveillance scenario, the face recognition (FR) systems pose a lot of challenges due to the captured low-resolution (LR) and noisy probe images. A new face super-resolution (SR) algorithm is proposed to design a recognition model ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

Copyright © Elsevier Ltd.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 February 2014

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ma FSun BLi S(2023)Facial Expression Recognition With Visual Transformers and Attentional Selective FusionIEEE Transactions on Affective Computing10.1109/TAFFC.2021.312214614:2(1236-1248)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TAFFC.2021.3122146
Mliki HFendri EChebil A(2022)Face Recognition Through Multi-Resolution ImagesInternational Journal of Software Innovation10.4018/IJSI.29201810:1(1-15)Online publication date: 4-Mar-2022
https://dl.acm.org/doi/10.4018/IJSI.292018
Haghighat MAbdel-Mottaleb MAlhalabi W(2021)Discriminant correlation analysis for feature level fusion with application to multimodal biometrics2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP.2016.7472000(1866-1870)Online publication date: 11-Mar-2021
https://dl.acm.org/doi/10.1109/ICASSP.2016.7472000
Xi XJiang WLü ZMiran SLuo ZGomez-Pilar J(2020)Daily Activity Monitoring and Fall Detection Based on Surface Electromyography and Plantar PressureComplexity10.1155/2020/95320672020Online publication date: 8-Jan-2020
https://dl.acm.org/doi/10.1155/2020/9532067
Ge SZhao SLi CLi J(2019)Low-Resolution Face Recognition in the Wild via Selective Knowledge DistillationIEEE Transactions on Image Processing10.1109/TIP.2018.288374328:4(2051-2062)Online publication date: 1-Apr-2019
https://dl.acm.org/doi/10.1109/TIP.2018.2883743
Zhou HLam K(2018)Age-invariant face recognition based on identity inference from appearance agePattern Recognition10.1016/j.patcog.2017.10.03676:C(191-202)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.1016/j.patcog.2017.10.036
Hao TWang QWu DSun J(2018)Adaptive recommendation for photo pose via deep learningMultimedia Tools and Applications10.1007/s11042-018-5705-277:17(22173-22184)Online publication date: 1-Sep-2018
https://dl.acm.org/doi/10.1007/s11042-018-5705-2
Lin GFan CZhu HMiu YKang X(2017)Visual feature coding based on heterogeneous structure fusion for image classificationInformation Fusion10.1016/j.inffus.2016.12.01036:C(275-283)Online publication date: 1-Jul-2017
https://dl.acm.org/doi/10.1016/j.inffus.2016.12.010
Alsalibi BVenkat IAl-Betar M(2017)A membrane-inspired bat algorithm to recognize faces in unconstrained scenariosEngineering Applications of Artificial Intelligence10.1016/j.engappai.2017.06.01864:C(242-260)Online publication date: 1-Sep-2017
https://dl.acm.org/doi/10.1016/j.engappai.2017.06.018
Chaudhry SChandra R(2017)Face detection and recognition in an unconstrained environment for mobile visual assistive systemApplied Soft Computing10.1016/j.asoc.2016.12.03553:C(168-180)Online publication date: 1-Apr-2017
https://dl.acm.org/doi/10.1016/j.asoc.2016.12.035
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents