research-article

A collaborative representation based projections method for feature extraction

Authors:

Changyin SunAuthors Info & Claims

Pattern Recognition, Volume 48, Issue 1

Pages 20 - 27

https://doi.org/10.1016/j.patcog.2014.07.009

Published: 01 January 2015 Publication History

Abstract

In graph embedding based methods, we usually need to manually choose the nearest neighbors and then compute the edge weights using the nearest neighbors via L2 norm (e.g. LLE). It is difficult and unstable to manually choose the nearest neighbors in high dimensional space. So how to automatically construct a graph is very important. In this paper, first, we give a L2-graph like L1-graph. L2-graph calculates the edge weights using the total samples, avoiding manually choosing the nearest neighbors; second, a L2-graph based feature extraction method is presented, called collaborative representation based projections (CRP). Like SPP, CRP aims to preserve the collaborative representation based reconstruction relationship of data. CRP utilizes a L2 norm graph to characterize the local compactness information. CRP maximizes the ratio between the total separability information and the local compactness information to seek the optimal projection matrix. CRP is much faster than SPP since CRP calculates the objective function with L2 norm while SPP calculate the objective function with L1 norm. Experimental results on FERET, AR, Yale face databases and the PolyU finger-knuckle-print database demonstrate that CRP works well in feature extraction and leads to a good recognition performance. We give a L2 norm graph based on collaborative representation.We propose a collaborative representation based projections (CRP) for feature extraction.CRP is a Rayleigh quotient form and can be calculated via generalized eigenvalue decomposition.

References

[1]

J. Wright, Y.I. Ma, J. Mairal, G. Sapiro, T.S. Huang, Shuicheng Yan, Sparse representation for computer vision and pattern recognition, in: Proceedings of the IEEE, 98(6), 2010, pp. 1031-1104.

[2]

J. Yang, J. Wright, T. Huang, Y. Ma, Image super-resolution as sparse representation of raw patches, in: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR2008), 2008.

[3]

S. Rao, R. Tron, R. Vidal, Y. Ma, Motion segmentation via robust subspace separation in the presence of outlying, incomplete, and corrupted trajectories, in: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR2008), 2008.

[4]

J. Mairal, G. Sapiro, M. Elad, Learning multiscale sparse representations for image and video restoration, SIAM MMS, 7 (2008) 214-241.

[5]

J. Wright, A. Yang, S. Sastry, Y. Ma, Robust face recognition via sparse representation, IEEE Trans. Pattern Anal. Mach. Intell., 31 (2009) 210-227.

Digital Library

[6]

Y. Xu, D. Zhang, J. Yang, J.-Y. Yang, A two-phase test sample sparse representation method for use with face recognition, IEEE Trans. Circuits Syst. Video Technol., 21 (2011) 1255-1262.

[7]

R. He, B.-G. Hu, W.-S. Zheng, Y. Guo, Two-stage sparse representation for robust recognition on large-scale database, in: Proceeding of 24th AAAI Conference on Artificial Intelligence, 2010, pp. 475-480.

Digital Library

[8]

R. He, W.-S. Zheng, B.-G. Hu, X.W. Kong, A regularized correntropy framework for robust pattern recognition, Neural Comput., 23 (2011) 2074-2100.

Digital Library

[9]

R. He, W.-S. Zheng, B.-G. Hu, Maximum correntropy criterion for robust face recognition, IEEE Trans. Pattern Anal. Mach. Intell., 33 (2011) 1561-1576.

Digital Library

[10]

M. Yang, L. Zhang, J. Yang, D. Zhang, Robust sparse coding for face recognition, in: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR2011), 2011, pp. 625-632.

Digital Library

[11]

R.O. Duda, P.E. Hart, D.G. Stork., John Wiley & Sons, New York, 2000.

[12]

S.J. Raudys, A.K. Jain, Small sample size effects in statistical pattern recognition: recommendations for practitioners, IEEE Trans. Pattern Ana1. Mach. Intel1., 13 (1991) 252-264.

Digital Library

[13]

D.L. Swets, J. Weng, Using discriminant eigenfeatures for image retrieval, IEEE Trans Pattern Anal. Mach. Intell., 18 (1996) 831-836.

Digital Library

[14]

V. Belhumeur, J. Hespanha, D. Kriegman, Eigenfaces vs Fisherfaces: recognition using class specific linear projection, IEEE Trans. Pattern Ana1. Mach. Intel1., 19 (1997) 711-720.

Digital Library

[15]

J. Yang, J. Yang, Why can LDA be performed in PCA transformed space?, Pattern Recog., 36 (2003) 563-566.

[16]

Z. Hong, J. Yang, Optimal discriminant plane for a small number of samples and design method of classifier on the plane, Pattern Recog., 24 (1991) 317-324.

Digital Library

[17]

J. Yang, H. Li, PCA based Sequential Feature Space Learning for Gene Selection, ICMLC2010, pp. 3079-3084.

[18]

T. Hastie, R. Tibshirani, Penalized discriminant analysis, Ann. Stat., 23 (1995) 73-102.

[19]

L.F. Chen, H.Y.M. Liao, M.T. Ko, G.J. Yu., A new LDA-based face recognition system which can solve the small sample size problem, Pattern Recog., 33 (2000) 1713-1726.

[20]

H. Yu, J. Yang, A direct LDA algorithm for high dimensional data - with application to face recognition, Pattern Recog., 34 (2001) 2067-2070.

[21]

Y. Pang, D. Tao, Y. Yuan, X. Li, Binary two-dimensional PCA, IEEE Trans. Syst. Man, Cybern. B, Cybern., 38 (2008) 1176-1180.

Digital Library

[22]

X. Zhuang, D. Dai, Inverse fisher discriminant criteria for small sample size problem and its application to face recognition, Pattern Recog., 38 (2005) 2192-2194.

Digital Library

[23]

Z. Jin, J. Yang, Z. Hu, Z. Lou, Face recognition based on the uncorrelated discrimination transformation, Pattern Recog., 34 (2001) 1405-1416.

[24]

X. Li, S.S. Lin, D. Yan, Xu, Discriminant locally linear embedding with high-order tensor data, IEEE Trans. Syst. Man, Cybern. B, Cybern., 38 (2008) 342-352.

Digital Library

[25]

M. Loog, Approximate pairwise accuracy criterion for multiclass linear dimension reduction: generalization of the Fisher criterion, IEEE Trans. Pattern Anal. Mach. Intell., 26 (2001) 762-766.

Digital Library

[26]

R. Lotlikar, R. Kothari, Fractional-step dimensionality reduction, IEEE Trans. Pattern Recog. Mach. Intell., 22 (2000) 623-627.

Digital Library

[27]

J. Lu, K.N. Plataniotic, A.N. Venetsanopouls, Face recognition using LDA-based algorithm, IEEE Trans. Neural Netw., 14 (2003) 195-200.

Digital Library

[28]

D. Tao, X. Li, X. Wu, S.J. Maybank, Geometric mean for subspace selection, IEEE Trans. Pattern Anal. Mach. Intell., 31 (2009) 260-274.

Digital Library

[29]

Y. Xu, D. Zhang, J. Yang, J.-Y. Yang, A two-phase test sample sparse representation method for use with face recognition, IEEE Trans. Circuits Syst. Video Technol., 21 (2011) 1255-1262.

[30]

W. Yang, J. Wang, M. Ren, J. Yang, Feature extraction based on laplacian bidirectional maximum margin criterion, Pattern Recog., 42 (2009) 2327-2334.

Digital Library

[31]

C. Ren, D. Dai, Incremental learning of bidirectional principal components for face recognition, Pattern Recog., 43 (2010) 318-330.

Digital Library

[32]

X.D. Jiang, Asymmetric principal component and discriminant analyses for pattern classification, IEEE Trans. Pattern Anal. Mach. Intell., 31 (2009) 931-937.

Digital Library

[33]

X.D. Jiang, B. Mandal, A. Kot, Eigenfeature regularization and extraction in face recognition, IEEE Trans. Pattern Anal. Mach. Intell., 30 (2008) 383-394.

Digital Library

[34]

X.G. Wang, X.O. Tang, A unified framework for subspace face recognition, IEEE Trans. Pattern Anal. Mach. Intell., 26 (2004) 1222-1228.

Digital Library

[35]

J.B. Tenenbaum, V.d.e. Silva, J.C. Langford, A global geometric framework for nonlinear dimensionality reduction, Science, 290 (2000) 2319-2323.

[36]

S.T. Roweis, L.K. Saul, Nonlinear dimension reduction by locally linear embedding, Science, 290 (2000) 2323-2326.

[37]

M. Belkin, P. Niyogi, Laplacian eigenmaps for dimensionality reduction and data representation, Neural Comput., 15 (2003) 1373-1396.

Digital Library

[38]

X. He, S. Yan, Y. Hu, P. Niyogi, H. Zhang, Face recognition using laplacianfaces, IEEE Trans. Pattern Ana1. Mach. Intel1., 27 (2005) 328-340.

Digital Library

[39]

J. Yang, D. Zhang, J. Yang, B. Niu, Globally maximizing, locally minimizing: unsupervised discriminant projection with applications to face and palm biometrics, IEEE Trans. Pattern Ana1. Mach. Intel1., 29 (2007) 650-664.

Digital Library

[40]

Y. Xu, A. Zhong, J. Yang, D. Zhang, LPP solution schemes for use with face recognition, Pattern Recog., 43 (2010) 4165-4176.

Digital Library

[41]

S. Yan, D. Xu, B. Zhang, H. Zhang, Graph embedding and extensions: a general framework for dimensionality reduction, IEEE Trans. Pattern Ana1. Mach. Intell., 29 (2007) 40-51.

Digital Library

[42]

H. Wang, S. Che, Z. Hu, W. Zheng, Locaility-preserved maximum information projection, IEEE Trans. Neural Netw., 19 (2008) 571-585.

Digital Library

[43]

L. Qiao, S. Chen, X. Tan, Sparsity preserving projections with applications to face recognition, Pattern Recog., 43 (2010) 331-341.

Digital Library

[44]

B. Cheng, J. Yang, S. Yan, Y. Fu, T. Huang, Learning with L1-graph for image analysis, IEEE Trans. Image Process., 19 (2010) 858-866.

Digital Library

[45]

L.M. Zhang, S. Chen, L. Qiao, Graph optimization for dimensionality reduction with sparsity constraints, Pattern Recog., 45 (2012) 1205-1210.

Digital Library

[46]

J. Yang, D. Chu, L. Zhang, Y. Xu, J. Yang, Sparse representation classifier steered discriminative projection with application to face recognition, IEEE Trans. Neural Netw. Learn. Syst., 24 (2013) 1023-1034.

[47]

L. Clemmensen, T. Hastie, D. Witten, b. Ersboll, Sparse discriminant analysis, Technometrics, 53 (2011) 406-413.

[48]

Z. Lai, M. Wan, Z. Jin, J. Yang, Sparse two-dimensional local discriminant projections for feature extraction, Neurocomputing, 74 (2011) 629-637.

Digital Library

[49]

Z. Lai, W.K. Wong, Z. Jin, J. Yang, Y. Xu, Sparse approximation to the eigen subspace for discrimination, IEEE Trans. Neural Netw. Learn. Syst., 23 (2012) 1948-1960.

[50]

L. Zhang, M. Yang, X. Feng, Sparse representation or collaborative representation: which helps face recognition, in: Proceedings of IEEE Conference on Computer Vision, 2011 (ICCV2011).

Digital Library

[51]

X. He, D. Cai, S. Yan, and H. Zhang, Neighborhood preserving embedding, in Proceedings of IEEE Conference on Computer Vision (ICCV2005), 2005, pp. 1208-1213.

Digital Library

[52]

D. Zhao, Z. Lin, and X. Tang. Laplacian PCA and its applications, in: Proceedings of IEEE Conference on Computer Vision (ICCV2007), 2007.

[53]

P.J. Phillips, H. Moon, S.A. Rizvi, P.J. Rauss, The FERET Evaluation Methodology for Face Recognition Algorithms, IEEE Trans. Pattern Ana1. Mach. Intel1., 22 (2000) 1090-1104.

Digital Library

[54]

P.J. Phillips, The facial recognition technology (FERET) database. (2004).

[55]

A.M. Martinez, R. Benavente, The AR Face Database {http://cobweb.ecn.purdue.edu/~aleix/ aleix_face_DB.html}.

[56]

A.M. Martinez, R. Benavente, The AR Face Database, June 1998.

[57]

L. Zhang, L. Zhang, D. Zhang, H. Zhu, Online finger-knuckle-print verification for personal authentication, Pattern Recog., 43 (2010) 2560-2571.

Digital Library

[58]

The Hong Kong PolyU FKP database: {http://www4.comp.polyu.edu.hk/~biometrics/FKP.htm}.

Cited By

Gou JYuan XXue YDu LYu JXia SZhang Y(2023)Discriminative and Geometry-Preserving Adaptive Graph Embedding for dimensionality reductionNeural Networks10.1016/j.neunet.2022.10.024157:C(364-376)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.neunet.2022.10.024
Su TFeng DHu HWang MChen M(2023)Locality preserving triplet discriminative projections for dimensionality reductionNeurocomputing10.1016/j.neucom.2022.11.043520:C(284-300)Online publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1016/j.neucom.2022.11.043
Zhao XNie FYu WLi X(2023)Fast neighborhood reconstruction with adaptive weights learningKnowledge-Based Systems10.1016/j.knosys.2022.110082259:COnline publication date: 10-Jan-2023
https://dl.acm.org/doi/10.1016/j.knosys.2022.110082
Show More Cited By

A collaborative representation based projections method for feature extraction
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
  2. Machine learning
    1. Machine learning algorithms

Recommendations

A regularized least square based discriminative projections for feature extraction

In this paper, we present a regularized least square based discriminative projections (RLSDP) method for feature extraction. First, we show that both sparse representation based classifier (SRC) and collaborative representation based classification (CRC)...
Enhanced regularized least square based discriminative projections for feature extraction

The regularized least square based discriminative projections (RLSDP) for extracting features was recently proposed, which aims to seek discriminant projection directions that maximize the between-class scatter and minimize the within-class compactness. ...
Collaborative representation based local discriminant projection for feature extraction
Abstract
This paper introduces a novel dimensionality reduction algorithm, called collaborative representation based local discriminant projection (CRLDP), for feature extraction. CRLDP utilizes collaborative representation relationships among samples to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Pattern Recognition

Pattern Recognition Volume 48, Issue 1

January 2015

279 pages

ISSN:0031-3203

Issue’s Table of Contents

Copyright © Elsevier Ltd.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 January 2015

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

57
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gou JYuan XXue YDu LYu JXia SZhang Y(2023)Discriminative and Geometry-Preserving Adaptive Graph Embedding for dimensionality reductionNeural Networks10.1016/j.neunet.2022.10.024157:C(364-376)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.neunet.2022.10.024
Su TFeng DHu HWang MChen M(2023)Locality preserving triplet discriminative projections for dimensionality reductionNeurocomputing10.1016/j.neucom.2022.11.043520:C(284-300)Online publication date: 1-Feb-2023
https://dl.acm.org/doi/10.1016/j.neucom.2022.11.043
Zhao XNie FYu WLi X(2023)Fast neighborhood reconstruction with adaptive weights learningKnowledge-Based Systems10.1016/j.knosys.2022.110082259:COnline publication date: 10-Jan-2023
https://dl.acm.org/doi/10.1016/j.knosys.2022.110082
Huang ZZhao SLiang ZWu J(2023)Robust Subspace Learning with Double Graph EmbeddingPattern Recognition and Computer Vision10.1007/978-981-99-8540-1_11(126-137)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1007/978-981-99-8540-1_11
Qiu HWang RSun DLiu XZhang LLiu Y(2022)A Smoothed Matrix Multivariate Elliptical Distribution-Based Projection Method for Feature ExtractionComputational Intelligence and Neuroscience10.1155/2022/25511372022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/2551137
Lu XLong JWen JFei LZhang BXu Y(2022)Locality preserving projection with symmetric graph embedding for unsupervised dimensionality reductionPattern Recognition10.1016/j.patcog.2022.108844131:COnline publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1016/j.patcog.2022.108844
Zhang HZhao SQiang WChen YJing L(2022)Feature extraction framework based on contrastive learning with adaptive positive and negative samplesNeural Networks10.1016/j.neunet.2022.09.029156:C(244-257)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1016/j.neunet.2022.09.029
Gou JHe XLu JMa HOu WYuan Y(2022)A class-specific mean vector-based weighted competitive and collaborative representation method for classificationNeural Networks10.1016/j.neunet.2022.02.021150:C(12-27)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1016/j.neunet.2022.02.021
An JZhao XShi MLiu XGuo J(2022)Joint neighborhood preserving and projected clustering for feature extractionNeurocomputing10.1016/j.neucom.2021.11.065488:C(572-580)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1016/j.neucom.2021.11.065
Zhang HQiang WZhang JChen YJing L(2022)Unified feature extraction framework based on contrastive learningKnowledge-Based Systems10.1016/j.knosys.2022.110028258:COnline publication date: 22-Dec-2022
https://dl.acm.org/doi/10.1016/j.knosys.2022.110028
Show More Cited By

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents