Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Robust Dimensionality Reduction via Low-rank Laplacian Graph Learning

Published: 01 April 2023 Publication History

Abstract

Manifold learning is a widely used technique for dimensionality reduction as it can reveal the intrinsic geometric structure of data. However, its performance decreases drastically when data samples are contaminated by heavy noise or occlusions, which leads to unsatisfying data processing performance. We propose a novel robust dimensionality reduction method via low-rank Laplacian graph learning for classification and clustering tasks to solve the above problem. First, we construct a low-rank Laplacian graph by combining manifold learning and subspace learning. This graph can capture both global and local structural information of the data. And we introduce rank constraints for the Laplacian graph to make it more discriminative. Second, we put the learning of projection matrix and sample affinity graph into a unified framework. The projection matrix is embedded into a robust low-rank Laplacian graph so that the low-dimensional mapping of data can maintain the structural information in the graph well. Finally, we add a regularization term to the projection matrix to make it have the ability of both feature extraction and feature selection. Therefore, the proposed model can resist the interference of noise or data damage to learn the optimal projection to achieve better performance in dimensionality reduction through such a data dimensionality reduction joint framework. Comprehensive experiments on various benchmark datasets with varying degrees of occlusions or corruptions are carried out to evaluate the performance of the proposed method. Compared with the state-of-the-art dimensionality reduction methods in the literature, the experimental results are inspiring, showing our method’s effectiveness and robustness in classification and clustering, especially in object recognition scenarios with noise or occlusions.

References

[1]
Peter N. Belhumeur, Joao P. Hespanha, and David J. Kriegman. 1997. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19, 7 (1997), 711–720.
[2]
Mikhail Belkin and Partha Niyogi. 2003. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15, 6 (2003), 1373–1396.
[3]
Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1 (2011), 1–122.
[4]
Jian-Feng Cai, Emmanuel J. Candès, and Zuowei Shen. 2010. A singular value thresholding algorithm for matrix completion. SIAM J. Optimiz. 20, 4 (2010), 1956–1982.
[5]
Emmanuel J. Candès, Xiaodong Li, Yi Ma, and John Wright. 2011. Robust principal component analysis?J. ACM 58, 3 (2011), 1–37.
[6]
Bin Cheng, Jianchao Yang, Shuicheng Yan, Yun Fu, and Thomas S. Huang. 2009. Learning with \(\ell ^{1}\) -graph for image analysis. IEEE Trans. Image Process. 19, 4 (2009), 858–866.
[7]
Kresimir Delac, Mislav Grgic, and Sonja Grgic. 2005. Independent comparative study of PCA, ICA, and LDA on the FERET data set. Int. J. Imag. Syst. Technol. 15, 5 (2005), 252–260.
[8]
Chuntao Ding and Li Zhang. 2015. Double adjacency graphs-based discriminant neighborhood embedding. Pattern Recogn. 48, 5 (2015), 1734–1742.
[9]
Ky Fan. 1949. On a theorem of Weyl concerning eigenvalues of linear transformations I. Proc. Natl. Acad. Sci. U.S.A. 35, 11 (1949), 652.
[10]
Yuqiang Fang, Ruili Wang, Bin Dai, and Xindong Wu. 2014. Graph-based learning via auto-grouped sparse regularization and kernelized extension. IEEE Trans. Knowl. Data Eng. 27, 1 (2014), 142–154.
[11]
Li Fei-Fei, Rob Fergus, and Pietro Perona. 2006. One-shot learning of object categories. IEEE Trans. Pattern Anal. Mach. Intell. 28, 4 (2006), 594–611.
[12]
Quanxue Gao, Qianqian Wang, Yunfang Huang, Xinbo Gao, Xin Hong, and Hailin Zhang. 2015. Dimensionality reduction by integrating sparse representation and Fisher criterion and its applications. IEEE Trans. Image Process. 24, 12 (2015), 5684–5695.
[13]
Jianping Gou, Ya Xue, Hongxing Ma, Yong Liu, Yongzhao Zhan, and Jia Ke. 2020. Double graphs-based discriminant projections for dimensionality reduction. Neural Comput. Appl. 32, 23 (2020), 17533–17550.
[14]
Palash Goyal and Emilio Ferrara. 2018. Graph embedding techniques, applications, and performance: A survey. Knowl.-Based Syst. 151 (2018), 78–94.
[15]
Rassoul Hajizadeh, Ali Aghagolzadeh, and Mehdi Ezoji. 2020. Local distances preserving based manifold learning. Expert Syst. Appl. 139 (2020), 112860.
[16]
Xiaofei He, Deng Cai, Shuicheng Yan, and Hong-Jiang Zhang. 2005. Neighborhood preserving embedding. In Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV’05). IEEE, 1208–1213.
[17]
Xiaofei He and Partha Niyogi. 2004. Locality preserving projections. Adv. Neural Info. Process. Syst. 16, 16 (2004), 153–160.
[18]
Xiaofei He, Shuicheng Yan, Yuxiao Hu, Partha Niyogi, and Hong-Jiang Zhang. 2005. Face recognition using Laplacianfaces. IEEE Trans. Pattern Anal. Mach. Intell. 27, 3 (2005), 328–340.
[19]
Rongyao Hu, Xiaofeng Zhu, Debo Cheng, Wei He, Yan Yan, Jingkuan Song, and Shichao Zhang. 2017. Graph self-representation method for unsupervised feature selection. Neurocomputing 220 (2017), 130–137.
[20]
Anil K. Jain, Robert P. W. Duin, and Jianchang Mao. 2000. Statistical pattern recognition: A review. IEEE Trans. Pattern Anal. Mach. Intell. 22, 1 (2000), 4–37.
[21]
Nojun Kwak. 2008. Principal component analysis based on L1-norm maximization. IEEE Trans. Pattern Anal. Mach. Intell. 30, 9 (2008), 1672–1680.
[22]
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278–2324.
[23]
Xuelong Li, Han Zhang, Rui Zhang, Yun Liu, and Feiping Nie. 2018. Generalized uncorrelated regression with adaptive graph for unsupervised feature selection. IEEE Trans. Neural Netw. Learn. Syst. 30, 5 (2018), 1587–1595.
[24]
Zhouchen Lin, Minming Chen, and Yi Ma. 2010. The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. Retrieved from https://arXiv:1009.5055.
[25]
Guangcan Liu, Zhouchen Lin, Shuicheng Yan, Ju Sun, Yong Yu, and Yi Ma. 2012. Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1 (2012), 171–184.
[26]
Guangcan Liu, Zhouchen Lin, Yong Yu, et al. 2010. Robust subspace segmentation by low-rank representation. In Proceedings of the International Conference on Machine Learning (ICML’10), Vol. 1. Citeseer, 8.
[27]
Guangcan Liu and Shuicheng Yan. 2011. Latent low-rank representation for subspace segmentation and feature extraction. In Proceedings of the International Conference on Computer Vision. IEEE, 1615–1622.
[28]
Junmin Liu, Yijun Chen, Jiangshe Zhang, and Zongben Xu. 2014. Enhancing low-rank subspace clustering by manifold regularization. IEEE Trans. Image Process. 23, 9 (2014), 4022–4030.
[29]
Jianglin Lu, Hailing Wang, Jie Zhou, Yudong Chen, Zhihui Lai, and Qinghua Hu. 2021. Low-rank adaptive graph embedding for unsupervised feature extraction. Pattern Recogn. 113 (2021), 107758.
[30]
Yuwu Lu, Zhihui Lai, Yong Xu, Xuelong Li, David Zhang, and Chun Yuan. 2015. Low-rank preserving projections. IEEE Trans. Cybernet. 46, 8 (2015), 1900–1913.
[31]
Yuwu Lu, Zhihui Lai, Yong Xu, Jane You, Xuelong Li, and Chun Yuan. 2016. Projective robust nonnegative factorization. Info. Sci. 364 (2016), 16–32.
[32]
Aleix Martinez and Robert Benavente. 1998. The ar face database: Cvc technical report, 24.
[33]
Feiping Nie, Xiaoqian Wang, and Heng Huang. 2014. Clustering and projected clustering with adaptive neighbors. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 977–986.
[34]
Feiping Nie, Dong Xu, Ivor Wai-Hung Tsang, and Changshui Zhang. 2010. Flexible manifold embedding: A framework for semi-supervised and unsupervised dimension reduction. IEEE Trans. Image Process. 19, 7 (2010), 1921–1932.
[35]
Feiping Nie, Wei Zhu, and Xuelong Li. 2016. Unsupervised feature selection with structured graph optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30.
[36]
Lishan Qiao, Songcan Chen, and Xiaoyang Tan. 2010. Sparsity preserving projections with applications to face recognition. Pattern Recogn. 43, 1 (2010), 331–341.
[37]
Sameer A. Nene, Shree K. Nayar, and Hiroshi Murase and others. 1996. Columbia object image library (coil-20), Citeseer.
[38]
Zhenwen Ren, Quansen Sun, Bin Wu, Xiaoqian Zhang, and Wenzhu Yan. 2019. Learning latent low-rank and sparse embedding for robust image feature extraction. IEEE Trans. Image Process. 29 (2019), 2094–2107.
[39]
Sam T. Roweis and Lawrence K. Saul. 2000. Nonlinear dimensionality reduction by locally linear embedding. Science 290, 5500 (2000), 2323–2326.
[40]
Ferdinando S. Samaria and Andy C. Harter. 1994. Parameterisation of a stochastic model for human face identification. In Proceedings of the IEEE Workshop on Applications of Computer Vision. IEEE, 138–142.
[41]
Saúl Solorio-Fernández, J. Ariel Carrasco-Ochoa, and José Fco. Martínez-Trinidad. 2020. A review of unsupervised feature selection methods. Artific. Intell. Rev. 53, 2 (2020), 907–948.
[42]
Xin Song, Heng-Chao Li, Lei Pan, Yang-Jun Deng, Pu Zhang, Li You, and Qian Du. 2021. Unsupervised robust projection learning by low-rank and sparse decomposition for hyperspectral feature extraction. IEEE Geosci. Remote Sens. Lett. 19 (2021), 1–5.
[43]
Chang Tang, Xinwang Liu, Xinzhong Zhu, Jian Xiong, Miaomiao Li, Jingyuan Xia, Xiangke Wang, and Lizhe Wang. 2019. Feature selective projection with low-rank embedding and dual Laplacian regularization. IEEE Trans. Knowl. Data Eng. 32, 9 (2019), 1747–1760.
[44]
Chang Tang, Xinzhong Zhu, Jiajia Chen, Pichao Wang, Xinwang Liu, and Jie Tian. 2018. Robust graph regularized unsupervised feature selection. Expert Syst. Appl. 96 (2018), 64–76.
[45]
Joshua B. Tenenbaum, Vin de Silva, and John C. Langford. 2000. A global geometric framework for nonlinear dimensionality reduction. Science 290, 5500 (2000), 2319–2323.
[46]
Qi Wang, Xiang He, and Xuelong Li. 2018. Locality and structure regularized low rank representation for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 57, 2 (2018), 911–923.
[47]
Jie Wen, Na Han, Xiaozhao Fang, Lunke Fei, Ke Yan, and Shanhua Zhan. 2018. Low-rank preserving projection via graph regularized reconstruction. IEEE Trans. Cybernet. 49, 4 (2018), 1279–1291.
[48]
Svante Wold, Kim Esbensen, and Paul Geladi. 1987. Principal component analysis. Chemo. Intell. Lab. Syst. 2, 1-3 (1987), 37–52.
[49]
Wai Keung Wong, Zhihui Lai, Jiajun Wen, Xiaozhao Fang, and Yuwu Lu. 2017. Low-rank embedding for robust image feature extraction. IEEE Trans. Image Process. 26, 6 (2017), 2905–2917.
[50]
John Wright, Allen Y. Yang, Arvind Ganesh, S. Shankar Sastry, and Yi Ma. 2008. Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31, 2 (2008), 210–227.
[51]
Yong Xu, Xiaozhao Fang, Jian Wu, Xuelong Li, and David Zhang. 2015. Discriminative transfer subspace learning via low-rank and sparse representation. IEEE Trans. Image Process. 25, 2 (2015), 850–863.
[52]
Shuicheng Yan, Dong Xu, Benyu Zhang, Hong-Jiang Zhang, Qiang Yang, and Stephen Lin. 2006. Graph embedding and extensions: A general framework for dimensionality reduction. IEEE Trans. Pattern Anal. Mach. Intell. 29, 1 (2006), 40–51.
[53]
Junfeng Yang, Wotao Yin, Yin Zhang, and Yilun Wang. 2009. A fast algorithm for edge-preserving variational multichannel image restoration. SIAM J. Imag. Sci. 2, 2 (2009), 569–592.
[54]
Ming Yin, Junbin Gao, and Zhouchen Lin. 2015. Laplacian regularized low-rank representation and its applications. IEEE Trans. Pattern Anal. Mach. Intell. 38, 3 (2015), 504–517.
[55]
Shanhua Zhan, Jigang Wu, Na Han, Jie Wen, and Xiaozhao Fang. 2019. Unsupervised feature extraction by low-rank and sparsity preserving embedding. Neural Netw. 109 (2019), 56–66.
[56]
Yupei Zhang, Ming Xiang, and Bo Yang. 2017. Low-rank preserving embedding. Pattern Recogn. 70 (2017), 112–125.
[57]
Xiaowei Zhao, Feiping Nie, Sen Wang, Jun Guo, Pengfei Xu, and Xiaojiang Chen. 2017. Unsupervised 2D dimensionality reduction with adaptive structure learning. Neural Comput. 29, 5 (2017), 1352–1374.
[58]
Nan Zhou, Yangyang Xu, Hong Cheng, Jun Fang, and Witold Pedrycz. 2016. Global and local structure preserving sparse subspace learning: An iterative approach to unsupervised feature selection. Pattern Recogn. 53 (2016), 87–101.
[59]
Pengfei Zhu, Wencheng Zhu, Weizhi Wang, Wangmeng Zuo, and Qinghua Hu. 2017. Non-convex regularized self-representation for unsupervised feature selection. Image Vision Comput. 60 (2017), 22–29.

Cited By

View all
  • (2024)Research on Efficient Image Feature Extraction based on Low-Rank Representation for Intelligent Reading Systems2024 Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC)10.1109/IPEC61310.2024.00051(256-260)Online publication date: 12-Apr-2024
  • (2024)A game model for semi-supervised subspace clustering with dynamic affinity and label learningSignal Processing10.1016/j.sigpro.2024.109436220:COnline publication date: 1-Jul-2024
  • (2024)Self-adaptive subspace representation from a geometric intuitionPattern Recognition10.1016/j.patcog.2023.110228149:COnline publication date: 25-Jun-2024
  • Show More Cited By

Index Terms

  1. Robust Dimensionality Reduction via Low-rank Laplacian Graph Learning

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Intelligent Systems and Technology
    ACM Transactions on Intelligent Systems and Technology  Volume 14, Issue 3
    June 2023
    451 pages
    ISSN:2157-6904
    EISSN:2157-6912
    DOI:10.1145/3587032
    • Editor:
    • Huan Liu
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 01 April 2023
    Online AM: 01 February 2023
    Accepted: 26 January 2023
    Revised: 18 February 2022
    Received: 06 October 2021
    Published in TIST Volume 14, Issue 3

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Dimensionality reduction
    2. feature extraction
    3. Laplacian graph learning
    4. low rank representation

    Qualifiers

    • Research-article

    Funding Sources

    • Key Program of National Natural Science Foundations of China

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)232
    • Downloads (Last 6 weeks)13
    Reflects downloads up to 10 Oct 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Research on Efficient Image Feature Extraction based on Low-Rank Representation for Intelligent Reading Systems2024 Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC)10.1109/IPEC61310.2024.00051(256-260)Online publication date: 12-Apr-2024
    • (2024)A game model for semi-supervised subspace clustering with dynamic affinity and label learningSignal Processing10.1016/j.sigpro.2024.109436220:COnline publication date: 1-Jul-2024
    • (2024)Self-adaptive subspace representation from a geometric intuitionPattern Recognition10.1016/j.patcog.2023.110228149:COnline publication date: 25-Jun-2024
    • (2024)Global and local structure projection learning for image feature extractionThe Journal of Supercomputing10.1007/s11227-024-06220-880:14(21001-21022)Online publication date: 5-Jun-2024
    • (2023)Fuzzy K-Nearest Neighbor Graph Regularized Non-Negative Matrix Factorization2023 IEEE 6th International Conference on Electronic Information and Communication Technology (ICEICT)10.1109/ICEICT57916.2023.10245236(891-895)Online publication date: 21-Jul-2023
    • (2023)Robust latent discriminative adaptive graph preserving learning for image feature extractionKnowledge-Based Systems10.1016/j.knosys.2023.110487268:COnline publication date: 23-May-2023
    • (2023)Efficient low-rank multi-component fusion with component-specific factors in image-recipe retrievalMultimedia Tools and Applications10.1007/s11042-023-15819-783:2(3601-3619)Online publication date: 18-May-2023

    View Options

    Get Access

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media