research-article

Tensorized Incomplete Multi-view Kernel Subspace Clustering

Authors:

Guang-Yu Zhang,

Chang-Dong WangAuthors Info & Claims

Volume 179, Issue C

https://doi.org/10.1016/j.neunet.2024.106529

Published: 01 November 2024 Publication History

Abstract

Recently considerable advances have been achieved in the incomplete multi-view clustering (IMC) research. However, the current IMC works are often faced with three challenging issues. First, they mostly lack the ability to recover the nonlinear subspace structures in the multiple kernel spaces. Second, they usually neglect the high-order relationship in multiple representations. Third, they often have two or even more hyper-parameters and may not be practical for some real-world applications. To tackle these issues, we present a Tensorized Incomplete Multi-view Kernel Subspace Clustering (TIMKSC) approach. Specifically, by incorporating the kernel learning technique into an incomplete subspace clustering framework, our approach can robustly explore the latent subspace structure hidden in multiple views. Furthermore, we impute the incomplete kernel matrices and learn the low-rank tensor representations in a mutual enhancement manner. Notably, our approach can discover the underlying relationship among the observed and missing samples while capturing the high-order correlation to assist subspace clustering. To solve the proposed optimization model, we design a three-step algorithm to efficiently minimize the unified objective function, which only involves one hyper-parameter that requires tuning. Experiments on various benchmark datasets demonstrate the superiority of our approach. The source code and datasets are available at: https://www.researchgate.net/publication/381828300_TIMKSC_20240629.

References

[1]

Abavisani M., Patel V.M., Multimodal sparse and low-rank subspace clustering, Information Fusion 39 (2018) 168–177.

[2]

Cai X., Huang D., Zhang G.-Y., Wang C.-D., Seeking commonness and inconsistencies: A jointly smoothed approach to multi-view subspace clustering, Information Fusion 91 (2023) 364–375.

[3]

Chen M.S., Huang L., Wang C.D., Huang D., Lai J.H., Relaxed multi-view clustering in latent embedding space, Information Fusion 68 (2021) 8–21.

[4]

Chen Y., Wang S., Xiao X., Liu Y., Hua Z., Zhou Y., Self-paced enhanced low-rank tensor kernelized multi-view subspace clustering, IEEE Transactions on Multimedia 24 (2021) 4054–4066.

[5]

Deng S., Wen J., Liu C., Yan K., Xu G., Xu Y., Projective incomplete multi-view clustering, IEEE Transactions on Neural Networks and Learning Systems (2023).

[6]

Fang X., Hu Y., Zhou P., Wu D.O., Unbalanced incomplete multi-view clustering via the scheme of view evolution: Weak views are meat; strong views do eat, IEEE Transactions on Emerging Topics in Computational Intelligence (2021).

[7]

Fang S.G., Huang D., Cai X.S., Wang C.D., He C., Tang Y., Efficient multi-view clustering via unified and discrete bipartite graph learning, IEEE Transactions on Neural Networks and Learning Systems (2023).

[8]

Gönen M., Alpaydın E., Multiple kernel learning algorithms, Journal of Machine Learning Research 12 (2011) 2211–2268.

Digital Library

[9]

He Q., Sun X., Yan Z., Li B., Fu K., Multi-object tracking in satellite videos with graph-based multitask modeling, IEEE Transactions on Geoscience and Remote Sensing 60 (2022) 1–13.

[10]

Hu, M., & Chen, S. (2019a). Doubly aligned incomplete multi-view clustering. In Proceedings of the international joint conference on artificial intelligence (pp. 2262–2268).

[11]

Hu, M., & Chen, S. (2019b). One-pass incomplete multi-view clustering. In Proceedings of the AAAI conference on artificial intelligence, vol. 33, no. 01 (pp. 3838–3845).

[12]

Hu W., Tao D., Zhang W., Xie Y., Yang Y., The twist tensor nuclear norm for video completion, IEEE transactions on neural networks and learning systems 28 (12) (2016) 2961–2973.

[13]

Huang L., Chao H.Y., Wang C.D., Multi-view intact space clustering, Pattern Recognition 86 (2019) 344–353.

[14]

Huang S., Kang Z., Tsang I.W., Xu Z., Auto-weighted multi-view clustering via kernelized graph learning, Pattern Recognition 88 (2019) 174–184.

Digital Library

[15]

Jin, J., Wang, S., Dong, Z., Liu, X., & Zhu, E. (2023). Deep incomplete multi-view clustering with cross-view partial sample and prototype alignment. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 11600–11609).

[16]

Kang Z., Lin Z., Zhu X., Xu W., Structured graph learning for scalable subspace clustering: From single view to multiview, IEEE Transactions on Cybernetics 52 (9) (2021) 8976–8986.

[17]

Kilmer M.E., Martin C.D., Factorization strategies for third-order tensors, Linear Algebra and its Applications 435 (3) (2011) 641–658.

[18]

Li, S. Y., Jiang, Y., & Zhou, Z. H. (2014). Partial multi-view clustering. In Proceedings of the AAAI conference on artificial intelligence, vol. 28, no. 1.

[19]

Li L., Wan Z., He H., Incomplete multi-view clustering with joint partition and graph learning, IEEE Transactions on Knowledge and Data Engineering (2021).

[20]

Li M., Xia J., Xu H., Liao Q., Zhu X., Liu X., Localized incomplete multiple kernel k-means with matrix-induced regularization, IEEE Transactions on Cybernetics (2021).

[21]

Liang Y.W., Huang D., Wang C.D., Yu P.S., Multi-view graph learning by joint modeling of consistency and inconsistency, IEEE Transactions on Neural Networks and Learning Systems (2022).

[22]

Liang J., Liu X., Bai L., Cao F., Wang D., Incomplete multi-view clustering via local and global co-regularization, Science China. Information Sciences 65 (5) (2022).

[23]

Liang, D. H., & Wang, C. D. (2019). Consistency Meets Inconsistency: A Unified Graph Learning Framework for Multi-view Clustering. In Proceedings of the 19th IEEE international conference on data mining (pp. 1204–1209).

[24]

Lin, Y., Gou, Y., Liu, Z., Li, B., Lv, J., & Peng, X. (2021). Completer: Incomplete multi-view clustering via contrastive prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 11174–11183).

[25]

Liu X., Incomplete multiple kernel alignment maximization for clustering, IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).

Digital Library

[26]

Liu X., Simplemkkm: Simple multiple kernel k-means, IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (4) (2022) 5174–5186.

[27]

Liu X., Hyperparameter-free localized simple multiple kernel K-means with global optimum, IEEE Transactions on Pattern Analysis and Machine Intelligence (2023).

[28]

Liu Y., Li G., Lin L., Cross-modal causal relational reasoning for event-level visual question answering, IEEE Transactions on Pattern Analysis and Machine Intelligence (2023).

[29]

Liu X., Li M., Tang C., Xia J., Xiong J., Liu L., et al., Efficient and effective regularized incomplete multi-view clustering, IEEE Transactions on Pattern Analysis and Machine Intelligence 43 (8) (2020) 2634–2646.

[30]

Liu C., Li R., Wu S., Che H., Jiang D., Yu Z., et al., Self-guided partial graph propagation for incomplete multiview clustering, IEEE Transactions on Neural Networks and Learning Systems (2023).

[31]

Liu, J., Liu, X., Zhang, Y., Zhang, P., Tu, W., Wang, S., et al. (2021). Self-representation subspace clustering for incomplete multi-view data. In Proceedings of the 29th ACM international conference on multimedia (pp. 2726–2734).

[32]

Liu J., Teng S., Fei L., Zhang W., Fang X., Zhang Z., et al., A novel consensus learning approach to incomplete multi-view clustering, Pattern Recognition 115 (2021).

[33]

Liu C., Wen J., Wu Z., Luo X., Huang C., Xu Y., Information recovery-driven deep incomplete multiview clustering network, IEEE Transactions on Neural Networks and Learning Systems (2023).

[34]

Liu, X., Zhu, X., Li, M., Tang, C., Zhu, E., Yin, J., et al. (2019). Efficient and effective incomplete multi-view clustering. In Proceedings of the AAAI conference on artificial intelligence, vol. 33, no. 01 (pp. 4392–4399).

[35]

Liu X., Zhu X., Li M., Wang L., Zhu E., Liu T., et al., Multiple kernel k k-means with incomplete kernels, IEEE Transactions on Pattern Analysis and Machine Intelligence 42 (5) (2019) 1191–1204.

[36]

Long, Z., Wang, Q., Ren, Y., Liu, Y., & Zhu, C. (2024). S2MVTC: a Simple yet Efficient Scalable Multi-View Tensor Clustering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 26213–26222).

[37]

Long Z., Zhu C., Chen J., Li Z., Ren Y., Liu Y., Multi-view MERA subspace clustering, IEEE Transactions on Multimedia (2023).

[38]

Long Z., Zhu C., Comon P., Ren Y., Liu Y., Feature space recovery for efficient incomplete multi-view clustering, IEEE Transactions on Knowledge and Data Engineering (2023).

[39]

Lu Y., Liu Y., Long Z., Chen Z., Zhu C., O-minus decomposition for multi-view tensor subspace clustering, IEEE Transactions on Artificial Intelligence (2023).

[40]

Maaten L.v.d., Hinton G., Visualizing data using t-SNE, Journal of Machine Learning Research 9 (Nov) (2008) 2579–2605.

[41]

Shao W., He L., Lu C.t., Philip S.Y., Online multi-view clustering with incomplete views, in: 2016 IEEE international conference on big data, IEEE, 2016, pp. 1012–1017.

[42]

Shao W., He L., Yu P.S., Multiple incomplete views clustering via weighted nonnegative matrix factorization with L2, 1 regularization, in: Joint European conference on machine learning and knowledge discovery in databases, Springer, 2015, pp. 318–334.

[43]

Tang C., Li Z., Wang J., Liu X., Zhang W., Zhu E., Unified one-step multi-view spectral clustering, IEEE Transactions on Knowledge and Data Engineering 35 (6) (2022) 6449–6460.

[44]

Wang Y., Chang D., Fu Z., Wen J., Zhao Y., Graph contrastive partial multi-view clustering, IEEE Transactions on Multimedia (2022).

[45]

Wang Y., Chang D., Fu Z., Wen J., Zhao Y., Incomplete multiview clustering via cross-view relation transfer, IEEE Transactions on Circuits and Systems for Video Technology 33 (1) (2022) 367–378.

[46]

Wang S., Liu X., Zhu X., Zhang P., Zhang Y., Gao F., et al., Fast parameter-free multi-view subspace clustering with consensus anchor guidance, IEEE Transactions on Image Processing 31 (2021) 556–568.

[47]

Wang J., Tang C., Wan Z., Zhang W., Sun K., Zomaya A.Y., Efficient and effective one-step multiview clustering, IEEE Transactions on Neural Networks and Learning Systems (2023).

[48]

Wang J., Tang C., Zheng X., Liu X., Zhang W., Zhu E., et al., Fast approximated multiple kernel K-means, IEEE Transactions on Knowledge and Data Engineering (2023).

[49]

Wang B., Wang Z., Sun X., He Q., Wang H., Fu K., TDNet: A novel transductive learning framework with conditional metric embedding for few-shot remote sensing image scene classification, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing (2023).

[50]

Wang, H., Zong, L., Liu, B., Yang, Y., & Zhou, W. (2019). Spectral perturbation meets incomplete multi-view data. In Proceedings of the 28th international joint conference on artificial intelligence (pp. 3677–3683).

[51]

Wen J., Xu Y., Liu H., Incomplete multiview spectral clustering with adaptive graph learning, IEEE Transactions on Cybernetics 50 (4) (2018) 1418–1429.

[52]

Wen J., Yan K., Zhang Z., Xu Y., Wang J., Fei L., et al., Adaptive graph completion based incomplete multi-view clustering, IEEE Transactions on Multimedia 23 (2020) 2493–2504.

[53]

Wen J., Zhang Z., Fei L., Zhang B., Xu Y., Zhang Z., et al., A survey on incomplete multiview clustering, IEEE Transactions on Systems, Man, and Cybernetics: Systems (2022).

[54]

Wen, J., Zhang, Z., Xu, Y., Zhang, B., Fei, L., & Liu, H. (2019). Unified embedding alignment with missing views inferring for incomplete multi-view clustering. In Proceedings of the AAAI conference on artificial intelligence, vol. 33, no. 01 (pp. 5393–5400).

[55]

Wen J., Zhang Z., Zhang Z., Fei L., Wang M., Generalized incomplete multiview clustering with flexible locality structure diffusion, IEEE Transactions on Cybernetics 51 (1) (2020) 101–114.

[56]

Wu J., Lin Z., Zha H., Essential tensor learning for multi-view spectral clustering, IEEE Transactions on Image Processing 28 (12) (2019) 5910–5922.

[57]

Xia D., Yang Y., Yang S., Li T., Incomplete multi-view clustering via kernelized graph learning, Information Sciences 625 (2023) 1–19.

[58]

Xie Y., Tao D., Zhang W., Liu Y., Zhang L., Qu Y., On unifying multi-view self-representations for clustering by tensor multi-rank minimization, International Journal of Computer Vision 126 (11) (2018) 1157–1179.

[59]

Xie Y., Zhang W., Qu Y., Dai L., Tao D., Hyper-Laplacian regularized multilinear multiview self-representations for clustering and semisupervised learning, IEEE Transactions on Cybernetics 50 (2) (2018) 572–586.

[60]

Xu J., Li C., Peng L., Ren Y., Shi X., Shen H.T., et al., Adaptive feature projection with distribution alignment for deep incomplete multi-view clustering, IEEE Transactions on Image Processing 32 (2023) 1354–1366.

[61]

Xu J., Ren Y., Tang H., Yang Z., Pan L., Yang Y., et al., Self-supervised discriminative feature learning for deep multi-view clustering, IEEE Transactions on Knowledge and Data Engineering 35 (7) (2022) 7470–7482.

[62]

Yang X., Deng C., Dang Z., Tao D., Deep multiview collaborative clustering, IEEE Transactions on Neural Networks and Learning Systems 34 (1) (2021) 516–526.

[63]

Yang M., Li Y., Hu P., Bai J., Lv J., Peng X., Robust multi-view clustering with incomplete information, IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (1) (2022) 1055–1069.

[64]

Yu X., Liu H., Lin Y., Wu Y., Zhang C., Auto-weighted sample-level fusion with anchors for incomplete multi-view clustering, Pattern Recognition 130 (2022).

[65]

Zhang C., Fu H., Hu Q., Cao X., Xie Y., Tao D., et al., Generalized latent multi-view subspace clustering, IEEE Transactions on Pattern Analysis and Machine Intelligence 42 (1) (2018) 86–99.

Digital Library

[66]

Zhang, C., Hu, Q., Fu, H., Zhu, P., & Cao, X. (2017). Latent multi-view subspace clustering. In Proceedings of the 30th IEEE conference on computer vision and pattern recognition (pp. 4279–4287).

[67]

Zhang G.Y., Wang C.D., Huang D., Zheng W.S., Zhou Y.R., TW-Co-k-means: Two-level weighted collaborative k-means for multi-view clustering, Knowledge-Based Systems 150 (2018) 127–138.

[68]

Zhang G.Y., Zhou Y.R., He X.Y., Wang C.D., Huang D., One-step kernel multi-view subspace clustering, Knowledge-Based Systems (2019).

[69]

Zhang G.Y., Zhou Y.R., Wang C.D., Huang D., He X.Y., Joint representation learning for multi-view subspace clustering, Expert Systems with Applications 166 (2021).

[70]

Zhao, H., Liu, H., & Fu, Y. (2016). Incomplete multi-modal visual data grouping. In Proceedings of the international joint conference on artificial intelligence (pp. 2392–2398).

[71]

Zhu X., Liu X., Li M., Zhu E., Liu L., Cai Z., et al., Localized incomplete multiple kernel k-means, in: Proceedings of the twenty-seventh international joint conference on artificial intelligence, in: International joint conferences on artificial intelligence organization, 2018.

[72]

Zhuge W., Tao H., Luo T., Zeng L.-L., Hou C., Yi D., Joint representation learning and clustering: A framework for grouping partial multiview data, IEEE Transactions on Knowledge and Data Engineering 34 (8) (2020) 3826–3840.

Index Terms

Tensorized Incomplete Multi-view Kernel Subspace Clustering
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning
        Cluster analysis
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Clustering and classification
  2. Information systems applications
    1. Data mining
      1. Clustering

Index terms have been assigned to the content through auto-classification.

Recommendations

Kernelized multi-view subspace clustering via auto-weighted graph learning
Abstract
Multi-view subspace clustering has been an important and powerful tool for partitioning multi-view data, especially multi-view high-dimensional data. Despite great success, most of the existing multi-view subspace clustering methods still suffer ...
One-step Kernel Multi-view Subspace Clustering
Abstract
Multi-view subspace clustering is essential to many scientific problems. However, most existing methods suffer from three aspects of issues. First, these methods usually adopt a two-step framework, lacking the ability to achieve an ...
Tensorized Multi-view Subspace Representation Learning
Abstract
Self-representation based subspace learning has shown its effectiveness in many applications. In this paper, we promote the traditional subspace representation learning by simultaneously taking advantages of multiple views and prior constraint. ... $_{}$

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Neural Networks

Neural Networks Volume 179, Issue C

Nov 2024

1557 pages

Issue’s Table of Contents

Copyright © 2024.

Publisher

Elsevier Science Ltd.

United Kingdom

Publication History

Published: 01 November 2024

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents