Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Multiple kernel clustering with local kernel reconstruction and global heat diffusion

Published: 16 May 2024 Publication History

Abstract

Multiple Kernel Clustering (MKC) is an effective approach for revealing nonlinear cluster structures in candidate kernels. However, existing MKC methods still face two key challenges. Firstly, the pairwise affinity in these methods is primarily determined by kernel similarity, disregarding the correlations among highly similar neighbors and resulting in redundant weight assignments and reduced clustering discriminability. Secondly, the direct utilization of affinity matrices overlooks high-order connections and introduces noise due to independent row-wise solving. To address these issues, we propose a novel local MKC method called LKRGDF. We begin by exploring affinity using the Local Kernel Reconstruction (LKR) model, reducing redundancy and enhancing clustering discriminability. Furthermore, we exploit the affinities with the Global heat kernel Diffusion (GD) procedure to capture long-range connections smoothly. The GD process acts as a low pass filter, focusing on small eigenvalues corresponding to top clusters. Finally, we integrate these smooth affinities within an auto-weighted Multiple Graph Fusion (MGF) framework to obtain a consensus graph. By assembling LKR, GD, and MGF in a sequential pipeline, our approach achieves the exploration and exploitation of local structures, gradually improving clustering performance while ensuring computational efficiency without the need for iterative steps. Extensive experiments on ten datasets demonstrate the superiority of our algorithm in terms of effectiveness and efficiency compared to state-of-the-art methods. The code for our method is publicly available at https://github.com/YanChenSCU/LKRGDF-2023.git.

Graphical abstract

Display Omitted

Highlights

A new reconstruction method is developed to explore the local kernel structure.
A heat diffusion method is developed to exploit the global kernel structure.
A new sequential MKC method is proposed to explore and exploit kernel structure.
The experimental results show both the effectiveness and efficiency of our method.

References

[1]
Zhou S., Liu X., Li M., Zhu E., Liu L., Zhang C., Yin J., Multiple kernel clustering with neighbor-kernel subspace segmentation, IEEE Trans. Neural Netw. Learn. Syst. 31 (4) (2020) 1351–1362.
[2]
Yang Y., Xu D., Nie F., Yan S., Zhuang Y., Image clustering using local discriminant models and global integration, IEEE Trans. Image Process. 19 (10) (2010) 2761–2773.
[3]
Roweis S.T., Saul L.K., Nonlinear dimensionality reduction by locally linear embedding, Science 290 (5500) (2000) 2323–2326.
[4]
Zhang T., Liu X., Gong L., Wang S., Niu X., Shen L., Late fusion multiple kernel clustering with local kernel alignment maximization, IEEE Trans. Multimed. 25 (2023) 993–1007.
[5]
Ren Z., Yang S.X., Sun Q., Wang T., Consensus affinity graph learning for multiple kernel clustering, IEEE Trans. Cybern. 51 (6) (2021) 3273–3284.
[6]
Li L., Wang S., Liu X., Zhu E., Shen L., Li K., Li K., Local sample-weighted multiple kernel clustering with consensus discriminative graph, IEEE Trans. Neural Netw. Learn. Syst. (2024) 1–14. In Press.
[7]
Ren Z., Li H., Yang C., Sun Q., Multiple kernel subspace clustering with local structural graph and low-rank consensus kernel learning, Knowl.-Based Syst. (ISSN ) 188 (2020).
[8]
Ren Z., Sun Q., Simultaneous global and local graph structure preserving for multiple kernel clustering, IEEE Trans. Neural Netw. Learn. Syst. 32 (5) (2021) 1839–1851.
[9]
M. Li, X. Liu, L. Wang, Y. Dou, J. Yin, E. Zhu, Multiple kernel clustering with local kernel alignment maximization, in: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, 2016, pp. 1704–1710.
[10]
X. Zhu, X. Liu, M. Li, E. Zhu, L. Liu, Z. Cai, J. Yin, W. Gao, Localized Incomplete Multiple Kernel k-means, in: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, 2018, pp. 3271–3277.
[11]
Liu J., Liu X., Xiong J., Liao Q., Zhou S., Wang S., Yang Y., Optimal neighborhood multiple kernel clustering with adaptive local kernels, IEEE Trans. Knowl. Data Eng. 34 (6) (2022) 2872–2885.
[12]
X. Liu, S. Zhou, L. Liu, C. Tang, S. Wang, J. Liu, Y. Zhang, Localized Simple Multiple Kernel K-Means, in: Proceedings of 2021 IEEE/CVF International Conference on Computer Vision, 2021, pp. 9293–9301.
[13]
L. Du, P. Zhou, L. Shi, H. Wang, M. Fan, W. Wang, Y.-D. Shen, Robust multiple kernel k-means using l21-norm, in: Proceedings of Twenty-Fourth International Joint Conference on Artificial Intelligence, 2015, pp. 3476–3482.
[14]
X. Liu, S. Zhou, Y. Wang, M. Li, Y. Dou, E. Zhu, J. Yin, Optimal Neighborhood Kernel Clustering with Multiple Kernels, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2017, pp. 2266–2272.
[15]
Huang H.-C., Chuang Y.-Y., Chen C.-S., Multiple kernel fuzzy clustering, IEEE Trans. Fuzzy Syst. 20 (1) (2012) 120–134.
[16]
Gönen M., Margolin A.A., Localized data fusion for kernel k-means clustering with application to cancer biology, Adv. Neural Inf. Process. Syst. 27 (2014) 1305–1313.
[17]
X. Liu, Y. Dou, J. Yin, L. Wang, E. Zhu, Multiple kernel k -means clustering with matrix-induced regularization, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2016, pp. 1888–1894.
[18]
Z. Kang, C. Peng, Q. Cheng, Twin Learning for Similarity and Clustering: A Unified Kernel Approach, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2017, pp. 2080–2086.
[19]
Z. Kang, C. Peng, Q. Cheng, Z. Xu, Unified Spectral Clustering With Optimal Graph, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2018, pp. 3366–3373.
[20]
Kang Z., Peng C., Cheng Q., Liu X., Peng X., Xu Z., Tian L., Structured graph learning for clustering and semi-supervised classification, Pattern Recognit. 110 (2021).
[21]
Zhou S., Zhu E., Liu X., Zheng T., Liu Q., Xia J., Yin J., Subspace segmentation-based robust multiple kernel clustering, Inf. Fusion 53 (2020) 145–154.
[22]
Zhao K., Wen L., Chen W., Xu Z., Low-rank kernel learning for graph-based clustering, Knowl.-Based Syst. 163 (JAN.1) (2019) 510–517.
[23]
Yang C., Ren Z., Sun Q., Wu M., Yin M., Sun Y., Joint correntropy metric weighting and block diagonal regularizer for robust multiple kernel subspace clustering, Inform. Sci. 500 (2019) 48–66.
[24]
Z. Kang, X. Lu, J. Yi, Z. Xu, Self-weighted Multiple Kernel Learning for Graph-based Clustering and Semi-supervised Classification, in: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, 2018, pp. 2312–2318.
[25]
Ren Z., Lei H., Sun Q., Yang C., Simultaneous learning coefficient matrix and affinity graph for multiple kernel clustering, Inform. Sci. 547 (2021) 289–306.
[26]
Lu Y., Wang L., Lu J., Yang J., Shen C., Multiple kernel clustering based on centered kernel alignment, Pattern Recognit. 47 (11) (2014) 3656–3664.
[27]
Liu X., SimpleMKKM: Simple multiple kernel K-means, IEEE Trans. Pattern Anal. Mach. Intell. 45 (4) (2023) 5174–5186.
[28]
Li X., Ren Z., Lei H., Huang Y., Sun Q., Multiple kernel clustering with pure graph learning scheme, Neurocomputing 424 (2021) 215–225.
[29]
Ren Z., Mukherjee M., Lloret J., Venu P., Multiple kernel driven clustering with locally consistent and selfish graph in industrial IoT, IEEE Trans. Ind. Inform. 17 (4) (2021) 2956–2963.
[30]
Z. Ren, Q. Sun, D. Wei, Multiple kernel clustering with kernel k-means coupled graph tensor learning, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, no. 11, 2021, pp. 9411–9418.
[31]
Wang S., Liu X., Zhu E., Tang C., Liu J., Hu J., Xia J., Yin J., Multi-view clustering via late fusion alignment maximization, in: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, International Joint Conferences on Artificial Intelligence Organization, 2019, pp. 3778–3784.
[32]
Sun M., Wang S., Zhang P., Liu X., Guo X., Zhou S., Zhu E., Projective multiple kernel subspace clustering, IEEE Trans. Multimed. 24 (2022) 2567–2579.
[33]
Wang S., Liu X., Liu L., Zhou S., Zhu E., Late fusion multiple kernel clustering with proxy graph refinement, IEEE Trans. Neural Netw. Learn. Syst. 34 (8) (2023) 4359–4370.
[34]
Saul L.K., Roweis S.T., Think globally, fit locally: Unsupervised learning of low dimensional manifolds, J. Mach. Learn. Res. 4 (Jun) (2003) 119–155.
[35]
Chung F., The heat kernel as the pagerank of a graph, Proc. Natl. Acad. Sci. 104 (50) (2007) 19735–19740.
[36]
Wu Z., Pan S., Long G., Jiang J., Zhang C., Beyond low-pass filtering: Graph convolutional networks with automatic filtering, IEEE Trans. Knowl. Data Eng. 35 (7) (2023) 6687–6697.
[37]
Zhang F., Hancock E.R., Graph spectral image smoothing using the heat kernel, Pattern Recognit. 41 (11) (2008) 3328–3342.
[38]
Hu B., Wang X., Zhou P., Du L., Multi-view outlier detection via graphs denoising, Information Fusion (ISSN ) 101 (2024) 102012.
[39]
Zhou P., Du L., Li X., Adaptive consensus clustering for multiple k-means via base results refining, IEEE Trans. Knowl. Data Engrg. 35 (10) (2023) 10251–10264.
[40]
Wai H.-T., Segarra S., Ozdaglar A.E., Scaglione A., Jadbabaie A., Blind community detection from low-rank excitations of a graph filter, IEEE Trans. Signal Process. 68 (2020) 436–451.
[41]
F. Nie, X. Wang, H. Huang, Clustering and projected clustering with adaptive neighbors, in: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2014, pp. 977–986.
[42]
Zhou P., Du L., Liu X., Shen Y.-D., Fan M., Li X., Self-paced clustering ensemble, IEEE Trans. Neural Netw. Learn. Syst. 32 (4) (2021) 1497–1511.
[43]
Wang W., Carreira-Perpinán M.A., Projection onto the probability simplex: An efficient algorithm with a simple proof, and an application, 2013, arXiv preprint arXiv:1309.1541.
[44]
Zhang X., Xue X., Sun H., Liu Z., Guo L., Guo X., Robust multiple kernel subspace clustering with block diagonal representation and low-rank consensus kernel, Knowl.-Based Syst. 227 (2021).
[45]
Deng Z., Choi K.-S., Jiang Y., Wang J., Wang S., A survey on soft subspace clustering, Inform. Sci. 348 (2016) 84–106.
[46]
Benedetti J.K., On the nonparametric estimation of regression functions, J. R. Stat. Soc. Ser. B Stat. Methodol. 39 (2) (1977) 248–253.
[47]
Karypis G., CLUTO-A Clustering Toolkit, University of Minnesota, 2002.
[48]
Macosko E.Z., Basu A., Satija R., Nemesh J., Shekhar K., Goldman M., Tirosh I., Bialas A.R., Kamitaki N., Martersteck E.M., et al., Highly parallel genome-wide expression profiling of individual cells using nanoliter droplets, Cell 161 (5) (2015) 1202–1214.
[49]
van der Maaten L., Hinton G., Visualizing data using t-SNE, J. Mach. Learn. Res. 9 (86) (2008) 2579–2605.

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Information Fusion
Information Fusion  Volume 105, Issue C
May 2024
619 pages

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 16 May 2024

Author Tags

  1. Multiple kernel clustering
  2. Local kernel reconstruction
  3. Global heat kernel diffusion
  4. Multiple graph fusion

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 0
    Total Downloads
  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 03 Oct 2024

Other Metrics

Citations

View Options

View options

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media