Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Kernel correlation–dissimilarity for Multiple Kernel k-Means clustering

Published: 02 July 2024 Publication History

Abstract

The main objective of the Multiple Kernel k-Means (MKKM) algorithm is to extract non-linear information and achieve optimal clustering by optimizing base kernel matrices. Current methods enhance information diversity and reduce redundancy by exploiting interdependencies among multiple kernels based on correlations or dissimilarities. Nevertheless, relying solely on a single metric, such as correlation or dissimilarity, to define kernel relationships introduces bias and incomplete characterization. Consequently, this limitation hinders efficient information extraction, ultimately compromising clustering performance. To tackle this challenge, we introduce a novel method that systematically integrates both kernel correlation and dissimilarity. Our approach comprehensively captures kernel relationships, facilitating more efficient classification information extraction and improving clustering performance. By emphasizing the coherence between kernel correlation and dissimilarity, our method offers a more objective and transparent strategy for extracting non-linear information and significantly improving clustering precision, supported by theoretical rationale. We assess the performance of our algorithm on 13 challenging benchmark datasets, demonstrating its superiority over contemporary state-of-the-art MKKM techniques.

Highlights

We propose a MKKM method that assesses kernel correlation–dissimilarity consistency.
We utilize Manhattan distance and Frobenius inner product for kernel similarity.
Integrating these measures improves performance and generalization in clustering.
We employ the splitting method to iteratively update indicators and kernel weights.
Results on 13 challenging datasets confirm algorithm’s effectiveness and convergence.

References

[1]
Noronha M.D., Henriques R., Madeira S.C., Zárate L.E., Impact of metrics on biclustering solution and quality: A review, Pattern Recognit. 127 (2022),.
[2]
Yang X., Deng C., Zheng F., Yan J., Liu W., Deep spectral clustering using dual autoencoder network, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 4061–4070,.
[3]
Sugahara K., Okamoto K., Hierarchical co-clustering with augmented matrices from external domains, Pattern Recognit. 142 (2023),.
[4]
Hartigan J.A., Wong M.A., Algorithm AS 136: A K-means clustering algorithm, J. R. Stat. Soc. Ser. C (Appl. Stat.) 28 (1) (1979) 100–108,.
[5]
Ng A.Y., Jordan M.I., Weiss Y., On spectral clustering: Analysis and an algorithm, in: Proceedings of the Advances in Neural Information Processing Systems, 2001, pp. 849–856. URL: https://proceedings.neurips.cc/paper_files/paper/2001/file/801272ee79cfde7fa5960571fee36b9b-Paper.pdf.
[6]
Xu L., Neufeld J., Larson B., Schuurmans D., Maximum margin clustering, in: Proceedings of the Advances in Neural Information Processing Systems, 2004, pp. 1537–1544. URL: https://proceedings.neurips.cc/paper_files/paper/2004/file/6403675579f6114559c90de0014cd3d6-Paper.pdf.
[7]
Ester M., Kriegel H.-P., Sander J., Xu X., A density-based algorithm for discovering clusters in large spatial databases with noise, in: Proceedings of the International Conference on Knowledge Discovery and Data Mining, 1996, pp. 226–231. URL: https://cdn.aaai.org/KDD/1996/KDD96-037.pdf.
[8]
J. MacQueen, Some Methods for Classification and Analysis of MultiVariate Observations, in: Proceedings of the Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1, 1967, pp. 281–297.
[9]
Sinaga K.P., Yang M.-S., Unsupervised K-means clustering algorithm, IEEE Access 8 (2020) 80716–80727,.
[10]
Huang S., Kang Z., Xu Z., Liu Q., Robust deep k-means: An effective and simple method for data clustering, Pattern Recognit. 117 (2021),.
[11]
D. Arthur, S. Vassilvitskii, K-Means++: The Advantages of Careful Seeding, in: Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms, 2007, pp. 1027–1035.
[12]
Mussabayev R., Mladenovic N., Jarboui B., Mussabayev R., How to use K-means for big data clustering?, Pattern Recognit. 137 (2023),.
[13]
Yang X., Deng C., Wei K., Yan J., Liu W., Adversarial learning for robust deep clustering, in: Proceedings of the Advances in Neural Information Processing Systems, 2020, pp. 9098–9108. URL: https://proceedings.neurips.cc/paper_files/paper/2020/file/6740526b78c0b230e41ae61d8ca07cf5-Paper.pdf.
[14]
Diallo B., Hu J., Li T., Khan G.A., Liang X., Wang H., Auto-attention mechanism for multi-view deep embedding clustering, Pattern Recognit. 143 (2023),.
[15]
Yang X., Deng C., Dang Z., Tao D., Deep multiview collaborative clustering, IEEE Trans. Neural Netw. Learn. Syst. 34 (1) (2023) 516–526,.
[16]
Yan W., Li Y., Yang M., Towards deeper match for multi-view oriented multiple kernel learning, Pattern Recognit. 134 (2023),.
[17]
Yu S., Tranchevent L., Liu X., Glanzel W., Suykens J.A., De Moor B., Moreau Y., Optimized data fusion for kernel k-means clustering, IEEE Trans. Pattern Anal. Mach. Intell. 34 (5) (2011) 1031–1039,.
[18]
Girolami M., Mercer kernel-based clustering in feature space, IEEE Trans. Neural Netw. 13 (3) (2002) 780–784,.
[19]
Abin A.A., Beigy H., Active constrained fuzzy clustering: A multiple kernels learning approach, Pattern Recognit. 48 (3) (2015) 953–967,.
[20]
Ferreira M.R., de Carvalho F.d.A., Kernel-based hard clustering methods in the feature space with automatic variable weighting, Pattern Recognit. 47 (9) (2014) 3082–3095,.
[21]
Chao G., Sun S., Bi J., A survey on multiview clustering, IEEE Trans. Artif. Intell. 2 (2) (2021) 146–168,.
[22]
Amini M.R., Usunier N., Goutte C., Learning from multiple partially observed views -an application to multilingual text categorization, in: Proceedings of the Advances in Neural Information Processing Systems, 2009, pp. 28–36. URL: https://proceedings.neurips.cc/paper_files/paper/2009/file/f79921bbae40a577928b76d2fc3edc2a-Paper.pdf.
[23]
Bickel S., Scheffer T., Multi-view clustering, in: Proceedings of the IEEE International Conference on Data Mining, 2004, pp. 19–26,.
[24]
Xu C., Tao D., Xu C., A survey on multi-view learning, 2013,. arXiv preprint arXiv:1304.5634.
[25]
Bach F.R., Lanckriet G.R.G., Jordan M.I., Multiple kernel learning, conic duality, and the SMO algorithm, in: Proceedings of the International Conference on Machine Learning, 2004, p. 6,.
[26]
Zien A., Ong C.S., Multiclass multiple kernel learning, in: Proceedings of the International Conference on Machine Learning, Vol. 227, 2007, pp. 1191–1198,.
[27]
Guo D., Zhang J., Liu X., Cui Y., Zhao C., Multiple kernel learning based multi-view spectral clustering, in: Proceedings of the International Conference on Pattern Recognition, 2014, pp. 3774–3779,.
[28]
Huang H.-C., Chuang Y.-Y., Chen C.-S., Multiple kernel fuzzy clustering, IEEE Trans. Fuzzy Syst. 20 (1) (2012) 120–134,.
[29]
Lu Y., Wang L., Lu J., Yang J., Shen C., Multiple kernel clustering based on centered kernel alignment, Pattern Recognit. 47 (11) (2014) 3656–3664,.
[30]
Gönen M., Margolin A.A., Localized data fusion for kernel K-means clustering with application to cancer biology, in: Proceedings of the Advances in Neural Information Processing Systems, 2014, pp. 1305–1313. URL: https://proceedings.neurips.cc/paper_files/paper/2014/file/6c29793a140a811d0c45ce03c1c93a28-Paper.pdf.
[31]
Du L., Zhou P., Shi L., Wang H., Fan M., Wang W., Shen Y.-D., Robust multiple kernel K-means using l 21-norm, in: Proceedings of the International Joint Conference on Artificial Intelligence, 2015, pp. 3476–3482. URL: https://www.ijcai.org/Proceedings/15/Papers/489.pdf.
[32]
Liu X., Dou Y., Yin J., Wang L., Zhu E., Multiple kernel K-means clustering with matrix-induced regularization, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30, 2016, pp. 1888–1894,.
[33]
Yao Y., Li Y., Jiang B., Chen H., Multiple kernel k-means clustering by selecting representative kernels, IEEE Trans. Neural Netw. Learn. Syst. 32 (11) (2021) 4983–4996,.
[34]
Liu X., Simplemkkm: Simple multiple kernel K-means, IEEE Trans. Pattern Anal. Mach. Intell. 45 (4) (2023) 5174–5186,.
[35]
Bauckhage C., K-means clustering is matrix factorization, 2015,. arXiv preprint arXiv:1512.07548.
[36]
Liu X.-G., On Rayleigh quotient matrices:Theory and applications, J. Comput. Math. 17 (6) (1999) 629–638. URL: http://www.jstor.org/stable/43692821.
[37]
Bang S., Yu Y., Wu W., Robust multiple kernel k-means clustering using min-max optimizationn, 2019,. arXiv preprint arXiv:1803.02458.
[38]
Demšar J., Statistical comparisons of classifiers over multiple data sets, J. Mach. Learn. Res. 7 (1) (2006) 1–30. URL: http://jmlr.org/papers/v7/demsar06a.html.
[39]
Wang R., Lu J., Lu Y., Nie F., Li X., Discrete and parameter-free multiple kernel k-means, IEEE Trans. Image Process. 31 (2022) 2796–2808,.

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Pattern Recognition
Pattern Recognition  Volume 150, Issue C
Jun 2024
726 pages

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 02 July 2024

Author Tags

  1. k-means
  2. Multiple kernel learning
  3. Consistency
  4. Frobenius inner product
  5. Manhattan distance

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 0
    Total Downloads
  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 03 Oct 2024

Other Metrics

Citations

View Options

View options

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media