research-article

Mutual Information-Driven Multi-View Clustering

Authors:

Chuanfu ZhangAuthors Info & Claims

CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

Pages 3268 - 3277

https://doi.org/10.1145/3583780.3614986

Published: 21 October 2023 Publication History

Abstract

In deep multi-view clustering, three intractable problems are posed ahead of researchers, namely, the complementarity exploration problem, the information preservation problem, and the cluster structure discovery problem. In this paper, we consider the deep multi-view clustering from the perspective of mutual information (MI), and attempt to address the three important concerns with a Mutual Information-Driven Multi-View Clustering (MIMC) method, which extracts the common and view-specific information hidden in multi-view data and constructs a clustering-oriented comprehensive representation. Specifically, three constraints based on MI are devised in response to three issues. Correspondingly, we minimize the MI between the common representation and view-specific representations to exploit the inter-view complementary information. Further, we maximize the MI between the refined data representations and original data representations to preserve the principal information. Moreover, to learn a clustering-friendly comprehensive representation, the MI between the comprehensive embedding space and cluster structure is maximized. Finally, we conduct extensive experiments on six benchmark datasets, and the experimental results indicate that the proposed MIMC outperforms other clustering methods.

References

[1]

Arthur Asuncion and David Newman. 2007. UCI machine learning repository.

[2]

Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. 2018. Mutual information neural estimation. In Proceedings of the International Conference on Machine Learning. 531--540.

[3]

Kamalika Chaudhuri, Sham M Kakade, Karen Livescu, and Karthik Sridharan. 2009. Multi-view clustering via canonical correlation analysis. In Proceedings of the International Conference on Machine Learning. 129--136.

Digital Library

[4]

Yongyong Chen, Xiaolin Xiao, and Yicong Zhou. 2020. Multi-view subspace clustering via simultaneously learning the representation tensor and affinity matrix. Pattern Recognition, Vol. 106 (2020), 107441.

[5]

Zhaoliang Chen, Lele Fu, Jie Yao, Wenzhong Guo, Claudia Plant, and Shiping Wang. 2023. Learnable graph convolutional network and feature fusion for multi-view learning. Information Fusion, Vol. 95 (2023), 109--119.

Digital Library

[6]

Jinfu Fan, Yang Yu, Linqing Huang, and Zhongjie Wang. 2023. GraphDPI: Partial label disambiguation by graph representation learning via mutual information maximization. Pattern Recognition, Vol. 134 (2023), 109133.

Digital Library

[7]

Marco Federici, Anjan Dutta, Patrick Forré, Nate Kushman, and Zeynep Akata. 2020. Learning Robust Representations via Multi-View Information Bottleneck. In Proceedings of the International Conference on Learning Representations.

[8]

Lele Fu, Zhaoliang Chen, Yongyong Chen, and Shiping Wang. 2022a. Unified Low-Rank Tensor Learning and Spectral Embedding for Multi-View Subspace Clustering. IEEE Transactions on Multimedia (2022).

Digital Library

[9]

Lele Fu, Zhaoliang Chen, Sujia Huang, Sheng Huang, and Shiping Wang. 2021. Multi-View Learning Via Low-Rank Tensor Optimization. In Proceedings of the IEEE International Conference on Multimedia and Expo. 1--6.

[10]

Lele Fu, Pengfei Lin, Athanasios V. Vasilakos, and Shiping Wang. 2020. An overview of recent multi-view clustering. Neurocomputing, Vol. 402 (2020), 148--161.

[11]

Lele Fu, Jinghua Yang, Chuan Chen, and Chuanfu Zhang. 2022b. Low-rank tensor approximation with local structure for multi-view intrinsic subspace clustering. Information Sciences, Vol. 606 (2022), 877--891.

Digital Library

[12]

Quanxue Gao, Huanhuan Lian, Qianqian Wang, and Gan Sun. 2020. Cross-Modal Subspace Clustering via Deep Canonical Correlation Analysis. In Proceedings of the AAAI Conference on Artificial Intelligence. 3938--3945.

[13]

Jipeng Guo, Yanfeng Sun, Junbin Gao, Yongli Hu, and Baocai Yin. 2023. Logarithmic Schatten-textdollarptextdollarp Norm Minimization for Tensorial Multi-View Subspace Clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 45, 3 (2023), 3396--3410.

[14]

Zhenhua Guo, Lei Zhang, and David Zhang. 2010. A completed modeling of local binary pattern operator for texture classification. IEEE Transactions on Image Processing, Vol. 19, 6 (2010), 1657--1663.

Digital Library

[15]

S. El Hajjar, Fadi Dornaika, and Fahed Abdallah. 2022. Multi-view spectral clustering via constrained nonnegative embedding. Information Fusion, Vol. 78 (2022), 209--217.

Digital Library

[16]

Zhanxuan Hu, Feiping Nie, Rong Wang, and Xuelong Li. 2020. Multi-view spectral clustering via integrating nonnegative embedding and spectral embedding. Information Fusion, Vol. 55 (2020), 251--259.

[17]

Chenglu Li, Hangjun Che, Man-Fai Leung, Cheng Liu, and Zheng Yan. 2023. Robust multi-view non-negative matrix factorization with adaptive graph and diversity constraints. Information Sciences, Vol. 634 (2023), 587--607.

Digital Library

[18]

Zhenglai Li, Chang Tang, Xinwang Liu, Xiao Zheng, Wei Zhang, and En Zhu. 2022. Consensus Graph Learning for Multi-View Clustering. IEEE Transactions on Multimedia, Vol. 24 (2022), 2461--2472.

Digital Library

[19]

Yijie Lin, Yuanbiao Gou, Xiaotian Liu, Jinfeng Bai, Jiancheng Lv, and Xi Peng. 2023. Dual Contrastive Prediction for Incomplete Multi-View Representation Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 45, 4 (2023), 4447--4461.

Digital Library

[20]

Suyuan Liu, Siwei Wang, Pei Zhang, Xinwang Liu, Kai Xu, Changwang Zhang, and Feng Gao. 2022. Efficient One-pass Multi-view Subspace Clustering with Consensus Anchors. In Proceedings of the AAAI Conference on Artificial Intelligence.

[21]

Shirui Luo, Changqing Zhang, Wei Zhang, and Xiaochun Cao. 2018. Consistent and Specific Multi-view Subspace Clustering. In Proceedings of the International Joint Conferences on Artificial Intelligence Organization. 3730--3737.

[22]

Yiqiao Mao, Xiaoqiang Yan, Qiang Guo, and Yangdong Ye. 2021. Deep Mutual Information Maximin for Cross-Modal Clustering. Proceedings of the AAAI Conference on Artificial Intelligence (2021), 8893--8901.

[23]

Feiping Nie, Jing Li, and Xuelong Li. 2016. Parameter-free auto-weighted multiple graph learning: A framework for multiview clustering and semi-supervised classification. In Proceedings of the International Joint Conferences on Artificial Intelligence Organization. 1881--1887.

[24]

Feiping Nie, Jing Li, and Xuelong Li. 2017. Self-weighted Multiview Clustering with Multiple Graphs. In Proceedings of the International Joint Conferences on Artificial Intelligence Organization. 2564--2570.

[25]

Carol L Novak, Steven A Shafer, et al. 1992. Anatomy of a color histogram. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 599--605.

[26]

Shachar Schnapp and Sivan Sabato. 2021. Active Feature Selection for the Mutual Information Criterion. In Proceedings of the AAAI Conference on Artificial Intelligence. 9497--9504.

[27]

Alexander Shapiro. 2003. Monte Carlo sampling methods. Handbooks in operations research and management science, Vol. 10 (2003), 353--425.

[28]

Jianqin Sun, Xianchao Xiu, Ziyan Luo, and Wanquan Liu. 2023. Learning High-Order Multi-View Representation by New Tensor Canonical Correlation Analysis. IEEE Transactions on Circuits and Systems for Video Technology (2023).

Digital Library

[29]

Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2019. Representation Learning with Contrastive Predictive Coding. arXiv preprint arXiv:1807.03748 (2019).

[30]

Amir Pouran Ben Veyseh, Franck Dernoncourt, My Tra Thai, Dejing Dou, and Thien Huu Nguyen. 2020. Multi-View Consistency for Relation Extraction via Mutual Information and Structure Prediction. In Proceedings of the AAAI Conference on Artificial Intelligence. 9106--9113.

[31]

Zhibin Wan, Changqing Zhang, Pengfei Zhu, and Qinghua Hu. 2021. Multi-View Information-Bottleneck Representation Learning. In Proceedings of the AAAI Conference on Artificial Intelligence. 10085--10092.

[32]

Feng Wang, Tao Kong, Rufeng Zhang, Huaping Liu, and Hang Li. 2023 a. Self-Supervised Learning by Estimating Twin Class Distribution. IEEE Transactions on Image Processing, Vol. 32 (2023), 2228--2236.

Digital Library

[33]

H. Wang, Y. Yang, and B. Liu. 2020. GMC: Graph-Based Multi-View Clustering. IEEE Transactions on Knowledge and Data Engineering, Vol. 32, 6 (2020), 1116--1129.

[34]

Qi Wang, Claire Boudreau, Qixing Luo, Pang-Ning Tan, and Jiayu Zhou. 2019. Deep Multi-view Information Bottleneck. In Proceedings of the International Conference on Data Mining. 37--45.

[35]

Qianqian Wang, Jiafeng Cheng, Quanxue Gao, Guoshuai Zhao, and Licheng Jiao. 2021. Deep Multi-View Subspace Clustering With Unified and Discriminative Learning. IEEE Transactions on Multimedia, Vol. 23 (2021), 3483--3493.

Digital Library

[36]

Shiping Wang, Zhaoliang Chen, Shide Du, and Zhouchen Lin. 2022. Learning Deep Sparse Regularizers With Applications to Multi-View Clustering and Semi-Supervised Classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 44, 9 (2022), 5042--5055.

[37]

Shiye Wang, Changsheng Li, Yanming Li, Ye Yuan, and Guoren Wang. 2023 b. Self-supervised information bottleneck for deep multi-view subspace clustering. IEEE Transactions on Image Processing, Vol. 32 (2023), 1555--1567.

Digital Library

[38]

Jie Wen, Zhihao Wu, Zheng Zhang, Lunke Fei, Bob Zhang, and Yong Xu. 2021. Structural Deep Incomplete Multi-view Clustering Network. In Proceedings of the ACM International Conference on Information and Knowledge Management. 3538--3542.

Digital Library

[39]

Wei Xia, Quanxue Gao, Qianqian Wang, Xinbo Gao, Chris Ding, and Dacheng Tao. 2023. Tensorized Bipartite Graph Learning for Multi-View Clustering. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 45, 4 (2023), 5187--5202.

Digital Library

[40]

Shunxin Xiao, Shide Du, Zhaoliang Chen, Yunhe Zhang, and Shiping Wang. 2023. Dual Fusion-Propagation Graph Neural Network for Multi-View Clustering. IEEE Transactions on Multimedia (2023), 1--13. doi=10.1109/TMM.2023.3248173.

Digital Library

[41]

Yuan Xie, Bingqian Lin, Yanyun Qu, Cuihua Li, Wensheng Zhang, Lizhuang Ma, Yonggang Wen, and Dacheng Tao. 2021. Joint Deep Multi-View Learning for Image Clustering. IEEE Transactions on Knowledge and Data Engineering, Vol. 33, 11 (2021), 3594--3606.

Digital Library

[42]

Yuan Xie, Dacheng Tao, Wensheng Zhang, Yan Liu, Lei Zhang, and Yanyun Qu. 2018. On Unifying Multi-view Self-Representations for Clustering by Tensor Multi-rank Minimization. International Journal of Computer Vision, Vol. 126, 11 (2018), 1157--1179.

Digital Library

[43]

Jie Xu, Huayi Tang, Yazhou Ren, Xiaofeng Zhu, and Lifang He. 2022. Multi-level Feature Learning for Contrastive Multi-view Clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 16051--16060.

[44]

Jing-Hua Yang, Chuan Chen, Hong-Ning Dai, Meng Ding, Le-Le Fu, and Zibin Zheng. 2022a. Hierarchical Representation for Multi-View Clustering: From Intra-Sample to Intra-View to Inter-View. In Proceedings of the ACM International Conference on Information and Knowledge Management. 2362--2371.

Digital Library

[45]

Jing-Hua Yang, Chuan Chen, Hong-Ning Dai, Meng Ding, Zhe-Bin Wu, and Zibin Zheng. 2022b. Robust corrupted data recovery and clustering via generalized transformed tensor low-rank representation. IEEE Transactions on Neural Networks and Learning Systems (2022).

[46]

Jing-Hua Yang, Chuan Chen, Hong-Ning Dai, Le-Le Fu, and Zibin Zheng. 2022c. A structure noise-aware tensor dictionary learning method for high-dimensional data clustering. Information Sciences, Vol. 612 (2022), 87--106.

Digital Library

[47]

Changqing Zhang, Yeqing Liu, and Huazhu Fu. 2019. AE2-Nets: Autoencoder in Autoencoder Networks. In Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition. 2577--2585.

[48]

Jing Zhao, Xijiong Xie, Xin Xu, and Shiliang Sun. 2017. Multi-view learning overview: Recent progress and new challenges. Information Fusion, Vol. 38 (2017), 43--54.

Digital Library

[49]

Pengfei Zhu, Binyuan Hui, Changqing Zhang, Dawei Du, Longyin Wen, and Qinghua Hu. 2019. Multi-view Deep Subspace Clustering Networks. ArXiv: 1908.01978 (2019).

[50]

Linlin Zong, Faqiang Miao, Xianchao Zhang, and Bo Xu. 2020. Multimodal Clustering via Deep Commonness and Uniqueness Mining. In Proceedings of the ACM International Conference on Information and Knowledge Management. 2357--2360.

Digital Library

Cited By

Fu LHuang SZhang LYang JZheng ZZhang CChen C(2024)Subspace-Contrastive Multi-View ClusteringACM Transactions on Knowledge Discovery from Data10.1145/3674839Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3674839
Li SLiu YChen QWebb GPan SSerra ESpezzano F(2024)Noise-Resilient Unsupervised Graph Representation Learning via Multi-Hop Feature Quality EstimationProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679758(1255-1265)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679758
Zheng YZhou GHuang HLuo XHuang ZZhao Q(2024)Unifying complete and incomplete multi-view clustering through an information-theoretic generative modelNeural Networks10.1016/j.neunet.2024.106901(106901)Online publication date: Nov-2024
https://doi.org/10.1016/j.neunet.2024.106901
Show More Cited By

Index Terms

Mutual Information-Driven Multi-View Clustering
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning
        Cluster analysis
2. Information systems
  1. Information systems applications
    1. Data mining
      1. Clustering

Recommendations

Multi-view Clustering with Graph Embedding for Connectome Analysis
CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management

Multi-view clustering has become a widely studied problem in the area of unsupervised learning. It aims to integrate multiple views by taking advantages of the consensus and complimentary information from multiple views. Most of the existing works in ...
Multi-view clustering via spectral partitioning and local refinement

A new multi-view clustering algorithm is proposed.The proposed MVNC algorithm uses spectral partitioning and local refinement.MVNC is compared to state-of-the-art algorithms using three real-world datasets.MVNC significantly outperforms the other ...
Contrastive and attentive graph learning for multi-view clustering
Abstract
Graph-based multi-view clustering aims to take advantage of multiple view graph information to provide clustering solutions. The consistency constraint of multiple views is the key of multi-view graph clustering. Most existing studies ...
Highlights
- A novel CAGL framework for multi-view clustering is proposed.
- CL is used to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

October 2023

5508 pages

ISBN:9798400701245

DOI:10.1145/3583780

General Chairs:
Ingo Frommholz
University of Wolverhampton, UK
,
Frank Hopfgartner
University of Koblenz, Germany
,
Mark Lee
University of Birmingham, UK
,
Michael Oakes
University of Birmingham, UK
,
Program Chairs:
Mounia Lalmas
Spotify, UK
,
Min Zhang
Tsinghua University, China
,
Rodrygo Santos
Federal University of Minas Gerais, Brazil

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Guangzhou Science and Technology Program
Sun Yat-Sen University Young Faculty Development Program
National Natural Science Foundation of China

Conference

CIKM '23

Sponsor:

CIKM '23: The 32nd ACM International Conference on Information and Knowledge Management

October 21 - 25, 2023

Birmingham, United Kingdom

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
301
Total Downloads

Downloads (Last 12 months)245
Downloads (Last 6 weeks)15

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Fu LHuang SZhang LYang JZheng ZZhang CChen C(2024)Subspace-Contrastive Multi-View ClusteringACM Transactions on Knowledge Discovery from Data10.1145/3674839Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3674839
Li SLiu YChen QWebb GPan SSerra ESpezzano F(2024)Noise-Resilient Unsupervised Graph Representation Learning via Multi-Hop Feature Quality EstimationProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679758(1255-1265)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679758
Zheng YZhou GHuang HLuo XHuang ZZhao Q(2024)Unifying complete and incomplete multi-view clustering through an information-theoretic generative modelNeural Networks10.1016/j.neunet.2024.106901(106901)Online publication date: Nov-2024
https://doi.org/10.1016/j.neunet.2024.106901
Zhao SLiao TFu LChen CBian JZheng Z(2024)Data-free knowledge distillation via generator-free data generation for Non-IID federated learningNeural Networks10.1016/j.neunet.2024.106627179(106627)Online publication date: Nov-2024
https://doi.org/10.1016/j.neunet.2024.106627

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents