Beyond the Known: Ambiguity-Aware Multi-view Learning
Pages 8518 - 8526
Abstract
The inherent variability and unpredictability in open multi-view learning scenarios infuse considerable ambiguity into the learning and decision-making processes of predictors. This demands that predictors not only recognize familiar patterns but also adaptively interpret unknown ones out of training scope. To address this challenge, we propose an Ambiguity-Aware Multi-view Learning Framework, which integrates four synergistic modules into an end-to-end framework to achieve generalizability and reliability beyond the known. By introducing the mixed samples to broaden the learning sample space, accompanied by corresponding soft labels to encapsulate their inherent uncertainty, the proposed method adapts to the distribution of potentially unknown samples in advance. Furthermore, an instance-level sparse inference is implemented to learn sparse approximated points in the multiple view embedding space, and individual view representations are gated by view-level confidence mappings. Finally, a multi-view consistent representation is obtained by dynamically assigning weights based on the degree of cluster-level dispersion. Extensive experiments demonstrate that our approach is effective and stable compared with other state-of-the-art methods in open-world recognition situations.
References
[1]
Zhaoliang Chen, Lele Fu, Jie Yao, Wenzhong Guo, Claudia Plant, and Shiping Wang. 2023. Learnable graph convolutional network and feature fusion for multi-view learning. Information Fusion, Vol. 95 (2023), 109--119.
[2]
Zhe Chen, Xiao-Jun Wu, Tianyang Xu, and Josef Kittler. 2023. Fast self-guided multi-view subspace clustering. IEEE Transactions on Image Processing, Vol. 32 (2023), 6514--6525.
[3]
Charles Corbière, Nicolas Thome, Avner Bar-Hen, Matthieu Cord, and Patrick Pérez. 2019. Addressing failure prediction by learning model confidence. Advances in Neural Information Processing Systems, Vol. 32 (2019).
[4]
Akshay Raj Dhamija, Manuel Günther, and Terrance Boult. 2018. Reducing network agnostophobia. Advances in Neural Information Processing Systems (2018), 9175--9186.
[5]
Shide Du, Zihan Fang, Shiyang Lan, Yanchao Tan, Manuel Günther, Shiping Wang, and Wenzhong Guo. 2023. Bridging trustworthiness and open-world learning: An exploratory neural approach for enhancing interpretability, generalization, and robustness. In Proceedings of the ACM International Conference on Multimedia. 8719--8729.
[6]
Chuanxing Geng, Sheng-jun Huang, and Songcan Chen. 2020. Recent advances in open set recognition: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 43, 10 (2020), 3614--3631.
[7]
Yu Geng, Zongbo Han, Changqing Zhang, and Qinghua Hu. 2021. Uncertainty-aware multi-view representation learning. In Proceedings of the AAAI Conference on Artificial Intelligence. 7545--7553.
[8]
Karol Gregor and Yann LeCun. 2010. Learning fast approximations of sparse coding. In Proceedings of the International Conference on Machine Learning. 399--406.
[9]
Zongbo Han, Fan Yang, Junzhou Huang, Changqing Zhang, and Jianhua Yao. 2022. Multimodal dynamics: Dynamical fusion for trustworthy multimodal classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 20707--20717.
[10]
Zongbo Han, Changqing Zhang, Huazhu Fu, and Joey Tianyi Zhou. 2021. Trusted multi-view classification. In International Conference on Learning Representations.
[11]
Mehadi Hassen and Philip K Chan. 2020. Learning a neural-network-based representation for open set recognition. In Proceedings of the 2020 SIAM International Conference on Data Mining. SIAM, 154--162.
[12]
Dong Huang, Chang-Dong Wang, and Jian-Huang Lai. 2023. Fast multi-view clustering via ensembles: Towards scalability, superiority, and simplicity. IEEE Transactions on Knowledge and Data Engineering, Vol. 35, 11 (2023), 11388--11402.
[13]
Xuhui Jia, Kai Han, Yukun Zhu, and Bradley Green. 2021. Joint representation learning and novel category discovery on single-and multi-modal data. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 610--619.
[14]
Paul Pu Liang, Zihao Deng, Martin Q Ma, James Y Zou, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2024. Factorized contrastive learning: Going beyond multi-view redundancy. Advances in Neural Information Processing Systems, Vol. 36 (2024), 32971--32998.
[15]
Yijie Lin, Yuanbiao Gou, Xiaotian Liu, Jinfeng Bai, Jiancheng Lv, and Xi Peng. 2022. Dual contrastive prediction for incomplete multi-view representation learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 45, 4 (2022), 4447--4461.
[16]
Walter J Scheirer, Anderson de Rezende Rocha, Archana Sapkota, and Terrance E Boult. 2012. Toward open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, 7 (2012), 1757--1772.
[17]
Chang Tang, Xiao Zheng, Xinwang Liu, Wei Zhang, Jing Zhang, Jian Xiong, and Lizhe Wang. 2021. Cross-view locality preserved diversity and consensus learning for multi-view unsupervised feature selection. IEEE Transactions on Knowledge and Data Engineering, Vol. 34, 10 (2021), 4705--4716.
[18]
Huayi Tang and Yong Liu. 2022. Deep safe multi-view clustering: Reducing the risk of clustering performance degradation caused by view increase. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 202--211.
[19]
S Vaze, K Han, A Vedaldi, and A Zisserman. 2022. Open-set recognition: A good closed-set classifier is all you need?. In International Conference on Learning Representations.
[20]
Ren Wang, Haoliang Sun, Yuling Ma, Xiaoming Xi, and Yilong Yin. 2023. MetaViewer: Towards a unified multi-view representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11590--11599.
[21]
Shiye Wang, Changsheng Li, Yanming Li, Ye Yuan, and Guoren Wang. 2023. Self-supervised information bottleneck for deep multi-view subspace clustering. IEEE Transactions on Image Processing, Vol. 32 (2023), 1555--1567.
[22]
Shicai Wei, Chunbo Luo, and Yang Luo. 2023. MMANet: Margin-aware distillation and modality-aware regularization for incomplete multimodal learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 20039--20049.
[23]
Zhihao Wu, Xincan Lin, Zhenghong Lin, Zhaoliang Chen, Yang Bai, and Shiping Wang. 2023. Interpretable graph convolutional network for multi-view semi-supervised learning. IEEE Transactions on Multimedia, Vol. 25 (2023), 8593--8606.
[24]
Kaiqiang Xiong, Rui Peng, Zhe Zhang, Tianxing Feng, Jianbo Jiao, Feng Gao, and Ronggang Wang. 2023. Cl-MVSNet: Unsupervised multi-view stereo with dual-level contrastive learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 3769--3780.
[25]
Cai Xu, Wei Zhao, Jinglong Zhao, Ziyu Guan, Yaming Yang, Long Chen, and Xiangyu Song. 2023. Progressive deep multi-view comprehensive representation learning. In Proceedings of the AAAI Conference on Artificial Intelligence. 10557--10565.
[26]
Jinglin Xu, Wenbin Li, Xinwang Liu, Dingwen Zhang, Ji Liu, and Junwei Han. 2020. Deep embedded complementary and interactive information for multi-view classification. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 6494--6501.
[27]
Hong-Ming Yang, Xu-Yao Zhang, Fei Yin, Qing Yang, and Cheng-Lin Liu. 2022. Convolutional prototype network for open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 44, 5 (2022), 2358--2370.
[28]
Mouxing Yang, Yunfan Li, Peng Hu, Jinfeng Bai, Jiancheng Lv, and Xi Peng. 2022. Robust multi-view clustering with incomplete information. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 45, 1 (2022), 1055--1069.
[29]
Hongyi Zhang, Moustapha Cissé, Yann N. Dauphin, and David Lopez-Paz. 2018. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations.
[30]
Tiejian Zhang, Xinwang Liu, En Zhu, Sihang Zhou, and Zhibin Dong. 2022. Efficient anchor learning-based multi-view clustering- A late fusion method. In Proceedings of the ACM International Conference on Multimedia. 3685--3693.
[31]
Qinghai Zheng, Jihua Zhu, Zhongyu Li, Zhiqiang Tian, and Chen Li. 2023. Comprehensive multi-view representation learning. Information Fusion, Vol. 89 (2023), 198--209.
[32]
Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. 2021. Learning placeholders for open-set recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4401--4410.
[33]
Peng Zhou and Liang Du. 2023. Learnable graph filter for multi-view clustering. In Proceedings of the ACM International Conference on Multimedia. 3089--3098.
Index Terms
- Beyond the Known: Ambiguity-Aware Multi-view Learning
Recommendations
Rank Consistency based Multi-View Learning: A Privacy-Preserving Approach
CIKM '15: Proceedings of the 24th ACM International on Conference on Information and Knowledge ManagementComplex media objects are often described by multi-view feature groups collected from diverse domains or information channels. Multi-view learning, which attempts to exploit the relationship among multiple views to improve learning performance, has ...
Multi-view transfer learning with a large margin approach
KDD '11: Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data miningTransfer learning has been proposed to address the problem of scarcity of labeled data in the target domain by leveraging the data from the source domain. In many real world applications, data is often represented from different perspectives, which ...
Visual Tracking via Multi-view Semi-supervised Learning
ACAI '18: Proceedings of the 2018 International Conference on Algorithms, Computing and Artificial IntelligenceIn this paper, we present a novel visual object tracking model via multi-view semi-supervised learning. Instead of concatenating multiple views into a single view directly to adapt to conventional machine learning algorithms, the combination of views is ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2024
11719 pages
ISBN:9798400706868
DOI:10.1145/3664647
- General Chairs:
- Jianfei Cai,
- Mohan Kankanhalli,
- Balakrishnan Prabhakaran,
- Susanne Boll,
- Program Chairs:
- Ramanathan Subramanian,
- Liang Zheng,
- Vivek K. Singh,
- Pablo Cesar,
- Lexing Xie,
- Dong Xu
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 October 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Key Research and Development Plan of China
- National Natural Science Foundation of China
Conference
MM '24
Sponsor:
MM '24: The 32nd ACM International Conference on Multimedia
October 28 - November 1, 2024
Melbourne VIC, Australia
Acceptance Rates
MM '24 Paper Acceptance Rate 1,150 of 4,385 submissions, 26%;
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 18Total Downloads
- Downloads (Last 12 months)18
- Downloads (Last 6 weeks)18
Reflects downloads up to 18 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in