Anchor Graph-Based Feature Selection for One-Step Multi-View Clustering
Abstract
Recently, multi-view clustering methods have been widely used in handling multi-media data and have achieved impressive performances. Among the many multi-view clustering methods, anchor graph-based multi-view clustering has been proven to be highly efficient for large-scale data processing. However, most existing anchor graph-based clustering methods necessitate post-processing to obtain clustering labels and are unable to effectively utilize the information within anchor graphs. To address this issue, we draw inspiration from regression and feature selection to propose <underline><bold>A</bold></underline>nchor <underline><bold>G</bold></underline>raph-Based <underline><bold>F</bold></underline>eature <underline><bold>S</bold></underline>election for <underline><bold>O</bold></underline>ne-Step <underline><bold>M</bold></underline>ulti-<underline><bold>V</bold></underline>iew <underline><bold>C</bold></underline>lustering (AGFS-OMVC). Our method combines embedding learning and sparse constraint to perform feature selection, allowing us to remove noisy anchor points and redundant connections in the anchor graph. This results in a clean anchor graph that can be projected into the label space, enabling us to obtain clustering labels in a single step without post-processing. Lastly, we employ the tensor Schatten <inline-formula><tex-math notation="LaTeX">$p$</tex-math></inline-formula>-norm as a tensor rank approximation function to capture the complementary information between different views, ensuring similarity between cluster assignment matrices. Experimental results on five real-world datasets demonstrate that our proposed method outperforms state-of-the-art approaches.
References
[1]
H. Liu and H. Motoda, Computational Methods of Feature Selection. London, U.K.: Chapman & Hall, 2007.
[2]
Y. Wang, “Survey on deep multi-modal data analytics: Collaboration, rivalry, and fusion,” ACM Trans. Multimedia Comput., Commun., Appl., vol. 17, no. 1s, pp. 1–25, 2021.
[3]
E. Elhamifar and R. Vidal, “Sparse subspace clustering: Algorithm, theory, and applications,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 11, pp. 2765–2781, Nov. 2013.
[4]
C. Tang et al., “Learning a joint affinity graph for multiview subspace clustering,” IEEE Trans. Multimedia, vol. 21, no. 7, pp. 1724–1736, Jul. 2019.
[5]
Q. Wang, J. Cheng, Q. Gao, G. Zhao, and L. Jiao, “Deep multi-view subspace clustering with unified and discriminative learning,” IEEE Trans. Multimedia, vol. 23, pp. 3483–3493, 2021.
[6]
Z. Chen, X.-J. Wu, T. Xu, and J. Kittler, “Fast self-guided multi-view subspace clustering,” IEEE Trans. Image Process., vol. 32, pp. 6514–6525, 2023.
[7]
H. Wang et al., “Kernelized multiview subspace analysis by self-weighted learning,” IEEE Trans. Multimedia, vol. 23, pp. 3828–3840, 2021.
[8]
J. Gao, J. Han, J. Liu, and C. Wang, “Multi-view clustering via joint nonnegative matrix factorization,” in Proc. 13th SIAM Int. Conf. Data Mining, 2013, pp. 252–260.
[9]
X. Zhang, L. Zhao, L. Zong, X. Liu, and H. Yu, “Multi-view clustering via multi-manifold regularized nonnegative matrix factorization,” in Proc. IEEE Int. Conf. Data Mining, 2014, pp. 1103–1108.
[10]
H. Zhao, Z. Ding, and Y. Fu, “Multi-view clustering via deep matrix factorization,” in Proc. Thirty-First AAAI Conf. Artif. Intell., 2017, pp. 2921–2927.
[11]
X. Li, G. Cui, and Y. Dong, “Graph regularized non-negative low-rank matrix factorization for image clustering,” IEEE Trans. Cybern., vol. 47, no. 11, pp. 3840–3853, Nov. 2017.
[12]
Z. Hu, F. Nie, R. Wang, and X. Li, “Multi-view spectral clustering via integrating nonnegative embedding and spectral embedding,” Inf. Fusion, vol. 55, pp. 251–259, 2020.
[13]
A. Kumar and P. Rai, “Co-regularized multi-view spectral clustering,” in Proc. Annu. Conf. Neural Inf. Process. Syst., 2011, pp. 1413–1421.
[14]
Y. Wang, L. Wu, X. Lin, and J. Gao, “Multiview spectral clustering via structured low-rank matrix factorization,” IEEE Trans. Neural Netw. Learn. Syst., vol. 29, no. 10, pp. 4833–4843, Oct. 2018.
[15]
W. Xia, Q. Wang, Q. Gao, X. Zhang, and X. Gao, “Self-supervised graph convolutional network for multi-view clustering,” IEEE Trans. Multimedia, vol. 24, pp. 3182–3192, 2022.
[16]
X. Li, H. Zhang, R. Wang, and F. Nie, “Multi-view clustering: A scalable and parameter-free bipartite graph fusion method,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 1, pp. 330–344, Jan. 2022.
[17]
Y. Li, F. Nie, H. Huang, and J. Huang, “Large-scale multi-view spectral clustering via bipartite graph,” in Proc. AAAI Conf. Artif. Intell., 2015, pp. 2750–2756.
[18]
H. Yang, Q. Gao, W. Xia, M. Yang, and X. Gao, “Multiview spectral clustering with bipartite graph,” IEEE Trans. Image Process., vol. 31, pp. 3591–3605, 2022.
[19]
W. Xia et al., “Tensorized bipartite graph learning for multi-view clustering,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 4, pp. 5187–5202, Apr. 2023.
[20]
B. Yang et al., “Fast multi-view clustering via nonnegative and orthogonal factorization,” IEEE Trans. Image Process., vol. 30, pp. 2575–2586, 2021.
[21]
B. Yang, X. Zhang, Z. Li, F. Nie, and F. Wang, “Efficient multi-view k-means clustering with multiple anchor graphs,” IEEE Trans. Knowl. Data Eng., vol. 35, no. 7, pp. 6887–6900, Jul. 2023.
[22]
W. Xia et al., “Multiview subspace clustering by an enhanced tensor nuclear norm,” IEEE Trans. Cybern., vol. 52, no. 9, pp. 8962–8975, Sep. 2022.
[23]
A. Kumar and H. Daume, “A co-training approach for multi-view spectral clustering,” in Proc. 28th Int. Conf. Mach. Learn., 2011, pp. 393–400.
[24]
X. Cai, F. Nie, W. Cai, and H. Huang, “Heterogeneous image features integration via multi-modal semi-supervised learning model,” in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 1737–1744.
[25]
F. Nie, J. Li, and X. Li, “Parameter-free auto-weighted multiple graph learning: A framework for multiview clustering and semi-supervised classification,” in Proc. Twenty-Fifth Int. Joint Conf. Artif. Intell., 2016, pp. 1881–1887.
[26]
Y. Wang et al., “Iterative views agreement: An iterative low-rank based structured optimization method to multi-view spectral clustering,” in Proc. Twenty-Fifth Int. Joint Conf. Artif. Intell., 2016, pp. 2153–2159.
[27]
S. Wang et al., “Fast parameter-free multi-view subspace clustering with consensus anchor guidance,” IEEE Trans. Image Process., vol. 31, pp. 556–568, 2022.
[28]
X. Shu et al., “Self-weighted anchor graph learning for multi-view clustering,” IEEE Trans. Multimedia, vol. 25, pp. 5485–5499, 2023.
[29]
R. Xia, Y. Pan, L. Du, and J. Yin, “Robust multi-view spectral clustering via low-rank and sparse decomposition,” in Proc. 28th AAAI Conf. Artif. Intell., 2014, pp. 2149–2155.
[30]
Q. Wang, R. Liu, M. Chen, and X. Li, “Robust rank-constrained sparse learning: A graph-based framework for single view and multiview clustering,” IEEE Trans. Cybern., vol. 52, no. 10, pp. 10228–10239, Oct. 2022.
[31]
D. D. Lee and H. S. Seung, “Algorithms for non-negative matrix factorization,” in Proc. Annu. Conf. Neural Inf. Process. Syst., 2000, pp. 556–562.
[32]
D. Cai, X. He, J. Han, and T. S. Huang, “Graph regularized nonnegative matrix factorization for data representation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 8, pp. 1548–1560, Aug. 2011.
[33]
Y. Wang, S. Wu, B. Mao, X. Zhang, and Z. Luo, “Correntropy induced metric based graph regularized non-negative matrix factorization,” Neurocomputing, vol. 204, pp. 172–182, 2016.
[34]
J. Huang, F. Nie, H. Huang, and C. H. Q. Ding, “Robust manifold nonnegative matrix factorization,” ACM Trans. Knowl. Discov. Data, vol. 8, no. 3, pp. 1–21, 2013.
[35]
F. Nie, H. Huang, X. Cai, and C. H. Q. Ding, “Efficient and robust feature selection via joint l2, 1-norms minimization,” in Proc. Annu. Conf. Neural Inf. Process. Syst., 2010, pp. 1813–1821.
[36]
Q. Wang, Q. Gao, X. Gao, and F. Nie, “$\ell _{2,p}$-Norm based PCA for image recognition,” IEEE Trans. Image Process., vol. 27, no. 3, pp. 1336–1346, Mar. 2018.
[37]
Z. Chen, X.-J. Wu, T. Xu, and J. Kittler, “Discriminative dictionary pair learning with scale-constrained structured representation for image classification,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 12, pp. 10225–10239, Dec. 2023.
[38]
Z. Chen, X.-J. Wu, and J. Kittler, “Relaxed block-diagonal dictionary pair learning with locality constraint for image recognition,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 8, pp. 3645–3659, Aug. 2022.
[39]
Q. Gao et al., “Enhanced tensor RPCA and its application,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 6, pp. 2133–2140, Jun. 2021.
[40]
M. Signoretto, L. D. Lathauwer, and J. A. Suykens, “Nuclear norms for tensors and their use for convex multilinear estimation,” Submitted Linear Algebra Appl., vol. 43, pp. 1–48, 2010.
[41]
Q. Wang, Q. Gao, X. Gao, and F. Nie, “$\ell _{2,p}$-Norm based PCA for image recognition,” IEEE Trans. Image Process., vol. 27, no. 3, pp. 1336–1346, Mar. 2018.
[42]
S. Liao, J. Li, Y. Liu, Q. Gao, and X. Gao, “Robust formulation for PCA: Avoiding mean calculation with $\ell _{2,p}$-norm maximization,” in Proc. Thirty-Second AAAI Conf. Artif. Intell., 2018, pp. 3604–3610.
[43]
M. Yang, Q. Luo, W. Li, and M. Xiao, “Multiview clustering of images with tensor rank minimization via nonconvex approach,” SIAM J. Imag. Sci., vol. 13, no. 4, pp. 2361–2392, 2020.
[44]
J. M. Winn and N. Jojic, “LOCUS: Learning object classes with unsupervised segmentation,” in Proc. 10th IEEE Int. Conf. Comput. Vis., 2005, pp. 756–763.
[45]
D. Dua and C. Graff, “UCI machine learning repository,” 2017. [Online]. Available: http://archive.ics.uci.edu/ml
[46]
L. Deng, “The MNIST database of handwritten digit images for machine learning research [best of the web],” IEEE Signal Process. Mag., vol. 29, no. 6, pp. 141–142, Nov. 2012.
[47]
L. Fei-Fei, R. Fergus, and P. Perona, “Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories,” Comput. Vis. Image Understanding, vol. 106, no. 1, pp. 59–70, 2007.
[48]
C. Apté, F. Damerau, and S. M. Weiss, “Automated learning of decision rules for text categorization,” ACM Trans. Inf. Syst., vol. 12, no. 3, pp. 233–251, 1994.
[49]
F. Nie, J. Li, and X. Li, “Self-weighted multiview clustering with multiple graphs,” in Proc. Int. Joint Conf. Artif. Intell., 2017, pp. 2564–2570.
[50]
F. Nie, G. Cai, J. Li, and X. Li, “Auto-weighted multi-view learning for image clustering and semi-supervised classification,” IEEE Trans. Image Process., vol. 27, no. 3, pp. 1501–1511, Mar. 2018.
[51]
S. Luo, C. Zhang, W. Zhang, and X. Cao, “Consistent and specific multi-view subspace clustering,” in Proc. AAAI Conf. Artif. Intell., 2018, pp. 3730–3737.
[52]
Z. Kang et al., “Large-scale multi-view subspace clustering in linear time,” in Proc. AAAI Conf. Artif. Intell., 2020, pp. 4412–4419.
[53]
S. Wang et al., “Fast parameter-free multi-view subspace clustering with consensus anchor guidance,” IEEE Trans. Image Process., vol. 31, pp. 556–568, 2022.
[54]
D. Ming, C. Ding, and F. Nie, “A probabilistic derivation of LASSO and L12-norm feature selections,” in Proc. AAAI Conf. Artif. Intell., 2019, pp. 4586–4593.
Recommendations
One-step graph-based incomplete multi-view clustering
AbstractExisting graph-based incomplete multi-view clustering methods mainly adopt the three-step strategy, i.e., graph completion, graph fusion (consensus representation learning) and subsequent k-means clustering. Such three-step schemes inevitably seek ...
Discrete correntropy-based multi-view anchor-graph clustering
AbstractGraph-based clustering commonly provides promising clustering effectiveness as it can preserve samples’ local geometric information. Inspired by it, multi-view graph clustering was developed to integrate complementary information among graphs of ...
Highlights- A new discrete correntropy-based multi-view anchor-graph clustering (DCMAC) model is proposed.
- A novel half-quadratic-based coordinate descent algorithm is proposed to solve the DCMAC model.
- To handle single-view clustering ...
Latent information-guided one-step multi-view fuzzy clustering based on cross-view anchor graph
AbstractAlthough graph-inspired clustering methods have achieved impressive success in the area of multi-view data analysis, current methods still face several challenges. First, classical graph construction approaches are computationally expensive in ...
Highlights- A novel one-step multi-view fuzzy clustering method is proposed.
- The final clustering result is generated without an additional post-processing step.
- The consensus matrix optimization is guided by the cross-view latent information.
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
1520-9210 © 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Press
Publication History
Published: 26 February 2024
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Oct 2024
Other Metrics
Citations
Cited By
View allView Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in