research-article

Public Access

Robust Embedded Deep K-means Clustering

Authors:

Yada ZhuAuthors Info & Claims

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Pages 1181 - 1190

https://doi.org/10.1145/3357384.3357985

Published: 03 November 2019 Publication History

Abstract

Deep neural network clustering is superior to the conventional clustering methods due to deep feature extraction and nonlinear dimensionality reduction. Nevertheless, deep neural network leads to a rough representation regarding the inherent relationship of the data points. Therefore, it is still difficult for deep neural network to exploit the effective structure for direct clustering. To address this issue, we propose a robust embedded deep K-means clustering (RED-KC) method. The proposed RED-KC approach utilizes the δ-norm metric to constrain the feature mapping process of the auto-encoder network, so that data are mapped to a latent feature space, which is more conducive to the robust clustering. Compared to the existing auto-encoder networks with the fixed prior, the proposed RED-KC is adaptive during the process of feature mapping. More importantly, the proposed RED-KC embeds the clustering process with the auto-encoder network, such that deep feature extraction and clustering can be performed simultaneously. Accordingly, a direct and efficient clustering could be obtained within only one step to avoid the inconvenience of multiple separate stages, namely, losing pivotal information and correlation. Consequently, extensive experiments are provided to validate the effectiveness of the proposed approach.

References

[1]

Jianlong Chang, Lingfeng Wang, Gaofeng Meng, Shiming Xiang, and Chunhong Pan. 2017. Deep adaptive image clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 5879--5887.

[2]

Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. 2018. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 40, 4 (2018), 834--848.

[3]

Bin Cheng, Guangcan Liu, Jingdong Wang, Zhongyang Huang, and Shuicheng Yan. 2011. Multi-task low-rank affinity pursuit for image segmentation. In IEEE International Conference on Computer Vision (ICCV). IEEE, 2439--2446.

Digital Library

[4]

Nat Dilokthanakul, Pedro AM Mediano, Marta Garnelo, Matthew CH Lee, Hugh Salimbeni, Kai Arulkumaran, and Murray Shanahan. 2016. Deep unsupervised clustering with gaussian mixture variational autoencoders. arXiv preprint arXiv:1611.02648 (2016).

[5]

Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. 2014. Learning a deep convolutional network for image super-resolution. In Proceedings of the European Conference on Computer Vision (ECCV) . Springer, 184--199.

[6]

Aysegul Dundar, Jonghoon Jin, and Eugenio Culurciello. 2015. Convolutional clustering for unsupervised learning. arXiv preprint arXiv:1511.06241 (2015).

[7]

Ehsan Elhamifar and Rene Vidal. 2013. Sparse subspace clustering: Algorithm, theory, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, 11 (2013), 2765--2781.

Digital Library

[8]

Adil Fahad, Najlaa Alshatri, Zahir Tari, Abdullah Alamri, Ibrahim Khalil, Albert Y. Zomaya, Sebti Foufou, and Abdelaziz Bouras. 2014. A survey of clustering algorithms for big data: Taxonomy and empirical analysis. IEEE Transactions on Emerging Topics in Computing, Vol. 2, 3 (2014), 267--279.

[9]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems (NIPS). 2672--2680.

[10]

Muhan Guo, Rui Zhang, Feiping Nie, and Xuelong Li. 2018. Embedding fuzzy k-means with nonnegative spectral clustering via incorporating side information. In ACM International Conference on Information and Knowledge Management (CIKM). 1567--1570.

Digital Library

[11]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 770--778.

[12]

John R. Hershey, Zhuo Chen, Jonathan Le Roux, and Shinji Watanabe. 2016. Deep clustering: Discriminative embeddings for segmentation and separation. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 31--35.

Digital Library

[13]

Junlin Hu, Jiwen Lu, and Yap-Peng Tan. 2014. Discriminative deep metric learning for face verification in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1875--1882.

Digital Library

[14]

Pan Ji, Mathieu Salzmann, and Hongdong Li. 2014. Efficient dense subspace clustering. In IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 461--468.

[15]

Pan Ji, Tong Zhang, Hongdong Li, Mathieu Salzmann, and Ian Reid. 2017. Deep subspace clustering networks. In Advances in Neural Information Processing Systems (NIPS). 24--33.

[16]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (NIPS). 1097--1105.

Digital Library

[17]

Marc T. Law, Raquel Urtasun, and Richard S. Zemel. 2017. Deep spectral clustering learning. In International Conference on Machine Learning (ICML). 1985--1994.

[18]

Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew P. Aitken, Alykhan Tejani, Johannes Totz, and Zehan Wang. 2017. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 105--114.

[19]

Kuang-Chih Lee, Jeffrey Ho, and David J. Kriegman. 2005. Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, 5 (2005), 684--698.

Digital Library

[20]

Xuelong Li, Han Zhang, Rui Zhang, Yun Liu, and Feiping Nie. 2019. Generalized Uncorrelated Regression with Adaptive Graph for Unsupervised Feature Selection. IEEE Transactions on Neural Networks and Learning Systems, Vol. 30, 5 (2019), 1587--1595.

[21]

Jie Liang, Jufeng Yang, Hsin-Ying Lee, Kai Wang, and Ming-Hsuan Yang. 2018. Sub-GAN: An unsupervised generative model via subspaces. In Proceedings of the European Conference on Computer Vision (ECCV). 698--714.

[22]

Wei-An Lin, Jun-Cheng Chen, Carlos D. Castillo, and Rama Chellappa. 2018. Deep density clustering of unconstrained faces. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 8128--8137.

[23]

Guangcan Liu, Zhouchen Lin, Shuicheng Yan, Ju Sun, Yong Yu, and Yi Ma. 2013. Robust recovery of subspace structures by low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, 1 (2013), 171--184.

Digital Library

[24]

Can-Yi Lu, Hai Min, Zhong-Qiu Zhao, Lin Zhu, De-Shuang Huang, and Shuicheng Yan. 2012. Robust and efficient subspace segmentation via least squares regression. In Proceedings of the European Conference on Computer Vision (ECCV) . Springer, 347--360.

Digital Library

[25]

Benjamin J. Meyer, Ben Harwood, and Tom Drummond. 2018. Deep metric learning and image classification with nearest neighbour gaussian kernels. In IEEE International Conference on Image Processing (ICIP). IEEE, 151--155.

[26]

Erxue Min, Xifeng Guo, Qiang Liu, Gen Zhang, Jianjing Cui, and Jun Long. 2018. A survey of clustering with deep learning: From the perspective of network architecture. IEEE Access, Vol. 6 (2018), 39501--39514.

[27]

Sameer A Nene, Shree K Nayar, Hiroshi Murase, et almbox. 1996. Columbia object image library (coil-20). (1996).

[28]

Andrew Y. Ng, Michael I. Jordan, and Yair Weiss. 2002. On spectral clustering: Analysis and an algorithm. In Advances in Neural Information Processing Systems (NIPS). 849--856.

[29]

Feiping Nie, Zhanxuan Hu, and Xuelong Li. 2018. Calibrated multi-task learning. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD). 2012--2021.

Digital Library

[30]

Feiping Nie, Sheng Yang, Rui Zhang, and Xuelong Li. 2019. A General Framework for Auto-Weighted Feature Selection via Global Redundancy Minimization. IEEE Transactions on Image Processing, Vol. 28, 5 (2019), 2428--2438.

[31]

Vishal M. Patel, Hien Van Nguyen, and René Vidal. 2013. Latent space sparse subspace clustering. In Proceedings of the IEEE International Conference on Computer Vision (ICCV). 225--232.

Digital Library

[32]

Vishal M. Patel and René Vidal. 2014. Kernel sparse subspace clustering. In IEEE International Conference on Image Processing (ICIP). IEEE, 2849--2853.

[33]

Xi Peng, Shijie Xiao, Jiashi Feng, Wei-Yun Yau, and Zhang Yi. 2016. Deep subspace clustering with sparsity prior. In International Joint Conferences on Artificial Intelligence (IJCAI). 1925--1931.

[34]

Xi Peng, Zhang Yi, and Huajin Tang. 2015. Robust subspace clustering via thresholding ridge regression. In AAAI Conference on Artificial Intelligence (AAAI). 3827--3833.

[35]

Alex Rodriguez and Alessandro Laio. 2014. Clustering by fast search and find of density peaks. Science, Vol. 344, 6191 (2014), 1492--1496.

[36]

Jianbo Shi and Jitendra Malik. 2000. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, 8 (2000), 888--905.

Digital Library

[37]

Chunfeng Song, Feng Liu, Yongzhen Huang, Liang Wang, and Tieniu Tan. 2013. Auto-encoder based data clustering. In Iberoamerican Congress on Pattern Recognition. Springer, 117--124.

Digital Library

[38]

Ulrike Von Luxburg. 2007. A tutorial on spectral clustering. Statistics and Computing, Vol. 17, 4 (2007), 395--416.

Digital Library

[39]

Wutao Wei, Bowei Xi, and Murat Kantarcioglu. 2018. Adversarial clustering: A grid based clustering algorithm against active adversaries . arXiv preprint arXiv:1804.04780 (2018).

[40]

Junyuan Xie, Ross Girshick, and Ali Farhadi. 2016. Unsupervised deep embedding for clustering analysis. In International Conference on Machine Learning (ICML). 478--487.

[41]

Ming Yin, Yi Guo, Junbin Gao, Zhaoshui He, and Shengli Xie. 2016. Kernel sparse subspace clustering on symmetric positive definite manifolds. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5157--5164.

[42]

Hongyang Zhang, Zhouchen Lin, Chao Zhang, and Junbin Gao. 2014. Robust latent low rank representation for subspace clustering. Neurocomputing, Vol. 145 (2014), 369--373.

[43]

Rui Zhang, Feiping Nie, Muhan Guo, Xian Wei, and Xuelong Li. 2019 b. Joint learning of fuzzy k-means and nonnegative spectral clustering with side information. IEEE Transactions on Image Processing, Vol. 28, 5 (2019), 2152--2162.

Digital Library

[44]

Rui Zhang, Feiping Nie, and Xuelong Li. 2017a. Embedded clustering via robust orthogonal least square discriminant analysis. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2332--2336.

[45]

Rui Zhang, Feiping Nie, and Xuelong Li. 2017b. Self-weighted spectral clustering with parameter-free constraint. Neurocomputing, Vol. 241 (2017), 164--170.

[46]

Rui Zhang, Feiping Nie, and Xuelong Li. 2018. Self-weighted supervised discriminative feature selection. IEEE Transactions on Neural Networks and Learning Systems, Vol. 29, 8 (2018), 3913--3918.

[47]

Rui Zhang, Feiping Nie, and Xuelong Li. 2019 a. Semisupervised learning with parameter-free similarity of label and side information . IEEE Transactions on Neural Networks and Learning Systems, Vol. 30, 2 (2019), 405--414.

[48]

Rui Zhang, Feiping Nie, Xuelong Li, and Xian Wei. 2019 c. Feature selection with multi-view data: A survey. Information Fusion, Vol. 50 (2019), 158--167.

[49]

Rui Zhang, Feiping Nie, Haiyang Wang, and Xuelong Li. 2019 d. Unsupervised Feature Selection via Adaptive Multi-Measure Fusion. IEEE Transactions on Neural Networks and Learning Systems (2019).

[50]

Pan Zhou, Yunqing Hou, and Jiashi Feng. 2018. Deep adversarial subspace clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 1596--1604.

Cited By

Rass SKönig SAhmad SGoman M(2024)Metricizing the Euclidean Space Toward Desired Distance Relations in Point CloudsIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.342024619(7304-7319)Online publication date: 2024
https://doi.org/10.1109/TIFS.2024.3420246
Hong XPapazachos Zdel Rincon JMiller P(2024)Network Intrusion Detection by Variational Component-Based Feature Saliency Gaussian Mixture ClusteringComputer Security. ESORICS 2023 International Workshops10.1007/978-3-031-54129-2_45(761-772)Online publication date: 12-Mar-2024
https://doi.org/10.1007/978-3-031-54129-2_45
Shad HZamiri MBahraini TMonsefi RHodtani G(2023)Information Theoretic Learning-based Deep Embedded Clustering (ITL-DEC)2023 13th International Conference on Computer and Knowledge Engineering (ICCKE)10.1109/ICCKE60553.2023.10326317(439-445)Online publication date: 1-Nov-2023
https://doi.org/10.1109/ICCKE60553.2023.10326317
Show More Cited By

Index Terms

Robust Embedded Deep K-means Clustering
1. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Unsupervised learning and clustering

Recommendations

Deep multi-view fuzzy k-means with weight allocation and entropy regularization
Abstract
Multi-view clustering is a rapidly evolving research topic that exploits cross-view data obtained from different domains or modalities to describe the target object more comprehensively. Currently, many multi-view deep clustering algorithms are ...
Learning Discriminatory Deep Clustering Models
Computer Analysis of Images and Patterns
Abstract
Deep convolutional auto-encoder (DCAE) allows to obtain useful features via its internal layer and provide an abstracted latent representation, which has been exploited for clustering analysis. DCAE allows a deep clustering method to extract ...
Hybrid Bisect K-Means Clustering Algorithm
BCGIN '11: Proceedings of the 2011 International Conference on Business Computing and Global Informatization

In this paper, we present a hybrid clustering algorithm that combines divisive and agglomerative hierarchical clustering algorithm. Our method uses bisect K-means for divisive clustering algorithm and Unweighted Pair Group Method with Arithmetic Mean (...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

November 2019

3373 pages

ISBN:9781450369763

DOI:10.1145/3357384

General Chairs:
Wenwu Zhu
Tsinghua University, China
,
Dacheng Tao
University of Massachusetts, USA
,
Xueqi Cheng
Institute of Computing Technology, CAS, China
,
Program Chairs:
Peng Cui
Tsinghua University, China
,
Elke Rundensteiner
Worcester Polytechnic Institute, USA
,
David Carmel
Amazon Research, USA
,
Qi He
LinkedIn, USA
,
Jeffrey Xu Yu
Chinese University of Hong Kong, China

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

DHS
NSF

Conference

CIKM '19

Sponsor:

CIKM '19: The 28th ACM International Conference on Information and Knowledge Management

November 3 - 7, 2019

Beijing, China

Acceptance Rates

CIKM '19 Paper Acceptance Rate 202 of 1,031 submissions, 20%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
964
Total Downloads

Downloads (Last 12 months)142
Downloads (Last 6 weeks)26

Reflects downloads up to 24 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Rass SKönig SAhmad SGoman M(2024)Metricizing the Euclidean Space Toward Desired Distance Relations in Point CloudsIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.342024619(7304-7319)Online publication date: 2024
https://doi.org/10.1109/TIFS.2024.3420246
Hong XPapazachos Zdel Rincon JMiller P(2024)Network Intrusion Detection by Variational Component-Based Feature Saliency Gaussian Mixture ClusteringComputer Security. ESORICS 2023 International Workshops10.1007/978-3-031-54129-2_45(761-772)Online publication date: 12-Mar-2024
https://doi.org/10.1007/978-3-031-54129-2_45
Shad HZamiri MBahraini TMonsefi RHodtani G(2023)Information Theoretic Learning-based Deep Embedded Clustering (ITL-DEC)2023 13th International Conference on Computer and Knowledge Engineering (ICCKE)10.1109/ICCKE60553.2023.10326317(439-445)Online publication date: 1-Nov-2023
https://doi.org/10.1109/ICCKE60553.2023.10326317
Abrar SSekmen ASamad M(2023)Effectiveness of Deep Image Embedding Clustering Methods on Tabular Data2023 15th International Conference on Advanced Computational Intelligence (ICACI)10.1109/ICACI58115.2023.10146161(1-7)Online publication date: 6-May-2023
https://doi.org/10.1109/ICACI58115.2023.10146161
Li YXie X(2023)Deep multi-view fuzzy k-means with weight allocation and entropy regularizationApplied Intelligence10.1007/s10489-023-05113-253:24(30593-30606)Online publication date: 22-Nov-2023
https://doi.org/10.1007/s10489-023-05113-2
Zhao PZhang YMa YZhao XFan X(2023)Discriminatively embedded fuzzy K-Means clustering with feature selection strategyApplied Intelligence10.1007/s10489-022-04376-553:16(18959-18970)Online publication date: 15-Feb-2023
https://doi.org/10.1007/s10489-022-04376-5
Gong FNie YXu HAl Hasan MXiong L(2022)Gromov-Wasserstein Multi-modal Alignment and ClusteringProceedings of the 31st ACM International Conference on Information & Knowledge Management10.1145/3511808.3557339(603-613)Online publication date: 17-Oct-2022
https://dl.acm.org/doi/10.1145/3511808.3557339
Nie FZhao XWang RLi XLi Z(2022)Fuzzy K-Means Clustering With Discriminative EmbeddingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2020.299574834:3(1221-1230)Online publication date: 1-Mar-2022
https://doi.org/10.1109/TKDE.2020.2995748
Wang RLi LWang PTao XLiu P(2022)Deep boundary-aware clustering by jointly optimizing unsupervised representation learningMultimedia Tools and Applications10.1007/s11042-021-11597-281:24(34309-34324)Online publication date: 24-Jan-2022
https://doi.org/10.1007/s11042-021-11597-2
Huang HYoo SXu CShen HZhuang YSmith JYang YCesar PMetze FPrabhakaran B(2021)Deep Clustering based on Bi-Space Association LearningProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3475270(12-21)Online publication date: 17-Oct-2021
https://dl.acm.org/doi/10.1145/3474085.3475270
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents