research-article

Fuzzy PCA-guided robust k-means clustering

Authors:

Katsuhiro Honda,

Hidetomo IchihashiAuthors Info & Claims

IEEE Transactions on Fuzzy Systems, Volume 18, Issue 1

Pages 67 - 79

https://doi.org/10.1109/TFUZZ.2009.2036603

Published: 01 February 2010 Publication History

Abstract

This paper proposes a new approach to robust clustering, in which a robust k-means partition is derived by using a noise-rejection mechanism based on the noise-clustering approach. The responsibility weight of each sample for the k-means process is estimated by considering the noise degree of the sample, and cluster indicators are calculated in a fuzzy principal-component-analysis (PCA) guided manner, where fuzzy PCA-guided robust k-means is performed by considering responsibility weights of samples. Then, the proposed method achieves cluster-core estimation in a deterministic way. The validity of the derived cluster cores is visually assessed through distance-sensitive ordering, which considers responsibility weights of samples. Numerical experiments demonstrate that the proposed method is useful for capturing cluster cores by rejecting noise samples, and we can easily assess cluster validity by using cluster-crossing curves.

References

[1]

C. Ding and X. He, "K-means clustering via principal component analysis," in Proc. Int. Conf. Mach. Learning, 2004, pp. 225-232.

Digital Library

[2]

J. B. MacQueen, "Some methods of classification and analysis of multivariate observations," in Proc. 5th Berkeley Symp. Math. Stat. Prob., 1967, pp. 281-297.

[3]

A. Nasser and D. Hamad, "K-means clustering algorithm in projected spaces," in Proc. 9th Int. Conf. Inf. Fusion, 2006, pp. 1-6.

[4]

K. Weike, P. Azad, and R. Dillmann, "Fast and robust feature-based recognition of multiple objects," in Proc. 6th IEEE-RAS Int. Conf. Humanoid Robots, 2006, pp. 264-269.

[5]

K. Fukunaga and D. R. Olsen, "An algorithm for finding intrinsic dimensionality of data," IEEE Trans. Comput., vol. C-20, no. 2, pp. 176-183, Feb. 1971.

Digital Library

[6]

G. E. Hinton, P. Dayan, and M. Revow, "Modeling the manifolds of images of handwritten digits," IEEE Trans. Neural Netw., vol. 8, no. 1, pp. 65-74, Jan. 1997.

Digital Library

[7]

J.-H. Na, M.-S. Park, and J.-Y. Choi, "Pre-clustered principal component analysis for fast training of new face databases," in Proc. Int. Conf. Control, Autom. Syst., 2007, pp. 1144-1149.

[8]

M. Xu and P. Fränti, "A heuristic K-means clustering algorithm by kernel PCA," in Proc. Int. Conf. Image Process., 2004, vol. 5, pp. 3503-3506.

[9]

T. Su and J. Dy, "A deterministic method for initializing K-means clustering," in Proc. 16th IEEE Int. Conf. Tools Artif. Intell., 2004, pp. 784- 786.

Digital Library

[10]

Y. Yabuuchi and J. Watada, "Fuzzy principal component analysis and its application," Biomed. Fuzzy Human Sci., vol. 3, pp. 83-92, 1997.

[11]

R. N. Davé, "Characterization and detection of noise in clustering," Pattern Recognit. Lett., vol. 12, no. 11, pp. 657-664, 1991.

Digital Library

[12]

R. N. Davé and R. Krishnapuram, "Robust clustering methods: A unified view," IEEE Trans. Fuzzy Syst., vol. 5, no. 2, pp. 270-293, May 1997.

Digital Library

[13]

P. J. Huber, Robust Statistics. New York: Wiley, 1981.

[14]

J. A. Cuesta-Albertos, A. Gordaliza, and D. C. Matrán, "Trimmed k- means: An attempt to robustify quantizers," Ann. Stat., vol. 25, no. 2, pp. 553-576, 1997.

[15]

L. A. Garcia-Escudero and A. Gordaliza, "Robustness properties of k- mean and trimmed k-means," J. Amer. Stat. Assoc., vol. 94, no. 447, pp. 956-969, Sep. 1999.

[16]

M.-S. Yang, K.-L. Wu, J.-N. Hsieh, and J. Yu, "Alpha-cut implemented fuzzy clustering algorithms and switching regressions," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 3, pp. 588-603, Jun. 2008.

Digital Library

[17]

N. R. Pal, K. Pal, J. M. Keller, and J. C. Bezdek, "A possibilistic fuzzy c-means clustering algorithm," IEEE Trans. Fuzzy Syst., vol. 13, no. 4, pp. 508-516, Aug. 2005.

Digital Library

[18]

R. Krishnapuram and J.M. Keller, "A possibilistic approach to clustering," IEEE Trans. Fuzzy Syst., vol. 1, no. 2, pp. 98-110, May 1993.

Digital Library

[19]

F. Masulli and S. Rovetta, "Soft transition from probabilistic to possibilistic fuzzy clustering," IEEE Trans. Fuzzy Syst., vol. 14, no. 4, pp. 516-527, Aug. 2006.

Digital Library

[20]

K. Tsuda, M. Minoh, and K. Ikeda, "Extracting straight lines by sequential fuzzy clustering," Pattern Recognit. Lett., vol. 17, pp. 643-649, 1996.

Digital Library

[21]

K. Inoue and K. Urahama, "Sequential fuzzy cluster extraction by a graph spectral method," Pattern Recognit. Lett., vol. 20, pp. 699-705, 1999.

Digital Library

[22]

U. Luxburg, "A tutorial on spectral clustering," Stat. Comput., vol. 17, no. 4, pp. 395-416, 2007.

Digital Library

[23]

H. Zha, C. Ding, M. Gu, X. He, and H. Simon, "Spectral relaxation for K-means clustering," in Proc. Adv. Neural Inf. Process. Syst. 14, 2002, pp. 1057-1064.

[24]

C. Ding and X. He, "Linearized cluster assignment via spectral ordering," in Proc. Int. Conf. Mach. Learning, 2004, pp. 233-240.

Digital Library

[25]

J. C. Bezdek, Pattern Recognition With Fuzzy Objective Function Algorithms. New York: Plenum, 1981.

Digital Library

[26]

C. M. Bishop, Pattern Recognition and Machine Learning. New York: Springer-Verlag, 2006.

Digital Library

[27]

S. Miyamoto, H. Ichihashi, and K. Honda, Algorithms for Fuzzy Clustering. Berlin Heidelberg: Springer-Verlag, 2008.

Digital Library

[28]

B. Schölkopf, A. Smola, and K. Müller, "Nonlinear component analysis as a kernel eigenvalue problem," Neural Comput., vol. 10, no. 5, pp. 1299- 1319, 1998.

Digital Library

[29]

A. Asuncion and D. J. Newman. (2007). UCI machine learning repository. School Inf. Comput. Sci., Univ. Calif., Irvine {Online}. Available: http://www.ics.uci.edu/mlearn/MLRepository.html

[30]

S. A. Nene, S. K. Nayar, and H. Murase, "Columbia object image library (COIL-20)," Dept. Comput. Sci., Columbia Univ., New York, Tech. Rep. CUCS-005-96, 1996.

[31]

J. C. Bezdek, R. J. Hathaway, and J. M. Huband, "Visual assessment of clustering tendency for rectangular dissimilarity matrices," IEEE Trans. Fuzzy Syst., vol. 15, no. 5, pp. 890-903, Oct. 2007.

Digital Library

[32]

K. Honda and H. Ichihashi, "Linear fuzzy clustering techniques with missing values and their application to local principal component analysis," IEEE Trans. Fuzzy Systems, vol. 12, no. 2, pp. 183-193, Apr. 2004.

Digital Library

[33]

K. Honda and H. Ichihashi, "Regularized linear fuzzy clustering and probabilistic PCA mixture models," IEEE Trans. Fuzzy Syst., vol. 13, no. 4, pp. 508-516, Aug. 2005.

Digital Library

[34]

K. Honda, H. Ichihashi, F. Masulli, and S. Rovetta, "Linear fuzzy clustering with selection of variables using graded possibilistic approach," IEEE Trans. Fuzzy Syst., vol. 15, no. 5, pp. 878-889, Oct. 2007.

Digital Library

Cited By

Kim TMolina MRheu MZhan EPeng W(2023)One AI Does Not Fit All: A Cluster Analysis of the Laypeople’s Perception of AI RolesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581340(1-20)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581340
Okabe AHonda KUbukata S(2022)Noise Fuzzy Clustering-Based Robust Non-negative Matrix Factorization with I-divergence CriterionIntegrated Uncertainty in Knowledge Modelling and Decision Making10.1007/978-3-030-98018-4_21(256-266)Online publication date: 18-Mar-2022
https://dl.acm.org/doi/10.1007/978-3-030-98018-4_21
Honda KHoshii KUbukata SNotsuy A(2020)A Noise Rejection Mechanism for pLSA-induced Fuzzy Co-clustering2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)10.1109/FUZZ48607.2020.9177597(1-8)Online publication date: 19-Jul-2020
https://dl.acm.org/doi/10.1109/FUZZ48607.2020.9177597
Show More Cited By

Fuzzy PCA-guided robust k-means clustering
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning

Recommendations

Agglomerative Fuzzy K-Means Clustering Algorithm with Selection of Number of Clusters

In this paper, we present an agglomerative fuzzy $k$-means clustering algorithm for numerical data, an extension to the standard fuzzy $k$-means algorithm by introducing a penalty term to the objective function to make the clustering process not ...
Improved k- means clustering algorithm for two dimensional data
CCSEIT '12: Proceedings of the Second International Conference on Computational Science, Engineering and Information Technology

Clustering is a procedure of organizing the objects in groups whose member exhibits some kind of similarity. So a cluster is a collection of objects which are alike and are different from the objects belonging to other clusters. K-Means is one of ...
Robust Rough-Fuzzy C-Means Algorithm: Design and Applications in Coding and Non-coding RNA Expression Data Clustering
Cognitive Informatics and Computational Intelligence: Theory and Applications

Cluster analysis is a technique that divides a given data set into a set of clusters in such a way that two objects from the same cluster are as similar as possible and the objects from different clusters are as dissimilar as possible. In this ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Fuzzy Systems

IEEE Transactions on Fuzzy Systems Volume 18, Issue 1

February 2010

233 pages

ISSN:1063-6706

Issue’s Table of Contents

Copyright © 2010.

Publisher

IEEE Press

Publication History

Published: 01 February 2010

Accepted: 26 August 2009

Received: 20 November 2008

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kim TMolina MRheu MZhan EPeng W(2023)One AI Does Not Fit All: A Cluster Analysis of the Laypeople’s Perception of AI RolesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581340(1-20)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581340
Okabe AHonda KUbukata S(2022)Noise Fuzzy Clustering-Based Robust Non-negative Matrix Factorization with I-divergence CriterionIntegrated Uncertainty in Knowledge Modelling and Decision Making10.1007/978-3-030-98018-4_21(256-266)Online publication date: 18-Mar-2022
https://dl.acm.org/doi/10.1007/978-3-030-98018-4_21
Honda KHoshii KUbukata SNotsuy A(2020)A Noise Rejection Mechanism for pLSA-induced Fuzzy Co-clustering2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)10.1109/FUZZ48607.2020.9177597(1-8)Online publication date: 19-Jul-2020
https://dl.acm.org/doi/10.1109/FUZZ48607.2020.9177597
Battulga LLee SNasridinov AYoo K(2020)Hash-tree PCA: accelerating PCA with hash-based groupingThe Journal of Supercomputing10.1007/s11227-019-02947-x76:10(8248-8264)Online publication date: 1-Oct-2020
https://dl.acm.org/doi/10.1007/s11227-019-02947-x
Ueno MHonda KUbukata SNotsu A(2019)Robust Non-negative Matrix Factorization Based on Noise Fuzzy Clustering MechanismProceedings of the 2019 2nd Artificial Intelligence and Cloud Computing Conference10.1145/3375959.3375966(1-5)Online publication date: 21-Dec-2019
https://dl.acm.org/doi/10.1145/3375959.3375966
Abadpour A(2016)Incorporating spatial context into fuzzy-possibilistic clustering using Bayesian inferenceJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/IFS-15181130:2(895-919)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.3233/IFS-151811
Abadpour A(2016)Rederivation of the fuzzypossibilistic clustering objective function through Bayesian inferenceFuzzy Sets and Systems10.1016/j.fss.2015.10.005305:C(29-53)Online publication date: 15-Dec-2016
https://dl.acm.org/doi/10.1016/j.fss.2015.10.005
Nakano THonda KUbukata SNotsu A(2016)Exclusive Item Partition with Fuzziness Tuning in MMMs-Induced Fuzzy Co-clusteringIntegrated Uncertainty in Knowledge Modelling and Decision Making10.1007/978-3-319-49046-5_16(185-194)Online publication date: 30-Nov-2016
https://dl.acm.org/doi/10.1007/978-3-319-49046-5_16
Fazendeiro PValente de Oliveira J(2015)Observer-Biased Fuzzy ClusteringIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2014.230643423:1(85-97)Online publication date: 1-Feb-2015
https://dl.acm.org/doi/10.1109/TFUZZ.2014.2306434
Abadpour A(2015)A sequential Bayesian alternative to the classical parallel fuzzy clustering modelInformation Sciences: an International Journal10.1016/j.ins.2015.05.007318:C(28-47)Online publication date: 10-Oct-2015
https://dl.acm.org/doi/10.1016/j.ins.2015.05.007
Show More Cited By

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents