Abstract
We propose relevance learning for unsupervised online vector quantization algorithm based on stochastic gradient descent learning according to the given vector quantization cost function. We consider several widely used models including the neural gas algorithm, the Heskes variant of self-organizing maps and the fuzzy c-means. We apply the relevance learning scheme for divergence based similarity measures between prototypes and data vectors in the vector quantization schemes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Arnonkijpanich, B., Hasenfuss, A., Hammer, B.: Local matrix adaptation in topographic neural maps. Neurocomputing 74(4), 522–539 (2011)
Backhaus, A., Kuwabara, A., Bauch, M., Monk, N., Sanguinetti, G., Fleming, A.: LEAFPROCESSOR: a new leaf phenotyping tool using contour bending energy and shape cluster analysis. New Phytologist 187(1), 251–261 (2010)
Bezdek, J.: Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum, New York (1981)
Bouveyron, C., Girard, S., Schmid, C.: High-dimensional data clustering. Computational Statistics and Data Analysis 57(1), 502–519 (2007)
Bunte, K., Hammer, B., Wismüller, A., Biehl, M.: Adaptive local dissimilarity measures for discriminative dimension reduction of labeled data. Neurocomputing 73, 1074–1092 (2010)
Camps-Valls, G., Bruzzone, L.: Kernel-based methods for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 43(6), 1351–1362 (2005)
Cesar, R.M., da Fontoura Costa, L.: Application and assessment of multiscale bending energy for morphometric characterization of neural cells. Rev. Sci. Instrum. 68(5), 2177–2186 (1997)
Cichocki, A., Amari, S.-I.: Families of alpha- beta- and gamma- divergences: Flexible and robust measures of similarities. Entropy 12, 1532–1568 (2010)
Dunn, J.: A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters. Journal of Cybernetics 3, 32–57 (1973)
Hammer, B., Villmann, T.: Generalized relevance learning vector quantization. Neural Networks 15(8-9), 1059–1068 (2002)
Heskes, T.: Energy functions for self-organizing maps. In: Oja, E., Kaski, S. (eds.) Kohonen Maps, pp. 303–316. Elsevier, Amsterdam (1999)
Izakian, H., Abraham, A.: Fuzzy c-means and fuzzy swarm for fuzzy clustering problem. Expert Systems with Applications 38(3), 1835–1838 (2011)
Kass, M., Witkin, A., Terzopoulos, D.: Snakes - Active Contour models. International Journal of Computer Vision 1(4), 321–331 (1987)
Kato, T.: On the adiabatic theorem of quantum mechanics. Journal of the Physical Society of Japan 5(6), 435–439 (1950)
Kohonen, T.: Self-Organizing Maps. Springer Series in Information Sciences, vol. 30. Springer, Heidelberg (1995), 2nd extended edn. (1997)
Landgrebe, D.: Signal Theory Methods in Multispectral Remote Sensing. Wiley, Hoboken, New Jersey (2003)
Lee, J., Verleysen, M.: Generalization of the lp norm for time series and its application to self-organizing maps. In: Cottrell, M. (ed.) Proc. of Workshop on Self-Organizing Maps (WSOM) 2005, Paris, Sorbonne, pp. 733–740 (2005)
Liu, Z.-Y., Xu, L.: Topological local principal component analysis. Neurocomputing 55(3-4), 739–745 (2003)
López-Rubio, E., noz Pérez, J.M., Gómez-Ruiz, J.: A principal components analysis self-organizing map. Neural Networks 2(2), 261–270 (2004)
Martinetz, T.M., Berkovich, S.G., Schulten, K.J.: ‘Neural-gas’ network for vector quantization and its application to time-series prediction. IEEE Trans. on Neural Networks 4(4), 558–569 (1993)
Möller, R., Hoffmann, H.: An extension of neural gas to local PCA. Neurocomputing 62, 305–326 (2004)
Mwebaze, E., Schneider, P., Schleif, F.-M., Aduwo, J., Quinn, J., Haase, S., Villmann, T., Biehl, M.: Divergence based classification in learning vector quantization. Neurocomputing 74(9), 1429–1435 (2011)
Oja, E., Lampinen, J.: Unsupervised learning for feature extraction. In: Zurada, J.M., Marks II, R.J., Robinson, C.J. (eds.) Computational Intelligence Imitating Life, pp. 13–22. IEEE Press, Los Alamitos (1994)
Principe, J.C., Fisher III, J., Xu, D.: Information theoretic learning. In: Haykin, S. (ed.) Unsupervised Adaptive Filtering, Wiley, New York (2000)
Rubner, J., Tavan, P.: A self-organizing network for principle-component analysis. Europhys. Letters 7(10), 693–698 (1989)
Schneider, P., Hammer, B., Biehl, M.: Adaptive relevance matrices in learning vector quantization. Neural Computation 21, 3532–3561 (2009)
Tipping, M., Bishop, C.: Mixtures of probabilistic principal component analyzers. Neural Computation 11, 443–482 (1999)
van Hulle, M.M.: Faithful representations with topographic maps. Neural Networks 12(6), 803–823 (1999)
Villmann, T., Cichocki, A., Principe, J.: Information theory related learning. In: Verleysen, M. (ed.) Proc. of European Symposium on Artificial Neural Networks (ESANN 2011), Evere, Belgium, d-side publications (2011) (page in press)
Villmann, T., Der, R., Herrmann, M., Martinetz, T.: Topology Preservation in Self-Organizing Feature Maps: Exact Definition and Measurement. IEEE Transactions on Neural Networks 8(2), 256–266 (1997)
Villmann, T., Haase, S.: Divergence based vector quantization. Neural Computation 23(5), 1343–1392 (2011)
Villmann, T., Hammer, B.: Theoretical aspects of kernel GLVQ with differentiable kernel. IfI Technical Report Series (IfI-09-12), pp. 133–141 (2009)
Villmann, T., Schleif, F.-M.: Functional vector quantization by neural maps. In: Chanussot, J. (ed.) Proceedings of First Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS 2009), pp. 1–4. IEEE Press, Los Alamitos (2009); ISBN 978-1-4244-4948-4
Xie, X., Beni, G.: A validity measure for fuzzy clustering. IEEE Transactions on Pat 13(8), 841–847 (1991)
Xu, R., Wunsch, D.: Survey of clustering algorithms. IEEE Transactions on Neural Networks 16(3), 645–678 (2005)
Young, I.T., Walker, J.E., Bowie, J.E.: An analysis technique for biological shape. i. Information and Control 25(4), 357–370 (1974)
Zalik, K., Zalik, B.: Validity index for clusters of different sizes and densities. Pattern Recognition Letters 32, 221–234 (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kästner, M., Backhaus, A., Geweniger, T., Haase, S., Seiffert, U., Villmann, T. (2011). Relevance Learning in Unsupervised Vector Quantization Based on Divergences. In: Laaksonen, J., Honkela, T. (eds) Advances in Self-Organizing Maps. WSOM 2011. Lecture Notes in Computer Science, vol 6731. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21566-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-642-21566-7_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21565-0
Online ISBN: 978-3-642-21566-7
eBook Packages: Computer ScienceComputer Science (R0)