Abstract
We offer an efficient approach based on difference of convex functions (DC) optimization for self-organizing maps (SOM). We consider SOM as an optimization problem with a nonsmooth, nonconvex energy function and investigated DC programming and DC algorithm (DCA), an innovative approach in nonconvex optimization framework to effectively solve this problem. Furthermore an appropriate training version of this algorithm is proposed. The numerical results on many real-world datasets show the efficiency of the proposed DCA based algorithms on both quality of solutions and topographic maps.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Amerijckx C, Legaty JD, Verleysen M (2003) Image compression using self-organizing maps. Syst Anal Modell Simul 43(11):1529–1543
Argyrou A (2009) Clustering hierarchical data using self-organizing map: a graph-theoretical approach. Advances in self-organizing maps. Lecture Notes in Computer Science, vol 5629. Springer, Heidelberg, pp 19–27
Astudillo CA, Oommen BJ (2014) Topology-oriented self-organizing maps: a survey. Pattern Anal Appl 17:223–248
Barreto GA, Araúo AFR, Ritter HJ (2003) Self-organizing feature maps for modeling and control of robotic manipulators. J Intell Robot Syst 36(4):407–450
Bradley BS, Mangasarian OL (1998) Feature selection via concave minimization and support vector machines. In: Machine learning proceedings of the fifteenth international conferences (ICML’98), San Francisco, California, MorganKaufmann, pp 82–90
Burger M, Graepel T, Obermayer K (1997) Phase transitions in stochastic self-organizing maps. Phys Rev E 56:3876–3890
ChandraShekar BH, Shoba G (2009) Classification of documents using Kohonens self-organizing map. Int J Comput Theory Eng 1(5):610–613
Chang L, Jun-min L, Chong-xiu Y (2013) Skin detection using a modified self-organizing mixture network. In: Automatic face and gesture recognition (FG), 2013 10th IEEE international conference and workshops, vol 1, no 6, pp 22–26
Collobert R, Sinz F, Weston J, Bottou L (2006) Trading convexity for scalability. IN: International conference on machine learning ICML
de Carvalho FDA, Bertrand P, De Melo FM (2012) Batch self-organizing maps based on city-block distances for interval variables. Hal-00706519, version 1
Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc B 39:1–38
Dickerson KB, Ventura D (2009) Music recommendation and query-by-content using self-organizing maps. In: Proceedings of the international joint conference on neural networks, pp 705–710
Doan NQ, Azzag H, Lebbah M (2012) Self-organizing map and tree topology for graph summarization. Artificial neural networks and machine learning ICANN 2012. Lecture Notes in Computer Science, vol 7553. Springer, Heidelberg, pp 363–370
Dozono H, Tokushima H, Hara S, Noguchi Y (2005) An algorithm of SOM using simulated annealing in the batch update phase for sequence analysis. In: International workshop on self-organizing maps (WSOM), pp 171–178
Fiannaca A, Fatta GD, Gaglio S, Rizzo R, Urso AM (2007) Improved SOM learning using simulated annealing. Lecture Notes in Computer Science,vol 4668. Springer, Heidelberg, pp 279–288
Fort JC, Letremy P, Cottrell M (2002) Advantages and drawbacks of the Batch Kohonen algorithm. IN: The European symposium on artificial neural networks conference - ESANN, pp 223–230
Graef G, Schaefer C (2002) Application of ART2 networks and self-organizing maps to collaborative filtering. Hypermedia: openness, structural awareness, and adaptivity. Lecture Notes in Computer Science, vol 2266. Springer, Heidelberg, pp 296–309
Graepel T, Burger M, Obermayer K (1998) Self-organizing maps: generalizations and new optimization techniques. Neurocomputing 21(13):173–190
Günter S, Bunke H (2002) Self-organizing map for clustering in the graph domain. Pattern Recognit Lett 23(4):405–417
Guo X, Wang H, Glass DH (2013) Bayesian self-organizing map for data classification and clustering. Int J Wavelets Multiresolut Inf Process 11(5):91–102
Hagenbuchner M, Sperduti A, Tsoi AC (2009) Graph self-organizing maps for cyclic and unbounded graphs. Neurocomputing 72(79):1419–1430
Hagenauer J, Helbich M (2013) Hierarchical self-organizing maps for clustering spatiotemporal data. Int J Geograph Inform Sci 27(10):2026–2042
Heskes T (1999) Energy functions for self organizingmaps. In Oya S, Kaski E (eds) KohonenMaps. Elsevier, Amsterdam pp 303–316
Heskes T (2001) Self-organization maps, vector quantization, and mixture modeling. IEEE Trans Neural Netw 12(6):1299–1305
Hiriart Urruty JB, Lemarechal C (1993) Convex analysis and minimization algorithms. Springer Verlag, Berlin Heidelberg
Ismail S, Shabri A, Samsudin R (2012) A hybrid model of self organizing maps and least square support vector machine for river flow forecasting. Hydrol Earth Syst Sci 16:4417–4433
Kaski S, Lagus K (1996) Comparing self-organizing maps. Lecture Notes in Computer Science, vol 1112. Springer, Heidelberg, pp 809–814
Khalilia M, Popescu M (2014) Topology preservation in fuzzy self-organizing maps. Advance trends in soft computing Studies in Fuzziness and Soft Computing, vol 312. Springer, Heidelberg, pp 105–114
Kiviluoto K (1996) Topology preservation in self-organizing maps. In: Proceedings of ICNN 96, IEEE international conference on neural networks, vol 1, pp 294–299
Kohonen T (1982) Analysis of a simple self-organizing process. Biol Cybern 44:135–140
Kohonen T (1997) Self-organization maps. Springer, Heidelberg
Krause N, Singer Y (2004) Leveraging the margin more carefully. In: International conference on machine learning ICML
Le Thi HA (2005) DC programming and DCA. http://lita.sciences.univ-metz.fr/~lethi
Le Thi HA, Pham Dinh T (1997) Solving a class of linearly constrained indefinite quadratic problems by D.C. algorithms. J Glob Optim 11:253–285
Le Thi HA, Pham Dinh T (2005) DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Ann Oper Res 133:23–46
Le Thi HA, Le Hoai M, Pham Dinh T (2007) Fuzzy clustering based on nonconvex optimization approaches using difference of convex (DC) functions algorithms. J Adv Data Anal Classif 2:1–20
Le Thi HA, Le Hoai M, Nguyen VV, Pham Dinh T (2008) A DC programming approach for feature selection in support vector machines learning. J Adv Data Anal Classif 2(3):259–278
Le Thi HA, Le Hoai M, Pham Dinh T, Huynh VN (2012) Binary classification via spherical separator by DC programming and DCA. J Glob Optim 1–15. doi:10.1007/s10898-012-9859-6
Le Thi HA, Le Hoai M, Huynh VN (2013) Block clustering based on DC programming and DCA. NECO Neural Comput 25(10):2776–2807
Le Thi HA, Pham Dinh T, Nguyen CN, Le Hoai M DC programming and DCA for diversity data mining. Optimization (to appear)
Le Thi HA, Le Hoai M, Pham Dinh T (2014) New and efficient DCA based algorithms for minimum sum-of-squares clustering. Patter Recognit 47:388–401
Lee M, Choi P, Woo Y (2002) A hybrid recommender system combining collaborative filtering with neural network. Adaptive hypermedia and adaptive web-based systems. Lecture Notes in Computer Science, vol 2347. Springer-Verlag, Berlin Heidelberg, pp 531–534
Lefebvre G, Zheng H, Laurent C (2006) Objectionable image detection by ASSOM competition. Image and video retrieval. Lecture Notes in Computer Science, vol 4071. Springer, Heidelberg, pp 201–210
Liu Y, Shen X, Doss H (2005) Multicategory \(\Psi \)-learning and support vector machine, computational tools. J Comput Graph Stat 14:219–236
Liu Y, Shen X (2006) Multicategory \(\Psi \)-learning. J Am Stat Assoc 101(474):500–509
Madalina O, Nathalie VV, Christine CA (2013) Multiple Kernel self-organizing maps. In: ESANN 2013 proceedings, European symposium on artificial neural networks, computational intelligence and machine learning. Bruges, Belgium, pp 83–88
Van Hulle Marc M (2012) Self-organizing maps. Handbook of natural computing. Springer-Verlag, Berlin, Heidelberg, pp 585–622
Marina R (2012) Graph mining based SOM: a tool to analyze economic stability. In: Johnsson M (ed) Applications of self-organizing maps. InTech Publisher, Vienna, pp 1–25
Matharage S, Alahakoon D, Rajapakse J, Huang P (2011) Fast growing self organizing map for text clustering. Neural information processing. Lecture Notes in Computer Science, vol 7063. Springer, Heidelberg, pp 406–415
Matsushita H, Nishio Y (2010) Batch-learning self-organizing map with weighted connections avoiding false-neighbor effects. In: The 2010 international joint conference on neural networks (IJCNN), pp 1–6
Manning CD, Raghavan P, Schtze H (2008) Introduction to information retrieval. Cambridge University Press, Cambridge
Neme A, Miramontes P (2014) Self-organizing map formation with a selectively refractory neighborhood. Neural Process Lett 39(1):1–24
Ogihara M, Matsumoto H, Marumo T, Kuroda C (2009) Clustering of pressure fluctuation data using self-organizing map. Engineering applications of neural networks. Communications in Computer and Information Science, vol 43. Springer, Heidelberg, pp 45–54
Olteanu M, Villa-Vialaneix N, Cierco-Ayrolles C (2013) Multiple Kernel self-organizing maps. Hal-00817920, version 1
O’Connell C, Kutics A, Nakagawa A (2013) Layered self-organizing map for image classification in unrestricted domains. Image analysis and processing—ICIAP 2013. Lecture Notes in Computer Science, vol 8156. Springer, Heidelberg, pp 310–319
Paul S, Gupta M (2013) Image segmentation by self organizing map with mahalanobis distance. Int J Emerg Technol Adv Eng 3(2):288–291
Pham Dinh T, Le Thi HA (1997) Convex analysis approach to D.C. programming: theory, algorithms and applications (dedicated to Professor Hoang Tuy on the occasion of his 70th birthday). Acta Math Vietnam 22:289–355
Pham Dinh T, Le Thi HA (1998) DC optimization algorithms for solving the trust region sub-problem. SIAM J Optim 8:476–505
Pratiwi D (2012) The use of self organizing map method and feature selection in image database classification system. Int J Comput Sci 9(3):377–381
Roh TH, Oh KJ, Han I (2003) The collaborative filtering recommendation based on SOM cluster-indexing CBR. Expert Syst Appl 25(3):413–423, ISSN 0957–4174, doi:10.1016/S0957-4174(03)00067-8
Raskutti B, Leckie C (1999) An evaluation of criteria for measuring the quality of clusters. In: Proceedings of the sixteenth international joint conference on artificial intelligence IJCAI ’99, pp 905–910
Ruan X, Gao Y, Song H, Chen J (2011) A new dynamic self-organizing method for mobile robot environment mapping. J Intell Learn Syst Appl 3:249–256
Saarikoski J, Laurikkala J, Järvelin K, Juhola M (2011) Self-organising maps in document classification: a comparison with six machine learning methods. Adaptive and natural computing algorithms. Lecture Notes in Computer Science, vol 6593. Springer, Heidelberg, pp 260–269
Sarlin P, Eklund T (2011) Fuzzy clustering of the self-organizing map: some applications on financial time series. In: Laaksonen J, Honkela T (eds) Advances in self-organizing maps. Lecture Notes in Computer Science, vol 6731. Springer, Berlin Heidelberg, pp 40–50
Shen X, Tseng GC, Zhang X, Wong WH (2003) \(\psi \)-Learning. J Am Stat Assoc 98:724–734
Smith T, Alahakoon D (2009) Growing self-organizing map for online continuous clustering. Foundations of Computational Intelligence, vol 4. Stud Comput Intell 204:49–83
Szymanski J, Duch W (2012) Self organizing maps for visualization of categories. Neural information processing, Lecture Notes in Computer Science, vol 7663. Springer, Heidelberg, pp 160–167
Tsoi AC, Hagenbuchner M, Sperduti A (2006) Self-organising map techniques for graph data applications to clustering of XML documents. Advanced data mining and applications. Lecture Notes in Computer Science, vol 4093. Springer, Heidelberg, pp 19–30
Van Laerhoven K (2001) Combining the self-organizing map and K-means clustering for on-line classification of sensor data. Artificial neural networks ICANN 2001. Lecture Notes in Computer Science, vol 2130. Springer, Heidelberg, pp 464–469
Vembu S, Baumann S (2004) A self-organizing map based knowledge discovery for music recommendation systems. Computer music modeling and retrieval. Lecture Notes in Computer Science, vol 3310. Springer-Verlag, Berlin Heidelberg, pp 119–129
Villa N, Boulet R (2007) Clustering a medieval social network by SOM using a kernel based distance measure. In: ESANN’2007 proceedings—European symposium on artificial neural networks Bruges (Belgium), d-side publi., ISBN 2-930307-07-2
Wang J, Shen Z, Pan W (2007) On transductive support vector machines. In: Proceeding of the international conference on machine learning ICML
Wehrens R (2010) Self-organising maps for image segmentation. In: Advances in data analysis, data handling and business intelligence studies in classification, Data Analysis, and Knowledge Organization, pp 373–383
Yin H (2008) The self-organizing maps: background, theories, extensions and applications. Stud Comput Intell (SCI) 115:715–762
Yuille AL, Rangarajan A (2002) The convex concave procedure (CCCP). Advances in neural information processing system, vol 14. MIT Press, Cambrige MA
Acknowledgments
We are very grateful to the anonymous referees and the associate editor for their really helpful and constructive comments that helped us to improve our paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Responsible editors: Toon Calders, Floriana Esposito, Eyke Hüllermeier, Rosa Meo.
Rights and permissions
About this article
Cite this article
Le Thi, H.A., Nguyen, M.C. Self-organizing maps by difference of convex functions optimization. Data Min Knowl Disc 28, 1336–1365 (2014). https://doi.org/10.1007/s10618-014-0369-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10618-014-0369-7