Abstract
A theoretical advantage of large margin classifiers such as support vector machines (SVM) concerns the empirical and structural risk minimization which balances the complexity of the model against its success at fitting the training data. Metaheuristics have been used to work with SVMs in order to select features, tune hypeparameters or even achieve a reduced-set of support vectors. In spite of such tasks being interesting, metaheuristics such as simulated annealing (SA) do not play an important role in the process of solving the quadratic optimization problem, which arises from support vector machines. To do so, well-known methods such as sequential minimal optimization, kernel adatron or even classical mathematical methods have been used with this goal. In this paper, we propose to use simulated annealing in order to solve such a quadratic optimization problem. Our proposal is interesting when compared with those aforementioned methods, since it is simple and achieved similar (or even higher) accuracy and high sparseness in the solution.
The authors would like to thank the IFCE and CAPES for supporting their research.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Anlauf, J.K., Biehl, M.: The adatron: an adaptive perceptron algorithm. Europhys. Lett. 10(7), 687 (1989)
Blake, C., Merz, C.J.: UCI repository of machine learning databases (1998)
Dumitrescu, D., Preuss, M., Stoean, C., Stoean, R.: Evolutionary support vector machines and their application for classification (2006)
Černý, V.: Thermodynamical approach to the traveling salesman problem: an efficient simulation algorithm. J. Optim. Theor. Appl. 45(1), 41–51 (1985)
Gill, P.E., Murray, W., Wright, M.H.: Practical Optimization. Academic Press, London (1981)
Hollander, M., Wolfe, D.A., Chicken, E.: Nonparametric Statistical Methods. Wiley, New York (2013)
Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983)
Li, Y., Tong, Y., Bai, B., Zhang, Y.: An improved particle swarm optimization for svm training. In: Third International Conference on Natural Computation (ICNC 2007), vol. 2, pp. 611–615. IEEE (2007)
Michalewicz, Z., Janikow, C.Z.: Handling constraints in genetic algorithms. In: ICGA, pp. 151–157 (1991)
Mierswa, I.: Evolutionary learning with kernels: a generic solution for large margin problems. In: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, pp. 1553–1560. ACM (2006)
Paquet, U.: Training support vector machines with particle swarms (2007)
Paquet, U., Engelbrecht, A.P.: A new particle swarm optimiser for linearly constrained optimisation. In: The 2003 Congress on Evolutionary Computation, CEC 2003, vol. 1, pp. 227–233. IEEE (2003)
Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. In: Advances in Kernel Methods - Support Vector Learning. MIT Press, Cambridge, MA, USA (1999)
Neto, A.R.R., Barreto, G.A.: Opposite maps: vector quantization algorithms for building reduced-set SVM and lSSVM classifiers. Neural Process. Lett. 37(1), 3–19 (2013)
Silva, A., Gonçalves, T.: Training support vector machines with an heterogeneous particle swarm optimizer. In: Tomassini, M., Antonioni, A., Daolio, F., Buesser, P. (eds.) ICANNGA 2013. LNCS, vol. 7824, pp. 100–109. Springer, Heidelberg (2013). doi:10.1007/978-3-642-37213-1_11
Silva, D.A., Silva, J.P., Neto, A.R.R.: Novel approaches using evolutionary computation for sparse lssvms. Neurocomput. 168, 908–916 (2015)
Stoean, R., Dumitrescu, D., Preuss, M., Stoean, C.: Evolutionary support vector regression machines. In: Eighth International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, SYNASC 2006, pp. 330–335 (2006)
Stoean, R., Preuss, M., Stoean, C., Dumitrescu, D.: Concerning the potential of evolutionary support vector machines. In: IEEE Congress on Evolutionary Computation, CEC 2007, pp. 1436–1443. IEEE (2007)
Vapnik, V.N.: Statistical Learning Theory. Wiley-Interscience, New York (1998)
Yuan, H., Zhang, Y., Zhang, D., Yang, G.: A modified particle swarm optimization algorithm for support vector machine training. In: 2006 6th World Congress on Intelligent Control and Automation, vol. 1, pp. 4128–4132. IEEE (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Dias, M.L.D., Neto, A.R.R. (2017). A Novel Simulated Annealing-Based Learning Algorithm for Training Support Vector Machines. In: Madureira, A., Abraham, A., Gamboa, D., Novais, P. (eds) Intelligent Systems Design and Applications. ISDA 2016. Advances in Intelligent Systems and Computing, vol 557. Springer, Cham. https://doi.org/10.1007/978-3-319-53480-0_34
Download citation
DOI: https://doi.org/10.1007/978-3-319-53480-0_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-53479-4
Online ISBN: 978-3-319-53480-0
eBook Packages: EngineeringEngineering (R0)