Abstract
Support Vector Machines (SVMs) have been successfully applied to solve a large number of classification and regression problems. However, SVMs suffer from the catastrophic forgetting phenomenon, which results in loss of previously learned information. Learn + + have recently been introduced as an incremental learning algorithm. The strength of Learn + + lies in its ability to learn new data without forgetting previously acquired knowledge and without requiring access to any of the previously seen data, even when the new data introduce new classes. To address thecatastrophic forgetting problem and to add the incremental learning capability to SVMs, we propose using an ensemble of SVMs trained with Learn + + . Simulation results on real-world and benchmark datasets suggest that the proposed approach is promising.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)
Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press, Cambridge (2000)
Grossberg, S.: Nonlinear neural networks: principles, mechanisms and architectures. Neural Networks 1(1), 17–61 (1988)
French, R.: Catastrophic forgetting in connectionist networks: Causes, Consequences and Solutions. Trends in Cognitive Sciences 3(4), 128–135 (1999)
Polikar, R., Udpa, L., Udpa, S., Honavar, V.: Learn++: An incremental learning algorithm for multilayer perceptrons. In: Proceedings of 25th. IEEE International Conference on Acoustics, Speech and Signal Processing, Istanbul, Turkey, vol. 6, pp. 3414–3417 (2000)
Polikar, R., Udpa, L., Udpa, S., Honavar, V.: Learn++: An incremental learning algorithm for supervised neural networks. IEEE Transactions on Systems, Man, and Cybernetics. Part C: Applications and Reviews 31(4), 497–508 (2001)
Freund, Y., Schapire, R.: A decision theoretic generalization of on-line learning and an application to boosting. Computer and System Sciences 57(1), 119–139 (1997)
Kasabov, N.: Evolving Connectionist Systems: Methods and Applications in Bioinformatics, Brain Study and Intelligent Machines. Springer, Heidelberg (2002)
Platt, J.: Fast Training of Support Vector Machines using Sequential Minimal Optimization. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods - Support Vector Learning. MIT Press, Cambridge (1998)
Domeniconi, C., Gunopulos, D.: Incremental Support Machine Construction. In: Proceedings of First IEEE Int. Conf. on Data Mining (ICDM 2001), pp. 589–592 (2001)
Mitra, P., Murthy, C.A., Pal, S.K.: Data condensation in large databases by incremental learning with support vector machines. In: Proceedings of 15th International Conference on Pattern Recognition, September 3-7, vol. 2, pp. 708–711 (2000)
Li, K., Huang, H.-K.: Incremental learning proximal support vector machine classifiers. In: Proceedings of International Conference on Machine Learning and Cybernetics, November 4-5, vol. 3, pp. 1635–1637 (2002)
An, J.-L., Wang, Z.-O., Ma, Z.-P.: An incremental learning algorithm for support vector machine. In: Proceedings of International Conference on Machine Learning and Cybernetics, November 2-5, vol. 2, pp. 1153–1156 (2003)
Li, Z.-W., Zhang, J.-P., Yang, J.: A heuristic algorithm to incremental support vector machine learning. In: Proceedings of 2004 International Conference on Machine Learning and Cybernetics, August 26-29, vol. 3, pp. 1764–1767 (2004)
Pavlov, D., Mao, J., Dom, B.: Scaling-up Support Vector Machines using The Boosting Algorithm. In: Proceedings of the International Conference on Pattern Recognition, Barcelona, Spain, September 3-7, pp. 19–22 (2000)
Valentini, G., Muselli, M., Ruffino, F.: Cancer Recognition with Bagged Ensembles of Support Vector Machines. Neurocomputing 56(1), 461–466 (2004)
Valentini, G., Muselli, M., Ruffino, F.: Bagged Ensembles of SVMs for Gene Expression Data Analysis. In: Proceeding of the International Joint Conference on Neural Networks, Portland, OR, USA, July 20-24, pp. 1844–1849 (2003)
Kim, H.-C., Pang, S., Je, H.-M., Kim, D., Bang, S.Y.: Constructing Support Vector Machine Ensemble. Pattern Recognition 36, 2757–2767 (2003)
Duan, K., Keerthi, S.S., Poo, A.N.: Evaluation of simple performance measures for tuning SVM hyperparameters. Neurocomputing 51, 41–59 (2003)
Muhlbaier, M., Topalis, A., Polikar, R.: Learn++.MT: A New Approach to Incremental Learning. In: Roli, F., Kittler, J., Windeatt, T. (eds.) MCS 2004. LNCS, vol. 3077, pp. 52–61. Springer, Heidelberg (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Erdem, Z., Polikar, R., Gurgen, F., Yumusak, N. (2005). Ensemble of SVMs for Incremental Learning. In: Oza, N.C., Polikar, R., Kittler, J., Roli, F. (eds) Multiple Classifier Systems. MCS 2005. Lecture Notes in Computer Science, vol 3541. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11494683_25
Download citation
DOI: https://doi.org/10.1007/11494683_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26306-7
Online ISBN: 978-3-540-31578-0
eBook Packages: Computer ScienceComputer Science (R0)