Abstract
In many practical situations in inductive learning algorithms, it is often expected to further improve the generalization capability after the learning process has been completed if new data are available. One of the common approaches is to add training data to the learning algorithm and retrain it, but retraining for each new data point or data set can be very expensive. In view of the learning methods of human beings, it seems natural to build posterior learning results upon prior results. In this paper, we apply Support Vector Machine(SVM) to the concept updating procedure. If initial concept would be built up by inductive algorithm, then concept updated is the normal solution corresponding to the initial concept learned. It was shown that concept learned would not change if the new available data located in error-insensitive zone. Especially, concept initially learned and updated by SVR induces an incremental SVR approximately learning method for large scale data. We tested our method on toys data sets and 7 regression bench mark data set. It shown that generalization capacity after updating with SVR was improved according to FVU or MSE on the independent test set.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)
Syed, N.A., Liu, H., Sung, K.: Handling concept drifts in incremental learning with support vector machines. In: Proceedings of 1st Intl. Conf. on Knowledge Discovery and Data Mining, pp. 317–321. AAAI, Menlo Park (1999)
Ralaivola, L., d’Alch-Buc, F.: Incremental Support Vector Machine Learning: A Local Approach. In: Dorffner, G., Bischof, H., Hornik, K. (eds.) ICANN 2001. LNCS, vol. 2130, pp. 322–330. Springer, Heidelberg (2001)
Engel, Y., Mannor, S., Meir, R.: Sparse online greedy support vector regression. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) ECML 2002. LNCS (LNAI), vol. 2430, pp. 84–96. Springer, Heidelberg (2002)
Herbster, M.: Learning additive models online with fast evaluating kernel. In: Proceedings of 14th Annual Conference on Computational Learning Theory (COLT), pp. 444–460. Springer, Heidelberg (2001)
Kivinen, J., Smola, A.J., Willianmson, R.C.: Online learning with kernel. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems, pp. 785–792 (2001)
Li, Y., Long, P.: The relaxed online maximum margin algorith. In: Solla, A., Leen, T.K., Mller, K.-R. (eds.) Advances in Neural Information Processing Systems, vol. 12, pp. 498–504. MIT Press, Cambridge (2001)
Ma, J., Theiler, J., Perkins, S.: Accurate online support vector regression. Neural Computation 15(11), 2683–2703 (2003)
Collobert, R., Bengio, S.: SVMTorch: Support vector machines for large-scale regression problems. Journal of Machine Learning Research 1, 143–160 (2001)
Osuna, E., Freund, R., Girosi, F.: An improved training algorithm for support vector machines. In: Proceedings, 1997 IEEE Workshop on Neural Networks for Signal Processing, pp. 276–285 (1997)
Flake, G., Lawrence, S.: Efficient svm regression training with smo. Machine Learning 46, 271–290 (2002)
Joachims, T.: Making large-sclae support vedtor machine learning practical. In: Scholkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods. MIT Press, Cambridge (1999)
Roosen, C.B., Hastie, T.J.: Logistic response projection pursuit, Tech. Rep. BL011214-930806-09TM, AT&T Bell Laboratories (1993)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, Y., He, Q. (2005). Concept Updating with Support Vector Machines. In: Fan, W., Wu, Z., Yang, J. (eds) Advances in Web-Age Information Management. WAIM 2005. Lecture Notes in Computer Science, vol 3739. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11563952_43
Download citation
DOI: https://doi.org/10.1007/11563952_43
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29227-2
Online ISBN: 978-3-540-32087-6
eBook Packages: Computer ScienceComputer Science (R0)