Abstract
In this paper we consider the statistical aspects of support vector machines (SVMs) in the classification context, and describe an approach to adaptively tuning the smoothing parameter(s) in the SVMs. The relation between the Bayes rule of classification and the SVMs is discussed, shedding light on why the SVMs work well. This relation also reveals that the misclassification rate of the SVMs is closely related to the generalized comparative Kullback-Leibler distance (GCKL) proposed in Wahba (1999, Scholkopf, Burges, & Smola (Eds.), Advances in Kernel Methods—Support Vector Learning. Cambridge, MA: MIT Press). The adaptive tuning is based on the generalized approximate cross validation (GACV), which is an easily computable proxy of the GCKL. The results are generalized to the unbalanced case where the fraction of members of the classes in the training set is different than that in the general population, and the costs of misclassification for the two kinds of errors are different. The main results in this paper have been obtained in several places elsewhere. Here we take the opportunity to organize them in one place and note how they fit together and reinforce one another. Mostly the work of the authors is reviewed.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Brown, M. P. S., Grundy, W. N., Lin, D., Cristianini, N., Sugnet, C. W., Furey, T. S., Ares, M., Jr., & Haussler, D. (2000). Knowledge-based analysis of microarray gene expression data by using support vector machines. In Proceedings of the National Academy of Sciences, 97:1, 262–267.
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. In D. Haussler (Ed.), In Proceedings of the 5th Annual ACMWorkshop on Computational Learning Theory. Pittsburgh, PA: ACM Press.
Burges, C. J. C. (1998). Atutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2:2, 121–167.
Cox, D. D., & O'Sullivan, F. (1990). Asymptotic analysis of penalized likelihood and related estimates. The Annals of Statistics, 18:4, 1676–1695.
Cortes, C., & Vapnik, V. N. (1995). Support vector networks. Machine Learning, 20, 273–297.
Hand, D. J. (1997). Construction and assessment of classification rules. Chichester, England: John Wiley & Sons.
Lin, Y. (1999). Support vector machines and the Bayes rule in classification. Data Mining and Knowledge Discovery, to appear.
Lin,Y., Lee,Y., & Wahba, G. (2002). Support vector machines for classification in nonstandard situations. Machine Learning, 46, 191–202.
Poggio, T., & Girosi, F. (1998). A sparse representation for function approximation. Neural Computation, 10, 1445–1454.
Vapnik, V. N. (1995). The nature of statistical learning theory. New York: Springer Verlag.
Wahba, G. (1990). Spline models for observational data. Philadelphia, PA: Society for Industrial and Applied Mathematics.
Wahba, G. (1999). Support vector machines, reproducing kernel Hilbert spaces and the randomized GACV. In B. Scholkopf, C. J. C. Burges, & A. J. Smola (Eds.), Advances in kernel methods-Support vector learning. Cambridge, MA: MIT Press.
Wahba, G., Lin, Y., & Zhang, H. (2000). GACV for support vector machines, or, another way to look at marginlike quantities. In A. J. Smola, P. Bartlett, B. Scholkopf, & D. Schurmans (Eds.), Advances in large margin classifiers. Cambridge, MA & London, England: MIT Press.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Lin, Y., Wahba, G., Zhang, H. et al. Statistical Properties and Adaptive Tuning of Support Vector Machines. Machine Learning 48, 115–136 (2002). https://doi.org/10.1023/A:1013951620650
Issue Date:
DOI: https://doi.org/10.1023/A:1013951620650