Abstract
The decomposition method is currently one of the major methods for solving support vector machines. An important issue of this method is the selection of working sets. In this paper through the design of decomposition methods for bound-constrained SVM formulations we demonstrate that the working set selection is not a trivial task. Then from the experimental analysis we propose a simple selection of the working set which leads to faster convergences for difficult cases. Numerical experiments on different types of problems are conducted to demonstrate the viability of the proposed method.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Blake, C. L. & Merz, C. J. (1998). UCI repository of machine learning databases. Irvine, CA. (Available at http://www.ics.uci.edu/~mlearn/mLRepository.html)
Chang, C.-C., Hsu, C.-W., & Lin, C.-J. (2000). The analysis of decomposition methods for support vector machines. IEEE Trans. Neural Networks, 11:4, 1003-1008.
Cristianini, N. & Shawe-Taylor, J. (2000). An introduction to support vector machines. Cambridge, UK: Cambridge University Press.
Fletcher, R. (1987). Practical methods of optimization. New York: John Wiley and Sons.
Friess, T.-T., Cristianini, N., & Campbell, C. (1998). The kernel adatron algorithm: A fast and simple learning procedure for support vector machines. In Proceeding of 15th Intl. Conf. Machine Learning. San Francisco, CA: Morgan Kaufman Publishers.
Ho, T. K. & Kleinberg, E. M. (1996). Building projectable classifiers of arbitrary complexity. In Proceedings of the 13th International Conference on Pattern Recognition (pp. 880-885). Vienna, Austria.
Joachims, T. (1998). Making large-scale SVM learning practical. In B. Schölkopf, C. J. C. Burges, & A. J. Smola (Eds.). Advances in kernel methods-support vector learning. Cambridge, MA: MIT Press.
Joachims, T. (2000). Private communication.
Laskov, P. (2002). Feasible direction decomposition algorithms for training support vector machines, Machine Learning, 46, 315-349.
Lin, C.-J. (2000). On the convergence of the decomposition method for support vector machines (Technical Report). Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan. To appear in IEEE Trans. Neural Network.
Lin, C.-J. & Moré, J. J. (1999). Newton's method for large-scale bound constrained problems. SIAM J. Optim., 9, 1100-1127. (Software available at http://www.mcs.anl.gov/more/tron)
Mangasarian, O. L. & Musicant, D. R. (1999). Successive overrelaxation for support vector machines. IEEE Trans. Neural Networks, 10:5, 1032-1037.
Michie, D., Spiegelhalter, D. J., & Taylor, C. C. (1994). Machine learning, neural and statistical classification.
Englewood Cliffs, N.J.: Prentice Hall. (Data available at anonymous ftp: ftp.ncc.up.pt/pub/statlog/)
Osuna, E., Freund, R., & Girosi, F. (1997). Training support vector machines: An application to face detection. In Proceedings of CVPR'97.
Platt, J. C. (1998). Fast training of support vector machines using sequential minimal optimization. In B. Schölkopf, C. J. C. Burges, & A. J. Smola (Eds.). Advances in kernel methods-support vector learning. Cambridge, MA: MIT Press.
Powell, M. J. D. (1973). On search directions for minimization. Math. Programming, 4, 193-201.
Saunders, C., Stitson, M. O., Weston, J., Bottou, L., Schölkopf, B., & Smola, A. (1998). Support vector machine reference manual (Technical Report No. CSD-TR-98-03). Egham, UK: Royal Holloway, University of London.
Schölkopf, B., Burges, C. J. C., & Smola, A. J. (Eds.). (1998). Advances in kernel methods-support vector learning. Cambridge, MA: MIT Press.
Vanderbei, R. (1994). LOQO: An interior point code for quadratic programming (Technical Report No. SOR 94-15). Statistics and Operations Research, Princeton University. (revised November, 1998)
Vapnik, V. (1995). The nature of statistical learning theory. New York, NY: Springer-Verlag.
Vapnik, V. (1998). Statistical learning theory. New York, NY: John Wiley.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hsu, CW., Lin, CJ. A Simple Decomposition Method for Support Vector Machines. Machine Learning 46, 291–314 (2002). https://doi.org/10.1023/A:1012427100071
Issue Date:
DOI: https://doi.org/10.1023/A:1012427100071