Abstract
The quadratic programming (QP) and the linear programming (LP) based method are recently the most popular learning methods from empirical data. Support vector machines (SVMs) are the newest models based on QP al- gorithm in solving the nonlinear regression and classification problems. The LP based learning also controls both the number of basis functions in a neural net- work (i.e., support vector machine) and the accuracy of learning machine. Both methods result in a parsimonious network. This results in data compression. Two different methods are compared in terms of number of SVs (possible compression achieved) and in generalization capability (i.e., error on unseen data).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Arthanari, T. S., Y. Dodge, 1993. Mathematical Programming in Statistics, J. Wiley & Sons., New York, NY
Bennett, K., 1999. Combining support vector and mathematical programming methods for induction. In B. Schölkopf, C. Burges, and A. Smola, Editors, Advances in Kernel Methods-SV Learning, pp 307–326, MIT Press, Cambridge, MA Charnes, A., W. W. Cooper, R. O. Ferguson, 1955. Optimal Estimation of Executive Compensation by Linear Programming, Manage. Sci., 1, 138
Cheney, E. W., A. A. Goldstein, 1958. Note on a paper by Zuhovickii concerning the Chebyshev problem for linear equations. J. Soc. Indust. Appl. Math., Volume 6, pp. 233–239
Eisenhart, C., 1962. Roger Joseph Boscovich and the Combination of Observationes, Actes International Symposium on R. J. Boskovic, pp. 19–25, Belgrade-Zagreb-Ljubljana, YU
Graepel, T., R. Herbrich, B. Schölkopf, A. Smola, P. Bartlett, K.-R. Müller, K. Obermayer, R. Williamson, 1999. Classification on proximity data with LP-machines, Proc. of the 9th Intl. Conf. on Artificial NN, ICANN 99, Edinburgh, 7-10 Sept.
Hadzic, I., 1999. Learning from Experimental Data by Linear Programming Selected Support Vectors, PhD Thesis, (work in progress), The University of Auckland, Auckland, NZ
Hadzic, I., V. Kecman, 1999. Learning from Data by Linear Programming, NZ Postgraduate Conference Proceedings, Auckland, Dec. 15-16
Kecman, V., 2001. Learning and Soft Computing, SVM, NN, and FLS, The MIT Press, Cambridge, MA
Kecman V., Hadzic I., 2000. Support Vectors Selection by Linear Programming, Proceedings of the International Joint Conference on Neural Networks (IJCNN 2000), Vol. 5, pp. 193–199, Como, Italy
Kelley E. J. jr., 1958. An application of linear programming to curve fitting (received by editors October 11, (1957), J. Soc. Indust. Appl. Math., Volume 6, pp. 15–22
Mangasarian, O. L., 1965. Linear and Nonlinear Separation of Patterns by Linear Programming, Operations Research 13, pp. 444–452.
Poggio, T., F. Girosi, 1993. Learning, Approximation and Networks, Lectures and Lectures’ Handouts-Course 9.520, MIT, Cambridge, MA.
Rice R. J., 1964. The Approximation of Functions, Addison-Wesley Publishing Company, Reading, Massachusetts, Palo Alto, London
Smola A., T. T. Friess, B. Schölkopf, 1998. Semiparametric Support Vector and Linear Programming Machines, NeuroCOLT2 Technical Report Series, NC2-TR-1998-024
Smola, A., B. Schölkopf, G. Rätsch, 1999. Linear Programs for Automatic Accuracy Control in Regression, submitted to ICANN’99
Stiefel, E., 1960. Note on Jordan Elimination, Linear Programming and Chebyshev Approximation, Numer. Math., 2, 1.
Vapnik, V. N., 1995. The Nature of Statistical Learning Theory, Springer Verlag Inc, New York, NY
Weston, J., A. Gammerman, M. O. Stitson, V. Vapnik, V. Vovk, C. Watkins, 1999. Support Vector Density Estimation, In B. Schölkopf, C. Burges, and A. Smola, Editors, Advances in Kernel Methods-SV Learning, p.p. 307–326, MIT Press, Cambridge, MA
Zhang, Q. H., J.-J. Fuchs, 1999. Building neural networks through linear programming, Proceedings of 14th IFAC Triennial World Congress, Vol. K, pp. 127–132, Pergamon Press
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kecman, V., Arthanari, T. (2001). Comparisons of QP and LP Based Learning from Empirical Data. In: Monostori, L., Váncza, J., Ali, M. (eds) Engineering of Intelligent Systems. IEA/AIE 2001. Lecture Notes in Computer Science(), vol 2070. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45517-5_36
Download citation
DOI: https://doi.org/10.1007/3-540-45517-5_36
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42219-8
Online ISBN: 978-3-540-45517-2
eBook Packages: Springer Book Archive