Article

Boosting support vector machines for text classification through parameter-free threshold relaxation

Authors:

James G. Shanahan,

Norbert RomaAuthors Info & Claims

CIKM '03: Proceedings of the twelfth international conference on Information and knowledge management

Pages 247 - 254

https://doi.org/10.1145/956863.956911

Published: 03 November 2003 Publication History

Abstract

Support vector machine (SVM) learning algorithms focus on finding the hyperplane that maximizes the margin (the distance from the separating hyperplane to the nearest examples) since this criterion provides a good upper bound of the generalization error. When applied to text classification, these learning algorithms lead to SVMs with excellent precision but poor recall. Various relaxation approaches have been proposed to counter this problem including: asymmetric SVM learning algorithms (soft SVMs with asymmetric misclassification costs); uneven margin based learning; and thresholding. A review of these approaches is presented here. In addition, in this paper, we describe a new threshold relaxation algorithm. This approach builds on previous thresholding work based upon the beta-gamma algorithm. The proposed thresholding strategy is parameter free, relying on a process of retrofitting and cross validation to set algorithm parameters empirically, whereas our previous approach required the specification of two parameters (beta and gamma). The proposed approach is more efficient, does not require the specification of any parameters, and similarly to the parameter-based approach, boosts the performance of baseline SVMs by at least 20% for standard information retrieval measures.

References

[1]

Arampatzis A., Unbiased S-D Threshold Optimization, Initial Query Degradation, Decay, and Incrementality, for Adaptive Document Filtering, Tenth Text Retrieval Conference (TREC-2001), 2002, 596--605.

[2]

Ault T., Yang Y., kNN, Rocchio and Metrics for Information Filtering at TREC-10, Tenth Text Retrieval Conference (TREC-2001), 2002, 84--93

[3]

Cancedda N. et al., Kernel Methods for Document Filtering, Eleventh Text Retrieval Conference (TREC-11), 2003.

[4]

Cortes, C., and Vapnik, V., Support vector networks. Machine Learning, 20, 273--297, 1995.

Digital Library

[5]

Evans, D. A., Shanahan, J., Tong, X., Roma, N., Stoica, E., Sheftel, V., Montgomery, J., Bennett, J., Fujita, S., Grefenstette, G. Topic Specific Optimization and Structuring. Tenth Text Retrieval Conference (TREC-2001), 2002, 132--141.

[6]

Joachims, T. Text categorization with support vector machines: Learning with many relevant features. In Proceedings 10th European Conference on Machine Learning (ECML), Springer Verlag, 1998. http://www-ai.cs.uni-dortmund.de/DOKIMENTE/Joachims_97a.ps.gz

Digital Library

[7]

Keerthi, S. S., Shevade, S. K., Bhattacharyya, C., Murthy, K. R. K. Improvements to Platt's SMO algorithm for SVM classifier design. Technical report, Dept of CSA, IISc, Bangalore, India, 1999.

[8]

LeCun, Y., Jackel, L. D., Bottou, L., Cortes, C., Denker, J. S., Drucker, H., Guyon, I., Muller, U. A., Sackinger, E., Simard, P. and Vapnik, V. Learning algorithms for classification: A comparison on handwritten digit recognition. Neural Networks: The Statistical Mechanics Perspective, 261--276, 1995.

[9]

Lewis D. D., The Reuters-21578 text categorization test collection. http://www.research.att.com/ lewis/reuters21578.html. Checked on 11 May 1998; Timestamp Tue Jan 20 21:07:21 EST 1998.

[10]

Lewis D. D., Applying Support Vector Machines to the TREC-2001 Batch Filtering and Routing Tasks, Tenth Text Retrieval Conference (TREC-2001), 2002, 286--294.

[11]

Li, Y., Zaragoza, H., Herbrich, R., Shawe-Taylor, J., Kandola, J. S. The Perceptron Algorithm with Uneven Margins. ICML 2002 379--386.

Digital Library

[12]

Mayfield J., McNamee P., Costello C., Piatko C., Banerjee A., JHU/APL at TREC 2001: Experiments in Filtering and in Arabic, Video, and Web Retrieval, at TREC-10, Tenth Text Retrieval Conference (TREC-2001), 2002, 322--332.

[13]

Morik, K., Brockhausen, P., Joachims, T. Combining statistical learning with a knowledge-based approach - A case study in intensive care monitoring. Proc. 16th Int'l Conf. on Machine Learning (ICML-99), 1999.

Digital Library

[14]

Osuna, E., Freund, R., and Girosi, F. Training support vector machines: An application to face detection. In Proceedings of Computer Vision and Pattern Recognition '97, 1997 130--136.

Digital Library

[15]

Platt, J. Fast training of SVMs using sequential minimal optimization. In: B. Scholkopf, C. Burges, and A. Smola (Eds.) Advances in Kernel Methods - Support Vector Learning, MIT Press, 1998.

Digital Library

[16]

Robertson S. E., Soboroff I., The TREC 2001 Filtering Track Report, Tenth Text Retrieval Conference (TREC-2001), 2002, 26--37.

[17]

Robertson S. E., Walker S., Zaragoza H., Herbrich H., Microsoft Cambridge at TREC 2002: Filtering Track, Eleventh Text Retrieval Conference (TREC-2002), 2003.

[18]

Salton G., Introduction to Modern Information Retrieval, Verlag: Mc Graw Hill, New York, 1983.

Digital Library

[19]

Shanahan J. G., Roma N., Improving SVM Text Classification Performance through Threshold Adjustment, European Conference on Machine Learning (ECML) 2003, To Appear.

[20]

Vapnik, V., The Nature of Statistical Learning Theory, Springer-Verlag, 1995

Digital Library

[21]

Vapnik, V., Statistical Learning Theory, Wiley, 1998

[22]

Voorhees E.M., Overview of TREC 2002, Eleventh Text Retrieval Conference (TREC-2002), 2002, 1--16.

[23]

Yang Y., A study on thresholding strategies for text categorization, Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'01), 2001, 137--145.

Digital Library

[24]

Zhai, C., Jansen, P., Stoica, E., Grot, N., Evans, D. A. Threshold Calibration in CLARIT Adaptive Filtering. Seventh Text Retrieval Conference (TREC-7), 1999, 149--156.

[25]

Y. Zhang and J. Callan. "YFilter at TREC-9". In Proceedings of the Ninth Text REtrieval Conference (TREC-9), (pp. 135--140). National Institute of Standards and Technology, 2001, special publication 500-249.

Cited By

Wang BLi ZTang JZhang KChen SRu L(2011)Learning to advertiseProceedings of the 15th Pacific-Asia conference on Advances in knowledge discovery and data mining - Volume Part II10.5555/2022850.2022892(506-518)Online publication date: 24-May-2011
https://dl.acm.org/doi/10.5555/2022850.2022892
Sun BMitra PLee Giles CMueller K(2011)Identifying, Indexing, and Ranking Chemical Formulae and Chemical Names in Digital DocumentsACM Transactions on Information Systems10.1145/1961209.196121529:2(1-38)Online publication date: 1-Apr-2011
https://dl.acm.org/doi/10.1145/1961209.1961215
Wang BLi ZTang JZhang KChen SRu L(2011)Learning to Advertise: How Many Ads Are Enough?Advances in Knowledge Discovery and Data Mining10.1007/978-3-642-20847-8_42(506-518)Online publication date: 2011
https://doi.org/10.1007/978-3-642-20847-8_42
Show More Cited By

Index Terms

Boosting support vector machines for text classification through parameter-free threshold relaxation

Recommendations

Twin Support Vector Machines for Pattern Classification

We propose Twin SVM, a binary SVM classifier that determines two nonparallel planes by solving two related SVM-type problems, each of which is smaller than in a conventional SVM. The Twin SVM formulation is in the spirit of proximal SVMs via generalized ...
An overview on twin support vector machines

Twin support vector machines (TWSVM) is based on the idea of proximal SVM based on generalized eigenvalues (GEPSVM), which determines two nonparallel planes by solving two related SVM-type problems, so that its computing cost in the training phase is 1/...
Incremental training of support vector machines using hyperspheres

In the conventional incremental training of support vector machines, candidates for support vectors tend to be deleted if the separating hyperplane rotates as the training data are added. To solve this problem, in this paper, we propose an incremental ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '03: Proceedings of the twelfth international conference on Information and knowledge management

November 2003

592 pages

ISBN:1581137230

DOI:10.1145/956863

General Chair:
Donald Kraft
Louisiana State University
,
Program Chairs:
Ophir Frieder
Illinois Institute of Technology
,
Joachim Hammer
University of Florida
,
Sajda Qureshi
University of Nebraska, Omaha
,
Len Seligman
The MITRE Corporation

Copyright © 2003 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2003

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

CIKM03

Sponsor:

CIKM03: 12th International Conference on Information and Knowledge Management

November 3 - 8, 2003

LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
1,220
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 19 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang BLi ZTang JZhang KChen SRu L(2011)Learning to advertiseProceedings of the 15th Pacific-Asia conference on Advances in knowledge discovery and data mining - Volume Part II10.5555/2022850.2022892(506-518)Online publication date: 24-May-2011
https://dl.acm.org/doi/10.5555/2022850.2022892
Sun BMitra PLee Giles CMueller K(2011)Identifying, Indexing, and Ranking Chemical Formulae and Chemical Names in Digital DocumentsACM Transactions on Information Systems10.1145/1961209.196121529:2(1-38)Online publication date: 1-Apr-2011
https://dl.acm.org/doi/10.1145/1961209.1961215
Wang BLi ZTang JZhang KChen SRu L(2011)Learning to Advertise: How Many Ads Are Enough?Advances in Knowledge Discovery and Data Mining10.1007/978-3-642-20847-8_42(506-518)Online publication date: 2011
https://doi.org/10.1007/978-3-642-20847-8_42
Saleh IDarwish KFahmy ALi HKumaran A(2010)Classifying Wikipedia articles into NE's using SVM's with threshold adjustmentProceedings of the 2010 Named Entities Workshop10.5555/1870457.1870471(85-92)Online publication date: 16-Jul-2010
https://dl.acm.org/doi/10.5555/1870457.1870471
Xiao YLiu BCao LHuang JKoudas NJones GWu XCollins-Thompson KAn A(2010)K-farthest-neighbors-based concept boundary determination for support vector data descriptionProceedings of the 19th ACM international conference on Information and knowledge management10.1145/1871437.1871708(1701-1704)Online publication date: 26-Oct-2010
https://dl.acm.org/doi/10.1145/1871437.1871708
Sun ALim ELiu Y(2009)On strategies for imbalanced text classification using SVMDecision Support Systems10.1016/j.dss.2009.07.01148:1(191-201)Online publication date: 1-Dec-2009
https://dl.acm.org/doi/10.1016/j.dss.2009.07.011
Yamahara HHarada FTakada HShimakawa H(2008)Personalizing Threshold Values on Behavior Detection with Collaborative FilteringProceedings of the 5th international conference on Ubiquitous Intelligence and Computing10.1007/978-3-540-69293-5_33(411-425)Online publication date: 23-Jun-2008
https://dl.acm.org/doi/10.1007/978-3-540-69293-5_33
Yamahara HTakada HShimakawa H(2007)Behavior detection based on touched objects with dynamic threshold determination modelProceedings of the 2nd European conference on Smart sensing and context10.5555/1775377.1775389(142-158)Online publication date: 23-Oct-2007
https://dl.acm.org/doi/10.5555/1775377.1775389
Yamahara HHarada FTakada HShimakawa H(2007)Improvement of behavior detection by dynamic thresholdProceedings of the 9th WSEAS International Conference on Data Networks, Communications, Computers10.5555/1353540.1353544(351-359)Online publication date: 5-Nov-2007
https://dl.acm.org/doi/10.5555/1353540.1353544
Sun BTan QMitra PGiles CWilliamson CZurko MPatel-Schneider PShenoy P(2007)Extraction and search of chemical formulae in text documents on the webProceedings of the 16th international conference on World Wide Web10.1145/1242572.1242607(251-260)Online publication date: 8-May-2007
https://dl.acm.org/doi/10.1145/1242572.1242607
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents