Article

Co-EM support vector learning

Authors:

Tobias SchefferAuthors Info & Claims

ICML '04: Proceedings of the twenty-first international conference on Machine learning

Page 16

https://doi.org/10.1145/1015330.1015350

Published: 04 July 2004 Publication History

Abstract

Multi-view algorithms, such as co-training and co-EM, utilize unlabeled data when the available attributes can be split into independent and compatible subsets. Co-EM outperforms co-training for many problems, but it requires the underlying learner to estimate class probabilities, and to learn from probabilistically labeled data. Therefore, co-EM has so far only been studied with naive Bayesian learners. We cast linear classifiers into a probabilistic framework and develop a co-EM version of the Support Vector Machine. We conduct experiments on text classification problems and compare the family of semi-supervised support vector algorithms under different conditions, including violations of the assumptions underlying multi-view learning. For some problems, such as course web page classification, we observe the most accurate results reported so far.

References

[1]

Baluja, S. (1998). Probabilistic modeling for face orientation discrimination: Learning from labeled and unlabeled data. Advances in Neural Information Processing Systems.

Digital Library

[2]

Bennett, K. (1999). Combining support vector and mathematical programming methods for classification. Advances in Kernel Methods - Support Vector Learning. MIT Press.

Digital Library

[3]

Blum, A., & Mitchell, T. (1998). Combining labeled and unlabeled data with co-training. Proceedings of the Conference on Computational Learning Theory (pp. 92--100).

Digital Library

[4]

Bradley, A. (1997). The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition, 30, 1145--1159.

Digital Library

[5]

Brefeld, U., Geibel, P., & Wysotzki, F. (2003). Support vector machines with example dependent costs. Proceedings of the European Conference on Machine Learning.

Digital Library

[6]

Collins, M., & Singer, Y. (1999). Unsupervised models for named entity classification. Proceedings of the Conference on Empirical Methods in Natural Language Processing.

[7]

Cooper, D., & Freeman, J. (1970). On the asymptotic improvement in the outcome of supervised learning provided by additional nonsupervised learning. IEEE Transactions on Computers, C-19, 1055--1063.

[8]

Cozman, F., Cohen, I., & Cirelo, M. (2003). Semi-supervised learning of mixture models. Proceedings of the International Conference on Machine Learning (pp. 99--106).

[9]

Dempster, A., Laird, N., & Rubin, D. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B, 39.

[10]

Denis, F., Laurent, A., Gilleron, R., & Tommasi, M. (2003). Text classification and co-training from positive and unlabeled examples. ICML Workshop on the Continuum from Labeled to Unlabeled Data.

[11]

Ghani, R. (2002). Combining labeled and unlabeled data for multiclass text categorization. Proceedings of the International Conference on Machine Learning.

Digital Library

[12]

Joachims, T. (1999a). Making large-scale SVM learning practical. Advances in Kernel Methods - Support Vector Learning. MIT Press.

Digital Library

[13]

Joachims, T. (1999b). Transductive inference for text classification using support vector machines. Proceedings of the International Conference on Machine Learning.

Digital Library

[14]

Joachims, T. (2003). Transductive learning via spectral graph partitioning. Proceedings of the International Conference on Machine Learning.

[15]

Kiritchenko, S., & Matwin, S. (2002). Email classification with co-training (Technical Report). University of Ottawa.

[16]

Kockelkorn, M., Lüneburg, A., & Scheffer, T. (2003). Using transduction and multi-view learning to answer emails. Proceedings of the European Conference on Principle and Practice of Knowledge Discovery in Databases.

[17]

McCallum, A., & Nigam, K. (1998). Employing EM in pool-based active learning for text classification. Proceedings of the International Conference on Machine Learning.

Digital Library

[18]

Mladenic, D. (2002). Learning word normalization using word suffix and context from unlabeled data. Proceedings of the International Conference on Machine Learning (pp. 427--434).

Digital Library

[19]

Muslea, I., Kloblock, C., & Minton, S. (2002a). Active + semi-supervised learning = robust multi-view learning. Proceedings of the International Conference on Machine Learning (pp. 435--442).

Digital Library

[20]

Muslea, I., Kloblock, C., & Minton, S. (2002b). Adaptive view validation: A first step towards automatic view detection. Proceedings of the International Conference on Machine Learning (pp. 443--450).

Digital Library

[21]

Nigam, K., & Ghani, R. (2000). Analyzing the effectiveness and applicability of co-training. Proceedings of Information and Knowledge Management.

Digital Library

[22]

Nigam, K., McCallum, A. K., Thrun, S., & Mitchell, T. M. (2000). Text classification from labeled and unlabeled documents using EM. Machine Learning, 39.

Digital Library

[23]

Provost, F., Fawcett, T., & Kohavi, R. (1998). The case against accuracy estimation for comparing inductive algorithms. Proceedings of the International Conference on Machine Learning (pp. 445--453).

Digital Library

[24]

Seeger, M. (2001). Learning with labeled and unlabeled data. (Technical Report). University of Edinburgh.

[25]

Shahshahani, B., & Landgrebe, D. (1994). The effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon. IEEE Transactions on Geoscience and Remote Sensing, 32, 1087--1095.

[26]

Vapnik, V. (1998). Statistical learning theory. Wiley.

Digital Library

Cited By

Kim HShin S(2024)L1-penalized AUC-optimization with a surrogate lossCommunications for Statistical Applications and Methods10.29220/CSAM.2024.31.2.20331:2(203-212)Online publication date: 31-Mar-2024
https://doi.org/10.29220/CSAM.2024.31.2.203
Diskin TBeer YOkun UWiesel A(2024)CFARnet: Deep learning for target detection with constant false alarm rateSignal Processing10.1016/j.sigpro.2024.109543223(109543)Online publication date: Oct-2024
https://doi.org/10.1016/j.sigpro.2024.109543
Sun CChen JZhao YHan HJing RTan GWu D(2024)Appformer: A novel framework for mobile app usage prediction leveraging progressive multi-modal data fusion and feature extractionExpert Systems with Applications10.1016/j.eswa.2024.125903(125903)Online publication date: Nov-2024
https://doi.org/10.1016/j.eswa.2024.125903
Show More Cited By

Co-EM support vector learning
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches

Recommendations

Semi-supervised learning combining transductive support vector machine with active learning

In typical data mining applications, labeling the large amounts of data is difficult, expensive, and time consuming, if annotated manually. To avoid manual labeling, semi-supervised learning uses unlabeled data along with the labeled data in the ...
Learning with progressive transductive support vector machine

Support vector machine (SVM) is a new learning method developed in recent years based on the foundations of statistical learning theory. By taking a transductive approach instead of an inductive one in support vector classifiers, the working set can be ...
Co-EM Support Vector Machine Based Text Classification from Positive and Unlabeled Examples
ICINIS '08: Proceedings of the 2008 First International Conference on Intelligent Networks and Intelligent Systems

This paper has brought about a novel method based on multi-view algorithms for learning from positive and unlabeled examples (LPU). First we, with an improved 1-DNF method, split the text feature into a positive feature set (PF) and a negative feature ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '04: Proceedings of the twenty-first international conference on Machine learning

July 2004

934 pages

ISBN:1581138385

DOI:10.1145/1015330

Conference Chair:
Carla Brodley
Purdue University/Tufts University

Copyright © 2004 Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 July 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

145
Total Citations
View Citations
834
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)1

Reflects downloads up to 30 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kim HShin S(2024)L1-penalized AUC-optimization with a surrogate lossCommunications for Statistical Applications and Methods10.29220/CSAM.2024.31.2.20331:2(203-212)Online publication date: 31-Mar-2024
https://doi.org/10.29220/CSAM.2024.31.2.203
Diskin TBeer YOkun UWiesel A(2024)CFARnet: Deep learning for target detection with constant false alarm rateSignal Processing10.1016/j.sigpro.2024.109543223(109543)Online publication date: Oct-2024
https://doi.org/10.1016/j.sigpro.2024.109543
Sun CChen JZhao YHan HJing RTan GWu D(2024)Appformer: A novel framework for mobile app usage prediction leveraging progressive multi-modal data fusion and feature extractionExpert Systems with Applications10.1016/j.eswa.2024.125903(125903)Online publication date: Nov-2024
https://doi.org/10.1016/j.eswa.2024.125903
Sun YVong CWang S(2024)Adversarial de-overlapping learning machines for supervised and semi-supervised learningInternational Journal of Machine Learning and Cybernetics10.1007/s13042-024-02389-9Online publication date: 7-Oct-2024
https://doi.org/10.1007/s13042-024-02389-9
Liu LZuo HMin F(2024)BSRU: boosting semi-supervised regressor through ramp-up unsupervised lossKnowledge and Information Systems10.1007/s10115-023-02044-966:5(2769-2797)Online publication date: 18-Jan-2024
https://doi.org/10.1007/s10115-023-02044-9
Adsule ARoy SSharma ASengupta S(2024)SiamALNet: A Semi-supervised Siamese Neural Network with Active Learning Approach for Auto-LabelingProceedings of World Conference on Artificial Intelligence: Advances and Applications10.1007/978-981-97-4496-1_20(257-269)Online publication date: 1-Oct-2024
https://doi.org/10.1007/978-981-97-4496-1_20
France KSheppard JSilva SPaquete L(2023)Factored Particle Swarm Optimization for Policy Co-training in Reinforcement LearningProceedings of the Genetic and Evolutionary Computation Conference10.1145/3583131.3590376(30-38)Online publication date: 15-Jul-2023
https://dl.acm.org/doi/10.1145/3583131.3590376
Gong MZhou HQin ALiu WZhao Z(2023)Self-Paced Co-Training of Graph Neural Networks for Semi-Supervised Node ClassificationIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.315768834:11(9234-9247)Online publication date: Nov-2023
https://doi.org/10.1109/TNNLS.2022.3157688
Shi MZhao XYin XChang XNiu FGuo J(2023)Multiview Latent Structure LearningKnowledge-Based Systems10.1016/j.knosys.2023.110707276:COnline publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1016/j.knosys.2023.110707
Shi HXie MHuang S(2023)Robust AUC maximization for classification with pairwise confidence comparisonsFrontiers of Computer Science10.1007/s11704-023-2709-518:4Online publication date: 16-Dec-2023
https://doi.org/10.1007/s11704-023-2709-5
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents