An uncertainty framework for classification
Pages 574 - 579
Abstract
We define a generalized likelihood function based on uncertainty measures and show that maximizing such a likelihood function for different measures induces different types of classifiers. In the probabilistic framework, we obtain classifiers that optimize the cross-entropy function. In the possibilistic framework, we obtain classifiers that maximize the interclass margin. Furthermore, we show that the support vector machine is a sub-class of these maximummargin classifiers.
References
[1]
Baum E.B., & Wilczek F. (1988). Supervised learning of probability distributions by neural networks. In Neural Information Processing Systems, editor D.Z. Anderson, pp.52-61.
[2]
Bezdek J.C., Keller J., Krisnapuram R., & Pal N.R., (1999). Fuuy Models and Algorithms for Pattern Recognition and Image Processing. Kluwer Academic Publishers.
[3]
Burges C.J.C. (1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2, pp. 121-167.
[4]
Cortes C., & Vapnik V. (1995). Support vector networks. Machine Learning, 20, pp.273-297.
[5]
Dempster A., Laird N., & Rubin D. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, B39, pp.1-38.
[6]
Devroye L., Gyorfi L., & Lugosi G. (1996). A Probabilistic neory of Pattern Recognition. New York: Springer.
[7]
Gupta M.M., & Qi J. (1991). Theory of t-norms and fuzzy inference methods. Fuuy Sets and Systems, 40(3), pp.431-450.
[8]
Hertz, J., Krogh A., & Palmer R.G. (1991). Introduction to the Theory of Neural Computation. Addison-Wesley.
[9]
Hinton G.E. (1989). Connectionist learning procedures. Artificial Intelligence, 40, pp.185-234.
[10]
Jordan M., & Jacobs R. (1994). Hierarchical mixture of experts and the EM algorithm. Neural Computation, 6, pp.181-214.
[11]
Jurgen, S. (1996). Pattern classification: a unified view of statistical and neural approaches. New York: Wiley.
[12]
Klir G. J., & Yuan B. (1995). Fuuy Sets and Fuzzy Logic - Theory and Applications, New Jersey: Prentice Hall.
[13]
Langley P., Iba W., & Thompson K. (1992). An analysis of Bayesian classifiers. Proceedings of the Tenth National Conference on Artificial Intelligence (AAAI-92), pp. 223- 228.
[14]
Platt J.C. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In Advances in Large Margin Classifiers, editors A. Smola, P. Bartlett, B. Scholkopf, & D. Schuurmans. Cambridge: MIT-Press.
[15]
Rosenblatt F. (1962). Principles of Neurodynamics. New York: Spartan.
[16]
Rumelhart D.E., Hinton G.E., & Williams R.J. (1986). "Learning internal representations by error propagation". In Parallel Distributed Processing: Explorations in the Micro-structure of Cognition, Volume 1: Foundations, editors D.E. Rumelhart, & J.L. McClelland, Chap.8, pp.318-362. MIT press, Cambridge.
[17]
Scholkopf B., Sung K., Burges C., Girosi F., Nigogi P., Poggio T., & Vapnik V. (1997). Comparing support vector machines with Gaussian kernels to radial basis function classifiers. IEEE Transactions on Signal Processing, 45, pp.2758-2765.
[18]
Sollich P. (1999). Probabilistic interpretations and Bayesian methods for support vector machines. Proceedings of the Ninth International Conference on Artificial Neural Networks (ICANN'99), pp.91-96.
[19]
Tanaka H., & Guo P. (1999). Discriminant analysis based on possibility distributions. Possibilistic Data Analysis for Operations Research, Chap.7, pp.149-164. Heidelberg: Physica-Verlag.
[20]
Yang Y., & Liu X. (1999). A re-examination of text categorization methods. Proceedings of the 22nd International Conference on R&D in Information Retrieval (SIGIR'99), pp.42-49.
[21]
Vapnik V. (1995). The Nature of Statistical Learning Theory. New York: Springer.
Index Terms
- An uncertainty framework for classification
Recommendations
Uncertainty Quantification for Text Classification
Advances in Information RetrievalAbstractThis half-day tutorial introduces modern techniques for practical uncertainty quantification specifically in the context of multi-class and multi-label text classification. First, we explain the usefulness of estimating aleatoric uncertainty and ...
Uncertainty Quantification for Text Classification
SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information RetrievalThis full-day tutorial introduces modern techniques for practical uncertainty quantification specifically in the context of multi-class and multi-label text classification. First, we explain the usefulness of estimating aleatoric uncertainty and ...
A Framework towards the Unification of Ensemble Classification Methods
ICMLA '13: Proceedings of the 2013 12th International Conference on Machine Learning and Applications - Volume 02Multiple classifier systems, also known as classifier ensembles, have received great attention in recent years because of the improved classification accuracy in different applications. A large variety of ensemble methods have been proposed in order to ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Sponsors
- HUGIN: Hugin Expert A/S
- Information Extraction and Transportation
- Hewlett-Packard
- Fair, Isaac and Company, Inc.: Fair, Isaac and Company, Inc.
- AT&T: AT&T Labs Research
Publisher
Morgan Kaufmann Publishers Inc.
San Francisco, CA, United States
Publication History
Published: 30 June 2000
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 51Total Downloads
- Downloads (Last 12 months)40
- Downloads (Last 6 weeks)11
Reflects downloads up to 23 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in