Abstract
Various ways of estimating probabilities, mainly within the Bayesian framework, are discussed. Their relevance and application to machine learning is given, and their relative performance empirically evaluated. A method of accounting for noisy data is given and also applied. The reliability of estimates is measured by a significance measure, which is also empirically tested. We briefly discuss the use of likelihood ratio as a significance measure.
Chapter PDF
Similar content being viewed by others
Keywords
- Prior Distribution
- Maximum Likelihood Estimator
- Probability Estimate
- Beta Distribution
- Bayesian Estimation
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Dana Angluin and Philip Laird. Learning from noisy examples. Machine Learning, 2:343–370, 1988.
Yvonne M. M. Bishop, Stephen E. Fienberg, and Paul W. Holland. Discrete Multivariate Analysis: Theory and Practice. MIT Press, Cambridge, Mass., 1975.
George E. P. Box and George C. Tiao. Bayesian Inference in Statistical Analysis. Addison-Wesley, Reading, Mass., 1973.
Bojan Cestnik. Estimating probabilities: A crucial task in machine learning. In Aiello, editor, ECAI-90, pages 147–149. Pitman, 1990.
Bojan Cestnik and Ivan Bratko. On estimating probabilities in tree pruning. In Yves Kodratoff, editor, Machine Learning—EWSL-91, pages 138–150. Lecture Notes in Artificial Intelligence 482, Springer, 1991.
Peter Clark and Tim Niblett. The CN2 induction algorithm. Machine Learning, 3(4):261–283, 1989.
Luc de Raedt. Interactive Theory Revision: An Inductive Logic Programming Approach. Academic Press, London, 1992.
Sašo Džeroski and Ivan Bratko. Using the m-estimate in inductive logic programming. In Logical Approaches to Machine Learning, August 1992. ECAI-92 Workshop Notes.
Sašo Džeroski, Bojan Cestnik, and Igor Petrovski. The use of Bayesian probability estimates in rule induction. Turing Institute Research Memorandum TIRM-92-051, The Turing Institute, Glasgow, 1992.
Stephen E. Fienberg and Paul W. Holland. On the choice of flattening constant for estimating multinomial probabilities. Journal of Multivariate Analysis, 2:127–134, 1972.
Marek Fisz. Probability Theory and Mathematical Statistics. John Wiley, New York, third edition, 1963.
Jean Dickinson Gibbons. Nonparametric Statistical Inference. McGraw-Hill, New York, 1971.
I. J. Good. The Estimation of Probabilities: An Essay on Modern Bayesian Techniques. M. I. T. Press, Cambridge, Mass, 1965.
David Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelliegence, 36:177–221, 1988.
Colin Howson and Peter Urbach. Scientific Reasoning: The Bayesian Approach. Open Court, La Salle, Illinois, 1989.
Nada Lavrač, Bojan Cestnik, and Sašo Džeroski. Search heuristics in empirical inductive logic programming. In Logical Approaches to Machine Learning, August 1992. ECAI-92 Workshop Notes.
Stephen Muggleton and Cao Feng. Efficient induction of logic programs. In Proceedings of the First Conference on Algorithmic Learning Theory, pages 473–491, Tokyo, 1990.
Stephen Muggleton, Ashwin Srinivasan, and Michael Bain. Compression, significance and accuracy. In IML 92, 1992.
J.R. Quinlan. Induction of decision trees. Machine Learning, 1:81–106, 1986.
Stuart J. Russell and Benjamin N. Grosof. Declarative bias: An overview. InD. Paul Benjamin, editor, Change of Representation and Inductive Bias, pages 267–308. Kluwer, Boston, 1990.
Cullen Schaffer. When does overfitting decrease prediction accuracy in induced decision trees and rule sets? In Yves Kodratoff, editor, Machine Learning—EWSL-91, pages 192–205. Lecture Notes in Artificial Intelligence 482, Springer, 1991.
Ashwin Srinivasan, Stephen Muggleton, and Michael Bain. The justification of logical theories. In Stephen Muggleton, editor, Machine Intelligence 13. Oxford University Press, 1993. To appear.
Michael Sutherland, Paul W. Holland, and Stephen E. Fienberg. Combining Bayes and frequency approaches to estimate a multinomial parameter. In Stephen E. Fienberg and Arnold Zellner, editors, Studies in Bayesian Econometrics and Statistics, pages 275–307. North-Holland, Amsterdam, 1974. volume 2.
Paul E. Utgoff. Machine Learning of Inductive Bias. Kluwer, Boston Mass., 1986.
Robert L. Winkler. Introduction to Bayesian Inference and Decision. Holt, Rhinehart and Winston, New York, 1972.
Robert L. Winkler. Information loss in noisy and dependent processes. In J. M. Bernardo, M. H. Groot, D. V. Lindley, and A. F. M. Smith, editors, Bayesian Statistics, volume 2, pages 559–570. North-Holland, 1985.
Robert L. Winkler and Leroy A. Franklin. Warner's randomized response model: A Bayesian approach. Journal of the American Statistical Association, 74(365):207–214, March 1979.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1993 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cussens, J. (1993). Bayes and pseudo-Bayes estimates of conditional probabilities and their reliability. In: Brazdil, P.B. (eds) Machine Learning: ECML-93. ECML 1993. Lecture Notes in Computer Science, vol 667. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-56602-3_133
Download citation
DOI: https://doi.org/10.1007/3-540-56602-3_133
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-56602-1
Online ISBN: 978-3-540-47597-2
eBook Packages: Springer Book Archive