Abstract
The dynamical evolution of weights in the AdaBoost algorithm contains useful information about the rôle that the associated data points play in the built of the AdaBoost model. In particular, the dynamics induces a bipartition of the data set into two (easy/hard) classes. Easy points are ininfluential in the making of the model, while the varying relevance of hard points can be gauged in terms of an entropy value associated to their evolution. Smooth approximations of entropy highlight regions where classification is most uncertain. Promising results are obtained when methods proposed are applied in the Optimal Sampling framework.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Y. Freund and R. E. Schapire, “A Decision-theoretic Generalization of Online Learning and an Application to Boosting,” Journal of Computer and System Sciences, vol. 55, pp. 119–139, August 1997.
R. E. Schapire, Y. Freund, P. Bartlett, and W. S. Lee, “Boosting the Margin: A New Explanation for the Effectiveness of Voting Methods,” The Annals of Statistics, vol. 26, no. 5, pp. 1651–1686, 1998.
J. Friedman, T. Hastie, and R. Tibshirani, “Additive logistic regression: a statistical view of boosting,“ The Annals of Statistics, 2000.
R. E. Schapire, “The boosting approach to machine learning: An overview,” in MSRI Workshop on Nonlinear Estimation and Classification, 2002.
J. Quinlan, “Bagging, Boosting, and C4.5,” in Thirteenth National Conference on Artificial Intelligence, (Cambridge), pp. 163–175, AAAI Press/MIT Press, 1996.
T. G. Dietterich, “An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization,” Machine Learning, vol. 40, no. 2, pp. 139–157, 2000.
A. J. Grove and D. Schuurmans, “Boosting in the limit: Maximizing the margin of learned ensembles,” in AAAI/IAAI, pp. 692–699, 1998.
G. Rätsch, T. Onoda, and K. Müller, “Soft margins for Adaboost,” Machine Learning, vol. 42, pp. 287–320, 2001.
V. Vapnik, The nature of statistical learning theory. Statistics for Engineering and Information Science, Springer Verlag, 2000.
W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C-The Art of Scientific Computing. Cambridge University Press, second ed., 1992.
W. Härdle, Applied Nonparametric Regression, vol. 19 of Econometric Society Monographs. Cambridge University Press, 1990.
V. Fedorov, Theory of Optimal Experiments. Academic Press, New York, 1972.
D. D. Lewis and J. Catlett, “Heterogeneous Uncertainty Sampling for Supervised Learning,” in Eleventh International Conference on Machine Learning (Cohen and Hirsh, eds.), (San Francisco), pp. 148–156, Morgan Kaufmann, 1994.
Y. Raviv and N. Intrator, “Variance Reduction via Noise and Bias Constraints.,” in Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems (A. Sharkey, ed.), (London), pp. 163–175, Springer-Verlag, 1999.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Caprile, B., Furlanello, C., Merler, S. (2002). Highlighting Hard Patterns via AdaBoost Weights Evolution. In: Roli, F., Kittler, J. (eds) Multiple Classifier Systems. MCS 2002. Lecture Notes in Computer Science, vol 2364. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45428-4_7
Download citation
DOI: https://doi.org/10.1007/3-540-45428-4_7
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43818-2
Online ISBN: 978-3-540-45428-1
eBook Packages: Springer Book Archive