Meta–Gradient Boosted Decision Tree Model for Weight and Target Learning

Yury Ustinovskiy, Valentina Fedorova, Gleb Gusev, Pavel Serdyukov
Proceedings of The 33rd International Conference on Machine Learning, PMLR 48:2692-2701, 2016.

Abstract

Labeled training data is an essential part of any supervised machine learning framework. In practice, there is a trade-off between the quality of a label and its cost. In this paper, we consider a problem of learning to rank on a large-scale dataset with low-quality relevance labels aiming at maximizing the quality of a trained ranker on a small validation dataset with high-quality ground truth relevance labels. Motivated by the classical Gauss-Markov theorem for the linear regression problem, we formulate the problems of (1) reweighting training instances and (2) remapping learning targets. We propose meta–gradient decision tree learning framework for optimizing weight and target functions by applying gradient-based hyperparameter optimization. Experiments on a large-scale real-world dataset demonstrate that we can significantly improve state-of-the-art machine-learning algorithms by incorporating our framework.

Cite this Paper

BibTeX
@InProceedings{pmlr-v48-ustinovskiy16, title = {Meta--Gradient Boosted Decision Tree Model for Weight and Target Learning}, author = {Ustinovskiy, Yury and Fedorova, Valentina and Gusev, Gleb and Serdyukov, Pavel}, booktitle = {Proceedings of The 33rd International Conference on Machine Learning}, pages = {2692--2701}, year = {2016}, editor = {Balcan, Maria Florina and Weinberger, Kilian Q.}, volume = {48}, series = {Proceedings of Machine Learning Research}, address = {New York, New York, USA}, month = {20--22 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v48/ustinovskiy16.pdf}, url = {https://proceedings.mlr.press/v48/ustinovskiy16.html}, abstract = {Labeled training data is an essential part of any supervised machine learning framework. In practice, there is a trade-off between the quality of a label and its cost. In this paper, we consider a problem of learning to rank on a large-scale dataset with low-quality relevance labels aiming at maximizing the quality of a trained ranker on a small validation dataset with high-quality ground truth relevance labels. Motivated by the classical Gauss-Markov theorem for the linear regression problem, we formulate the problems of (1) reweighting training instances and (2) remapping learning targets. We propose meta–gradient decision tree learning framework for optimizing weight and target functions by applying gradient-based hyperparameter optimization. Experiments on a large-scale real-world dataset demonstrate that we can significantly improve state-of-the-art machine-learning algorithms by incorporating our framework.} }
Endnote
%0 Conference Paper %T Meta–Gradient Boosted Decision Tree Model for Weight and Target Learning %A Yury Ustinovskiy %A Valentina Fedorova %A Gleb Gusev %A Pavel Serdyukov %B Proceedings of The 33rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2016 %E Maria Florina Balcan %E Kilian Q. Weinberger %F pmlr-v48-ustinovskiy16 %I PMLR %P 2692--2701 %U https://proceedings.mlr.press/v48/ustinovskiy16.html %V 48 %X Labeled training data is an essential part of any supervised machine learning framework. In practice, there is a trade-off between the quality of a label and its cost. In this paper, we consider a problem of learning to rank on a large-scale dataset with low-quality relevance labels aiming at maximizing the quality of a trained ranker on a small validation dataset with high-quality ground truth relevance labels. Motivated by the classical Gauss-Markov theorem for the linear regression problem, we formulate the problems of (1) reweighting training instances and (2) remapping learning targets. We propose meta–gradient decision tree learning framework for optimizing weight and target functions by applying gradient-based hyperparameter optimization. Experiments on a large-scale real-world dataset demonstrate that we can significantly improve state-of-the-art machine-learning algorithms by incorporating our framework.
RIS
TY - CPAPER TI - Meta–Gradient Boosted Decision Tree Model for Weight and Target Learning AU - Yury Ustinovskiy AU - Valentina Fedorova AU - Gleb Gusev AU - Pavel Serdyukov BT - Proceedings of The 33rd International Conference on Machine Learning DA - 2016/06/11 ED - Maria Florina Balcan ED - Kilian Q. Weinberger ID - pmlr-v48-ustinovskiy16 PB - PMLR DP - Proceedings of Machine Learning Research VL - 48 SP - 2692 EP - 2701 L1 - http://proceedings.mlr.press/v48/ustinovskiy16.pdf UR - https://proceedings.mlr.press/v48/ustinovskiy16.html AB - Labeled training data is an essential part of any supervised machine learning framework. In practice, there is a trade-off between the quality of a label and its cost. In this paper, we consider a problem of learning to rank on a large-scale dataset with low-quality relevance labels aiming at maximizing the quality of a trained ranker on a small validation dataset with high-quality ground truth relevance labels. Motivated by the classical Gauss-Markov theorem for the linear regression problem, we formulate the problems of (1) reweighting training instances and (2) remapping learning targets. We propose meta–gradient decision tree learning framework for optimizing weight and target functions by applying gradient-based hyperparameter optimization. Experiments on a large-scale real-world dataset demonstrate that we can significantly improve state-of-the-art machine-learning algorithms by incorporating our framework. ER -
APA
Ustinovskiy, Y., Fedorova, V., Gusev, G. & Serdyukov, P.. (2016). Meta–Gradient Boosted Decision Tree Model for Weight and Target Learning. Proceedings of The 33rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 48:2692-2701 Available from https://proceedings.mlr.press/v48/ustinovskiy16.html.

Related Material