article

Learning to Rank for Information Retrieval

Author:

Tie-Yan LiuAuthors Info & Claims

Foundations and Trends in Information Retrieval, Volume 3, Issue 3

Pages 225 - 331

https://doi.org/10.1561/1500000016

Published: 01 March 2009 Publication History

Abstract

Learning to rank for Information Retrieval (IR) is a task to automatically construct a ranking model using training data, such that the model can sort new objects according to their degrees of relevance, preference, or importance. Many IR problems are by nature ranking problems, and many IR technologies can be potentially enhanced by using learning-to-rank techniques. The objective of this tutorial is to give an introduction to this research direction. Specifically, the existing learning-to-rank algorithms are reviewed and categorized into three approaches: the pointwise, pairwise, and listwise approaches. The advantages and disadvantages with each approach are analyzed, and the relationships between the loss functions used in these approaches and IR evaluation measures are discussed. Then the empirical evaluations on typical learning-to-rank methods are shown, with the LETOR collection as a benchmark dataset, which seems to suggest that the listwise approach be the most effective one among all the approaches. After that, a statistical ranking theory is introduced, which can describe different learning-to-rank algorithms, and be used to analyze their query-level generalization abilities. At the end of the tutorial, we provide a summary and discuss potential future work on learning to rank.

References

[1]

S. Agarwal, T. Graepel, R. Herbrich, S. Har-Peled, and D. Roth, "Generalization bounds for the area under the ROC curve," Journal of Machine Learning , vol. 6, pp. 393-425, 2005.

Digital Library

[2]

S. Agarwal and P. Niyogi, "Stability and generalization of bipartite ranking algorithms," in COLT 2005 , pp. 32-47, 2005.

[3]

E. Agichtein, E. Brill, S. T. Dumais, and R. Ragno, "Learning user interaction models for predicting web search result preferences," in SIGIR 2006 , pp. 3-10, 2006.

[4]

H. Almeida, M. Goncalves, M. Cristo, and P. Calado, "A combined component approach for finding collection-adapted ranking functions based on genetic programming," in SIGIR 2007 , pp. 399-406, 2007.

[5]

M.-R. Amini, T.-V. Truong, and C. Goutte, "A boosting algorithm for learning bipartite ranking functions with partially labeled data," in SIGIR 2008 , pp. 99-106, 2008.

[6]

S. Andrews, I. Tsochantaridis, and T. Hofmann, "Support vector machines for multiple-instance learning," in NIPS 2003 , pp. 561-568, 2003.

[7]

J. A. Aslam and M. Montague, "Models for metasearch," in SIGIR 2001 , pp. 276-284, 2001.

[8]

R. Baeza-Yates and B. Ribeiro-Neto, Modern Information Retrieval . Addison Wesley, May 1999.

Digital Library

[9]

B. Bartell, G. W. Cottrell, and R. Belew, "Learning to retrieve information," in SCC 1995 , 1995.

[10]

P. L. Bartlett and S. Mendelson, "Rademacher and Gaussian complexities risk bounds and structural results," Journal of Machine Learning , pp. 463-482, 2002.

[11]

O. Bousquet, S. Boucheron, and G. Lugosi, "Introduction to statistical learning theory," in Advanced Lectures on Machine Learning , pp. 169-207, Berlin/Heidelberg: Springer, 2004.

[12]

O. Bousquet and A. Elisseeff, "Stability and generalization," The Journal of Machine Learning Research , vol. 2, pp. 449-526, 2002.

[13]

C. J. Burges, R. Ragno, and Q. V. Le, "Learning to rank with nonsmooth cost functions," in NIPS 2006 , pp. 395-402, 2006.

[14]

C. J. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender, "Learning to rank using gradient descent," in ICML 2005 , pp. 89-96, 2005.

Digital Library

[15]

G. Cao, J. Nie, L. Si, and J. Bai, "Learning to rank documents for ad-hoc retrieval with regularized models," in SIGIR 2007 Workshop on Learning to Rank for Information Retrieval , 2007.

[16]

Y. Cao, J. Xu, T.-Y. Liu, H. Li, Y. Huang, and H.-W. Hon, "Adapting ranking SVM to document retrieval," in SIGIR 2006 , pp. 186-193, 2006.

[17]

Z. Cao, T. Qin, T.-Y. Liu, M.-F. Tsai, and H. Li, "Learning to rank: From pairwise approach to listwise approach," in ICML 2007 , pp. 129-136, 2007.

Digital Library

[18]

B. Carterette, V. Pavlu, E. Kanoulas, J. A. Aslam, and J. Allan, "Evaluation over thousands of queries," in SIGIR 2008 , pp. 651-658, 2008.

[19]

V. R. Carvalho, J. L. Elsas, W. W. Cohen, and J. G. Carbonell, "A metalearning approach for robust rank learning," in SIGIR 2008 Workshop on Learning to Rank for Information Retrieval , 2008.

[20]

S. Chakrabarti, R. Khanna, U. Sawant, and C. Bhattacharyya, "Structured learning for non-smooth ranking losses," in SIGKDD 2008 , pp. 88-96, 2008.

[21]

O. Chapelle, Q. Le, and A. Smola, "Large margin optimization of ranking measures," in NIPS Workshop on Machine Learning for Web Search 2007 , 2007.

[22]

W. Chen, Y. Lan, T.-Y. Liu, and H. Li, "A unified view on loss functions in learning to rank," Technical Report, Microsoft Research, MSR-TR-2009-39, 2009.

[23]

P. Chirita, J. Diederich, and W. Nejdl, "MailRank: Using ranking for spam detection," in CIKM 2005 , pp. 373-380, New York, NY, USA: ACM, 2005.

[24]

W. Chu and Z. Ghahramani, "Gaussian processes for ordinal regression," Journal of Machine Learning Research , vol. 6, pp. 1019-1041, 2005.

Digital Library

[25]

W. Chu and Z. Ghahramani, "Preference learning with Gaussian processes," in ICML 2005 , pp. 137-144, 2005.

Digital Library

[26]

W. Chu and S. S. Keerthi, "New approaches to support vector ordinal regression," in ICML 2005 , pp. 145-152, 2005.

Digital Library

[27]

S. Clemencon, G. Lugosi, and N. Vayatis, "Ranking and empirical minimization of U -statistics," The Annals of Statistics , vol. 36, pp. 844-874, 2008.

[28]

S. Clemencon and N. Vayatis, "Ranking the best instances," Journal of Machine Learning Research , vol. 8, pp. 2671-2699, December 2007.

Digital Library

[29]

W. W. Cohen, R. E. Schapire, and Y. Singer, "Learning to order things," in NIPS 1998 , Vol. 10, pp. 243-270, 1998.

[30]

M. Collins, "Ranking algorithms for named-entity extraction: Boosting and the voted perceptron," in ACL 2002 , pp. 7-12, 2002.

[31]

W. S. Cooper, F. C. Gey, and D. P. Dabney, "Probabilistic retrieval based on staged logistic regression," in SIGIR 1992 , pp. 198-210, 1992.

[32]

C. Cortes, M. Mohri, and A. Rastogi, "Magnitude-preserving ranking algorithms," in ICML 2007 , pp. 169-176, 2007.

Digital Library

[33]

D. Cossock and T. Zhang, "Subset ranking using regression," in COLT 2006 , pp. 605-619, 2006.

[34]

K. Crammer and Y. Singer, "Pranking with ranking," in NIPS 2002 , pp. 641-647, 2002.

[35]

N. Craswell, D. Hawking, R. Wilkinson, and M. Wu, "Overview of the TREC 2003 Web track," in TREC 2003 , pp. 78-92, 2003.

[36]

K. Dave, S. Lawrence, and D. Pennock, "Mining the peanut gallery: Opinion extraction and semantic classification of product reviews," in WWW 2003 , pp. 519-528, New York, NY, USA: ACM Press, 2003.

[37]

S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman, "Indexing by latent semantic analysis," Journal of the American Society for Information Science , vol. 41, pp. 391-407, 1990.

[38]

F. Diaz, "Regularizing query-based retrieval scores," Information Retrieval , vol. 10, pp. 531-562, 2007.

Digital Library

[39]

H. Drucker, B. Shahrary, and D. C. Gibbon, "Support vector machines: Relevance feedback and information retrieval," Information Processing and Management , vol. 38, pp. 305-323, 2002.

Digital Library

[40]

K. Duh and K. Kirchhoff, "Learning to rank with partially-labeled data," in SIGIR 2008 , pp. 251-258, 2008.

[41]

W. Fan, E. A. Fox, P. Pathak, and H. Wu, "The effects of fitness functions on genetic programming based ranking discovery for web search," Journal of American Society for Information Science and Technology , vol. 55, pp. 628- 636, 2004.

Digital Library

[42]

W. Fan, M. Gordon, and P. Pathak, "Discovery of context-specific ranking functions for effective information retrieval using genetic programming," IEEE Transactions on Knowledge and Data Engineering , vol. 16, pp. 523-527, 2004.

Digital Library

[43]

W. Fan, M. Gordon, and P. Pathak, "A generic ranking function discovery framework by genetic programming for information retrieval," Information Processing and Management , vol. 40, pp. 587-602, 2004.

Digital Library

[44]

W. Fan, M. Gordon, and P. Pathak, "Genetic programming-based discovery of ranking functions for effective web search," Journal of Management of Information Systems , vol. 21, pp. 37-56, 2005.

Digital Library

[45]

W. Fan, M. Gordon, and P. Pathak, "On linear mixture of expert approaches to information retrieval," Decision Support System , vol. 42, pp. 975-987, 2006.

Digital Library

[46]

W. Fan, M. D. Gordon, W. Xi, and E. A. Fox, "Ranking function optimization for effective web search by genetic programming: An empirical study," in HICSS 2004 , pp. 40105, 2004.

[47]

Y. Freund, R. Iyer, R. E. Schapire, and Y. Singer, "An efficient boosting algorithm for combining preferences," Journal of Machine Learning Research , vol. 4, pp. 933-969, 2003.

Digital Library

[48]

Y. Freund and R. E. Schapire, "A decision-theoretic generalization of online learning and an application to boosting," Journal of Computer and System Sciences , vol. 55, pp. 119-139, 1995.

Digital Library

[49]

N. Fuhr, "Optimum polynomial retrieval functions based on the probability ranking principle," ACM Transactions on Information Systems , vol. 7, pp. 183-204, 1989.

Digital Library

[50]

G. Fung, R. Rosales, and B. Krishnapuram, "Learning rankings via convex hull separation," in NIPS 2005 Workshop on Learning to Rank , 2005.

[51]

X.-B. Geng, T.-Y. Liu, and T. Qin, "Feature selection for ranking," in SIGIR 2007 , pp. 407-414, 2007.

[52]

X.-B. Geng, T.-Y. Liu, T. Qin, H. Li, and H.-Y. Shum, "Query-dependent ranking using k -nearest neighbor," in SIGIR 2008 , pp. 115-122, 2008.

[53]

F. C. Gey, "Inferring probability of relevance using the method of logistic regression," in SIGIR 1994 , pp. 222-231, 1994.

[54]

S. Guiasu and A. Shenitzer, "The principle of maximum entropy," The Mathematical Intelligencer , vol. 7, pp. 42-48, 1985.

[55]

J. Guiver and E. Snelson, "Learning to rank with softrank and Gaussian processes," in SIGIR 2008 , pp. 259-266, 2008.

[56]

Z. Gyöngyi, H. Garcia-Molina, and J. Pedersen, "Combating web spam with trustrank," in VLDB 2004 , pp. 576-587, VLDB Endowment, 2004.

[57]

Z. Gyongyi, H. Garcia-Molina, and J. Pedersen, "Combating web spam with trustrank," in VLDB 2004 , pp. 576-587, 2004.

[58]

E. Harrington, "Online ranking/collaborative filtering using the perceptron algorithm," in ICML 2003 , Vol. 20, pp. 250-257, 2003.

[59]

E. F. Harrington, "Online ranking/collaborative filtering using the perceptron algorithm," in ICML 2003 , pp. 250-257, 2003.

[60]

B. He and I. Ounis, "A study of parameter tuning for term frequency normalization," in CIKM 2003 , pp. 10-16, 2003.

[61]

Y. He and T.-Y. Liu, "Are algorithms directly optimizing IR measures really direct?," Technical Report, Microsoft Research, MSR-TR-2008-154, 2008.

[62]

R. Herbrich, T. Graepel, and C. Campbell, "Bayes point machines," Journal of Machine Learning Research , vol. 1, pp. 245-279, 2001.

Digital Library

[63]

R. Herbrich, K. Obermayer, and T. Graepel, "Large margin rank boundaries for ordinal regression," in Advances in Large Margin Classifiers , pp. 115-132, 2000.

[64]

W. Hersh, C. Buckley, T. J. Leone, and D. Hickam, "OHSUMED: An interactive retrieval evaluation and new large test collection for research," in

[65]

K. Järvelin and J. Kekäläinen, "IR evaluation methods for retrieving highly relevant documents," in SIGIR 2000 , pp. 41-48, 2000.

[66]

K. Järvelin and J. Kekäläinen, "Cumulated gain-based evaluation of IR techniques," ACM Transactions on Information Systems , vol. 20, pp. 422-446, 2002.

Digital Library

[67]

R. Jin, H. Valizadegan, and H. Li, "Ranking refinement and its application to information retrieval," in WWW 2008 , pp. 397-406, 2008.

[68]

T. Joachims, "Optimizing search engines using clickthrough data," in KDD 2002 , pp. 133-142, 2002.

[69]

T. Joachims, "Evaluating retrieval performance using clickthrough data," Text Mining , pp. 79-96, 2003.

[70]

T. Joachims, "A support vector method for multivariate performance measures," in CML 2005 , pp. 377-384, 2005.

[71]

I. Kang and G. Kim, "Query type classification for web document retrieval," in SIGIR 2003 , pp. 64-71, 2003.

[72]

J. M. Kleinberg, "Authoritative sources in a hyperlinked environment," Journal of ACM , vol. 46, pp. 604-632, 1999.

Digital Library

[73]

S. Kramer, G. Widmer, B. Pfahringer, and M. D. Groeve, "Prediction of ordinal classes using regression trees," Funfamenta Informaticae , vol. 34, pp. 1-15, 2000.

[74]

J. Lafferty and C. Zhai, "Document language models, query models and risk minimization for information retrieval," in SIGIR 2001 , pp. 111-119, 2001.

[75]

Y. Lan and T.-Y. Liu, "Generalization analysis of listwise learning-to-rank algorithms," in ICML 2009 , 2009.

[76]

Y. Lan, T.-Y. Liu, T. Qin, Z. Ma, and H. Li, "Query-level stability and generalization in learning to rank," in ICML 2008 , pp. 512-519, 2008.

Digital Library

[77]

F. Li and Y. Yang, "A loss function analysis for classification methods in text categorization," in ICML 2003 , pp. 472-479, 2003.

[78]

P. Li, C. Burges, and Q. Wu, "McRank: Learning to rank using multiple classification and gradient boosting," in NIPS 2007 , pp. 845-852, 2007.

[79]

T.-Y. Liu, J. Xu, T. Qin, W.-Y. Xiong, and H. Li, "LETOR: Benchmark dataset for research on learning to rank for information retrieval," in SIGIR '07 Workshop on Learning to Rank for Information Retrieval , 2007.

[80]

Y. Liu, T.-Y. Liu, T. Qin, Z. Ma, and H. Li, "Supervised rank aggregation," in WWW 2007 , pp. 481-490, 2007.

[81]

R. D. Luce, Individual Choice Behavior . New York: Wiley, 1959.

[82]

C. L. Mallows, "Non-null ranking models," Biometrika , vol. 44, pp. 114-130, 1975.

[83]

M. E. Maron and J. L. Kuhns, "On relevance, probabilistic indexing and information retrieval," Journal of the ACM , vol. 7, pp. 216-244, 1960.

Digital Library

[84]

I. Matveeva, C. Burges, T. Burkard, A. Laucius, and L. Wong, "High accuracy retrieval with multiple nested ranker," in SIGIR 2006 , pp. 437-444, 2006.

[85]

D. A. Metzler and W. B. Croft, "A Markov random field model for term dependencies," in SIGIR 2005 , pp. 472-479, 2005.

[86]

D. A. Metzler, W. B. Croft, and A. McCallum, "Direct maximization of rankbased metrics for information retrieval," in CIIR Technical Report , 2005.

[87]

D. A. Metzler and T. Kanungo, "Machine learned sentence selection strategies for query-biased summarization," in SIGIR 2008 Workshop on Learning to Rank for Information Retrieval , 2008.

[88]

T. Minka and S. Robertson, "Selection bias in the LETOR datasets," in SIGIR 2008 Workshop on Learning to Rank for Information Retrieval , 2008.

[89]

T. Mitchell, Machine Learning . McGraw Hill, 1997.

[90]

R. Nallapati, "Discriminative models for information retrieval," in SIGIR 2004 , pp. 64-71, 2004.

[91]

L. Nie, B. D. Davison, and X. Qi, "Topical link analysis for web search," in SIGIR 2006 , pp. 91-98, 2006.

[92]

L. Page, S. Brin, R. Motwani, and T. Winograd, "The pagerank citation ranking: Bringing order to the web," Technical Report, Stanford Digital Library Technologies Project, 1998.

[93]

T. Pahikkala, E. Tsivtsivadze, A. Airola, J. Boberg, and T. Salakoski, "Learning to rank with pairwise regularized least-squares," in SIGIR 2007 Workshop on Learning to Rank for Information Retrieval , 2007.

[94]

B. Pang and L. Lee, "Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales," in ACL 2005 , pp. 115-124, NJ, USA: Association for Computational Linguistics Morristown, 2005.

[95]

R. L. Plackett, "The analysis of permutations," Applied Statistics , vol. 24, pp. 193-202, 1975.

[96]

J. M. Ponte and W. B. Croft, "A language modeling approach to information retrieval," in SIGIR 1998 , pp. 275-281, 1998.

[97]

T. Qin, T.-Y. Liu, W. Lai, X.-D. Zhang, D.-S. Wang, and H. Li, "Ranking with multiple hyperplanes," in SIGIR 2007 , pp. 279-286, 2007.

[98]

T. Qin, T.-Y. Liu, and H. Li, "A general approximation framework for direct optimization of information retrieval measures," Technical Report, Microsoft Research, MSR-TR-2008-164, 2008.

[99]

T. Qin, T.-Y. Liu, M.-F. Tsai, X.-D. Zhang, and H. Li, "Learning to search web pages with query-level loss functions," Technical Report, Microsoft Research, MSR-TR-2006-156, 2006.

[100]

T. Qin, T.-Y. Liu, J. Xu, and H. Li, "How to make LETOR more useful and reliable," in SIGIR 2008 Workshop on Learning to Rank for Information Retrieval , 2008.

[101]

T. Qin, T.-Y. Liu, X.-D. Zhang, Z. Chen, and W.-Y. Ma, "A study of relevance propagation for web search," in SIGIR 2005 , pp. 408-415, 2005.

[102]

T. Qin, T.-Y. Liu, X.-D. Zhang, M.-F. Tsai, D.-S. Wang, and H. Li, "Querylevel loss functions for information retrieval," Information Processing and Management , vol. 44, pp. 838-855, 2007.

Digital Library

[103]

T. Qin, T.-Y. Liu, X.-D. Zhang, D. Wang, and H. Li, "Learning to rank relational objects and its application to web search," in WWW 2008 , pp. 407-416, 2008.

[104]

T. Qin, T.-Y. Liu, X.-D. Zhang, D.-S. Wang, and H. Li, "Global ranking using continuous conditional random fields," in NIPS 2008 , pp. 1281-1288, 2008.

[105]

F. Radlinski and T. Joachims, "Query chain: Learning to rank from implicit feedback," in KDD 2005 , pp. 239-248, 2005.

[106]

F. Radlinski and T. Joachims, "Active exploration for learning rankings from clickthrough data," in KDD 2007 , 2007.

[107]

F. Radlinski, R. Kleinberg, and T. Joachims, "Learning diverse rankings with multi-armed bandits," in ICML 2008 , pp. 784-791, 2008.

Digital Library

[108]

S. Rajaram and S. Agarwal, "Generalization bounds for k -partite ranking," in NIPS 2005 WorkShop on Learning to Rank , 2005.

[109]

L. Rigutini, T. Papini, M. Maggini, and F. Scarselli, "Learning to rank by a neural-based sorting algorithm," in SIGIR 2008 Workshop on Learning to Rank for Information Retrieval , 2008.

[110]

S. Robertson and H. Zaragoza, "On rank-based effectiveness measures and optimization," Information Retrieval , vol. 10, pp. 321-339, 2007.

Digital Library

[111]

S. E. Robertson, "Overview of the okapi projects," Journal of Documentation , vol. 53, pp. 3-7, 1997.

[112]

J. J. Rochhio, "Relevance feedback in information retrieval," The SMART Retrieval System - Experiments in Automatic Document Processing , pp. 313-323, 1971.

[113]

A. Shakery and C. Zhai, "A probabilistic relevance propagation model for hypertext retrieval," in CIKM 2006 , pp. 550-558, 2006.

[114]

A. Shashua and A. Levin, "Ranking with large margin principles: Two approaches," in NIPS 2002 , pp. 937-944, 2002.

[115]

J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis . Cambridge University Press, 2004.

Digital Library

[116]

A. Singhal, "Modern information retrieval: A brief overview," IEEE Data Engineering Bulletin , vol. 24, pp. 35-43, 2001.

[117]

T. Tao and C. Zhai, "Regularized estimation of mixture models for robust pseudo-relevance feedback," in SIGIR 2006 , pp. 162-169, 2006.

[118]

T. Tao and C. Zhai, "An exploration of proximity measures in information retrieval," in SIGIR 2007 , pp. 295-302, 2007.

[119]

M. Taylor, J. Guiver, S. Robertson, and T. Minka, "SoftRank: Optimising non-smooth rank metrics," in WSDM 2008 , pp. 77-86, 2008.

[120]

M. Taylor, H. Zaragoza, N. Craswell, S. Robertson, and C. J. Burges, "Optimisation methods for ranking functions with multiple parameters," in CIKM 2006 , pp. 585-593, 2006.

[121]

A. Trotman, "Learning to rank," Information Retrieval , vol. 8, pp. 359-381, 2005.

Digital Library

[122]

M.-F. Tsai, T.-Y. Liu, T. Qin, H.-H. Chen, and W.-Y. Ma, "FRank: A ranking method with fidelity loss," in SIGIR 2007 , pp. 383-390, 2007.

[123]

I. Tsochantaridis, T. Hofmann, T. Joachims, and Y. Altun, "Support vector machine learning for interdependent and structured output space," in ICML 2004 , pp. 104-111, 2004.

Digital Library

[124]

N. Usunier, M.-R. Amini, and P. Gallinari, "Generalization error bounds for classifiers trained with interdependent data," in NIPS 2005 , pp. 1369-1376, 2005.

[125]

V. N. Vapnik, The Nature of Statistical Learning Theory . Springer, 1995.

[126]

V. N. Vapnik, Statistical Learning Theory . Wiley-Interscience, 1998.

[127]

A. Veloso, H. M. de Almeida, M. A. Goncalves, and W. Meira, Jr., "Learning to rank at query-time using association rules," in SIGIR 2008 , pp. 267-274, 2008.

[128]

W. Xi, J. Lind, and E. Brill, "Learning effective ranking functions for newsgroup search," in SIGIR 2004 , pp. 394-401, 2004.

[129]

F. Xia, T.-Y. Liu, J. Wang, W. Zhang, and H. Li, "Listwise approach to learning to rank - Theorem and algorithm," in ICML 2008 , pp. 1192-1199, 2008.

Digital Library

[130]

J. Xu, Y. Cao, H. Li, and M. Zhao, "Ranking definitions with supervised learning methods," in WWW 2005 , pp. 811-819, New York, NY, USA: ACM Press, 2005.

[131]

J. Xu and H. Li, "AdaRank: A boosting algorithm for information retrieval," in SIGIR 2007 , pp. 391-398, 2007.

[132]

J. Xu, T.-Y. Liu, M. Lu, H. Li, and W.-Y. Ma, "Directly optimizing IR evaluation measures in learning to rank," in SIGIR 2008 , pp. 107-114, 2008.

[133]

G.-R. Xue, Q. Yang, H.-J. Zeng, Y. Yu, and Z. Chen, "Exploiting the hierarchical structure for link analysis," in SIGIR 2005 , pp. 186-193, 2005.

[134]

J.-Y. Yeh and J.-Y. Lin, and etc, "Learning to rank for information retrieval using genetic programming," in SIGIR 2007 Workshop in Learning to Rank for Information Retrieval , 2007.

[135]

H. Yu, "SVM selective sampling for ranking with application to data retrieval," in KDD 2005 , pp. 354-363, 2005.

[136]

Y. Yue, T. Finley, F. Radlinski, and T. Joachims, "A support vector method for optimizing average precision," in SIGIR 2007 , pp. 271-278, 2007.

[137]

Y. Yue and T. Joachims, "Predicting diverse subsets using structural SVM," in ICML 2008 , pp. 1224-1231, 2008.

Digital Library

[138]

C. Zhai, "Language models," Foundations and Trends in Information Retrieval , 2008.

[139]

C. Zhai, W. W. Cohen, and J. Lafferty, "Beyond independent relevance: Methods and evaluation metrics for subtopic retrieval," in SIGIR 2003 , pp. 10-17, 2003.

[140]

C. Zhai and J. Lafferty, "Model-based feedback in the language modeling approach to information retrieval," in CIKM 2001 , pp. 403-410, 2001.

[141]

C. Zhai and J. Lafferty, "A risk minimization framework for information retrieval," Information Processing and Management , vol. 42, pp. 31-55, 2006.

Digital Library

[142]

Z. Zheng, H. Zha, and G. Sun, "Query-level learning to rank using isotonic regression," in SIGIR 2008 Workshop on Learning to Rank for Information Retrieval , 2008.

[143]

K. Zhou, G.-R. Xue, H. Zha, and Y. Yu, "Learning to rank with ties," in SIGIR 2008 , pp. 275-282, 2008.

[144]

O. Zoeter, M. Taylor, E. Snelson, J. Guiver, N. Craswell, and M. Szummer, "A decision theoretic framework for ranking using implicit feedback," in SIGIR 2008 Workshop on Learning to Rank for Information Retrieval , 2008.

Cited By

Keyhanipour A(2025)Graph-induced rank-aggregation using information fusion operatorsThe Journal of Supercomputing10.1007/s11227-024-06595-881:1Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s11227-024-06595-8
Lehmann CSulimov PStockinger K(2024)Is Your Learned Query Optimizer Behaving As You Expect? A Machine Learning PerspectiveProceedings of the VLDB Endowment10.14778/3654621.365462517:7(1565-1577)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.14778/3654621.3654625
Cecchetti JTonellotto NPerego ROosterhuis HBast HXiong C(2024)Learning to Rank for Non Independent and Identically Distributed DatasetsProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672513(71-79)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672513
Show More Cited By

Learning to Rank for Information Retrieval
1. Information systems
  1. Information retrieval

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Foundations and Trends in Information Retrieval

Foundations and Trends in Information Retrieval Volume 3, Issue 3

March 2009

109 pages

ISSN:1554-0669

EISSN:1554-0677

Issue’s Table of Contents

Publisher

Now Publishers Inc.

Hanover, MA, United States

Publication History

Published: 01 March 2009

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

756
Total Citations
View Citations
5
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 19 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Keyhanipour A(2025)Graph-induced rank-aggregation using information fusion operatorsThe Journal of Supercomputing10.1007/s11227-024-06595-881:1Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s11227-024-06595-8
Lehmann CSulimov PStockinger K(2024)Is Your Learned Query Optimizer Behaving As You Expect? A Machine Learning PerspectiveProceedings of the VLDB Endowment10.14778/3654621.365462517:7(1565-1577)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.14778/3654621.3654625
Cecchetti JTonellotto NPerego ROosterhuis HBast HXiong C(2024)Learning to Rank for Non Independent and Identically Distributed DatasetsProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672513(71-79)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672513
Wang XYu HMeng XCao HZhang HSun HLiu XHu C(2024)MTL-TRANSFER: Leveraging Multi-task Learning and Transferred Knowledge for Improving Fault Localization and Program RepairACM Transactions on Software Engineering and Methodology10.1145/365444133:6(1-31)Online publication date: 27-Jun-2024
https://dl.acm.org/doi/10.1145/3654441
Moon SKerr GSilavong FMoran S(2024)API-Miner: an API-to-API Specification Recommendation EngineProceedings of the 1st IEEE/ACM Workshop on Software Engineering Challenges in Financial Firms10.1145/3643665.3648049(9-16)Online publication date: 16-Apr-2024
https://dl.acm.org/doi/10.1145/3643665.3648049
Gupta SOosterhuis Hde Rijke MSerra ESpezzano F(2024)Practical and Robust Safety Guarantees for Advanced Counterfactual Learning to RankProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679531(737-747)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679531
Fröbe MMackenzie JMitra BNardini FPotthast MHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)ReNeuIR at SIGIR 2024: The Third Workshop on Reaching Efficiency in Neural Information RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657994(3051-3054)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657994
Gorantla SBhansali EDeshpande ALouis AHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Optimizing Learning-to-Rank Models for Ex-Post Fair RelevanceProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657751(1525-1534)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657751
Thomas PSpielman SCraswell NMitra BHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Large Language Models can Accurately Predict Searcher PreferencesProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657707(1930-1940)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657707
Liu YZhang RGuo Jde Rijke MFan YCheng XHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Multi-granular Adversarial Attacks against Black-box Neural Ranking ModelsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657704(1391-1400)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657704
Show More Cited By

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents