Abstract
From the original game console, the Xbox has rapidly evolved into a comprehensive entertainment platform where tens of millions of users could not only play video games but also watch movies and TVs, listen music and enjoy Apps. Therefore, building a cross media ranker to provide relevant and personalized search results for Xbox users has become an interesting and imperative task. In this paper, we present our recent progress on improving Xbox’s cross media ranker by mining massive click log data and generating multi-class relevance labels. Our experimental results have shown that incorporating the click likelihoods into the label generation yields better click-performance and meanwhile maintains comparable NDCG values, as compared to solely using the human labels generated by a small number of human judges.
Chapter PDF
Similar content being viewed by others
References
Voorhees, E.M.: Variations in relevance judgments and the measurement of retrieval effectiveness. Information Processing & Management 36, 697–716 (2000)
Bailey, P., Craswell, N., Soboroff, I., Thomas, P., de Vries, A.P., Yilmaz, E.: Relevance assessment: are judges exchangeable and does it matter. In: Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 667–674. ACM (2008)
Agrawal, R., Halverson, A., Kenthapadi, K., Mishra, N., Tsaparas, P.: Generating labels from clicks. In: Proceedings of the Second ACM International Conference on Web Search and Data Mining, pp. 172–181. ACM (2009)
Xu, J., Chen, C., Xu, G., Li, H., Abib, E.R.T.: Improving quality of training data for learning to rank using click-through data. In: Proceedings of the Third ACM International Conference on Web Search and Data Mining, pp. 171–180. ACM (2010)
von Alan, R.H., March, S.T., Park, J., Ram, S.: Design science in information systems research. MIS Quarterly 28, 75–105 (2004)
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., Hullender, G.: Learning to rank using gradient descent. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 89–96. ACM (2005)
Burges, C.J., Ragno, R., Le, Q.V.: Learning to rank with nonsmooth cost functions. In: NIPS, pp. 193-200 (2006)
Cao, Z., Qin, T., Liu, T.-Y., Tsai, M.-F., Li, H.: Learning to rank: from pairwise approach to listwise approach. In: Proceedings of the 24th International Conference on Machine Learning, pp. 129–136. ACM, New York (2007)
Chapelle, O., Chang, Y.: Yahoo! Learning to Rank Challenge Overview. Journal of Machine Learning Research-Proceedings Track 14, 1–24 (2011)
Craswell, N., Zoeter, O., Taylor, M., Ramsey, B.: An experimental comparison of click position-bias models. In: Proceedings of the 2008 International Conference on Web Search and Data Mining, pp. 87–94. ACM (2008)
Burges, C.J.: From ranknet to lambdarank to lambdamart: An overview. Learning 11, 23–581 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Li, J., Ye, X., Li, D. (2014). Improving Xbox Search Relevance by Click Likelihood Labeling. In: Nah, F.FH. (eds) HCI in Business. HCIB 2014. Lecture Notes in Computer Science, vol 8527. Springer, Cham. https://doi.org/10.1007/978-3-319-07293-7_71
Download citation
DOI: https://doi.org/10.1007/978-3-319-07293-7_71
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07292-0
Online ISBN: 978-3-319-07293-7
eBook Packages: Computer ScienceComputer Science (R0)