Nothing Special   »   [go: up one dir, main page]

skip to main content
article

Aggregated Search

Published: 06 March 2017 Publication History

Abstract

The goal of aggregated search is to provide integrated search acrossmultiple heterogeneous search services in a unified interface-a singlequery box and a common presentation of results. In the web searchdomain, aggregated search systems are responsible for integrating resultsfrom specialized search services, or verticals, alongside the coreweb results. For example, search portals such as Google, Bing, andYahoo! provide access to vertical search engines that focus on differenttypes of media images and video, different types of search taskssearch for local businesses and online products, and even applicationsthat can help users complete certain tasks language translation andmath calculations.Aggregated search systems perform two mains tasks. The first taskvertical selection is to predict which verticals if any to present inresponse to a user's query. The second task vertical presentation is topredict where and how to present each selected vertical alongside thecore web results.The goal of this work is to provide a comprehensive summary of previousresearch in aggregated search. We first describe why aggregatedsearch requires unique solutions. Then, we discuss different sources ofevidence that are likely to be available to an aggregated search system,as well as different techniques for integrating evidence in order to makevertical selection and presentation decisions. Next, we survey differentevaluation methodologies for aggregated search and discuss prioruser studies that have aimed to better understand how users behavewith aggregated search interfaces. Finally, we review different advancedtopics in aggregated search.

References

[1]
Abou-Assaleh, T. and Gao, W. Geographic ranking for a local search engine. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '07, pages 911-911, New York, NY, USA, 2007.
[2]
Agrawal, R., Gollapudi, S., Halverson, A., and Ieong, S. Diversifying search results. In Proceedings of the 2nd ACM International Conference on Web Search and Data Mining, WSDM '09, pages 5-14, New York, NY, USA, 2009.
[3]
Al-Maskari, A. and Sanderson, M. The effect of user characteristics on search effectiveness in information retrieval. Information Processing and Management, 47(5):719-729, September 2011.
[4]
Arguello, J. Improving aggregated search coherence. In Proceedings of the 37th European Conference on Advances in Information Retrieval, volume 9022 of ECIR '15, pages 25-36, Springer-Verlag, Berlin, Heidelberg, 2015.
[5]
Arguello, J. and Capra, R. The effects of aggregated search coherence on search behavior. ACM Transactions on Information Systems, 11(1), 2016.
[6]
Arguello, J. and Capra, R. The effect of aggregated search coherence on search behavior. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management, CIKM '12, pages 1293-1302, New York, NY, USA, 2012.
[7]
Arguello, J. and Capra, R. The effects of vertical rank and border on aggregated search coherence and search behavior. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, CIKM '14, pages 539-548, New York, NY, USA, 2014.
[8]
Arguello, J., Callan, J., and Diaz, F. Classification-based resource selection. In Proceedings of the 18th ACM Conference on Information and Knowledge Management, CIKM '09, pages 1277-1286, New York, NY, USA, 2009a.
[9]
Arguello, J., Diaz, F., Callan, J., and Crespo, J.-F. Sources of evidence for vertical selection. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '09, pages 315-322, New York, NY, USA, 2009b.
[10]
Arguello, J., Diaz, F., and Paiement, J.-F. Vertical selection in the presence of unlabeled verticals. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '10, pages 691-698, New York, NY, USA, 2010.
[11]
Arguello, J., Diaz, F., and Callan, J. Learning to aggregate vertical results into web search results. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management, CIKM '11, pages 201-210, New York, NY, USA, 2011a.
[12]
Arguello, J., Diaz, F., Callan, J., and Carterette, B. A methodology for evaluating aggregated search results. In Proceedings of the 33rd European Conference on Advances in Information Retrieval, ECIR '11, pages 141- 152, Springer-Verlag, Berlin, Heidelberg, 2011b.
[13]
Arguello, J., Wu, W.-C., Kelly, D., and Edwards, A. Task complexity, vertical display, and user interaction in aggregated search. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '12, pages 435-444, New York, NY, USA, 2012.
[14]
Arguello, J., Capra, R., and Wu, W.-C. Factors affecting aggregated search coherence and search behavior. In Proceedings of the 22nd ACM International Conference on Information and Knowledge Management, CIKM '13, pages 1989-1998, New York, NY, USA, 2013.
[15]
Aula, A., Khan, R. M., and Guan, Z. How does search behavior change as search becomes more difficult? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '10, pages 35-44, New York, NY, USA, 2010.
[16]
Bailey, P., Craswell, N., White, R. W., Chen, L., Satyanarayana, A., and Tahaghoghi, S. M. Evaluating search systems using result page context. In Proceedings of the 3rd Symposium on Information Interaction in Context, IIiX '10, pages 105-114, New York, NY, USA, 2010a.
[17]
Bailey, P., Craswell, N., White, R. W., Chen, L., Satyanarayana, A., and Tahaghoghi, S. Evaluating whole-page relevance. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '10, pages 767-768, New York, NY, USA, 2010b.
[18]
Bennett, P. N., Svore, K., and Dumais, S. T. Classification-enhanced ranking. In Proceedings of the 19th International Conference on World Wide Web, WWW '10, pages 111-120, New York, NY, USA, 2010.
[19]
Beverly, R. and Afergan, M. Machine learning for efficient neighbor selection in unstructured p2p networks. In Proceedings of the 2Nd USENIX Workshop on Tackling Computer Systems Problems with Machine Learning Techniques, SYSML '07, pages 1:1-1:6, USENIX Association, Berkeley, CA, USA, 2007.
[20]
Bharat, K., Kamba, T., and Albers, M. Personalized, interactive news on the web. Multimedia Systems, 6(5):349-358, 1998.
[21]
Bian, J., Chang, Y., Fu, Y., and Chen, W.-Y. Learning to blend vitality rankings from heterogeneous social networks. Neurocomputing, 97:390-397, 2012. ISSN 0925-2312.
[22]
Bilal, D. Children's use of the yahooligans! web search engine. iii. cognitive and physical behaviors on fully self-generated search tasks. Journal of the American Society for Information Science and Technology, 53(13):1170-1183, 2002.
[23]
Bota, H., Zhou, K., Jose, J. M., and Lalmas, M. Composite retrieval of heterogeneous web search. In Proceedings of the 23rd International Conference on World Wide Web, WWW '14, pages 119-130, New York, NY, USA, 2014.
[24]
Bota, H., Zhou, K., and Jose, J. J. Advances in Information Retrieval: 37th European Conference on IR Research, ECIR 2015, Vienna, Austria, March 29 - April 2, 2015. Proceedings, chapter Exploring Composite Retrieval from the Users' Perspective, pages 13-24. Springer International Publishing, 2015.
[25]
Bota, H., Zhou, K., and Jose, J. M. Playing your cards right: The effect of entity cards on search behaviour and workload. In Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval, CHIIR '16, pages 131-140, New York, NY, USA, 2016.
[26]
Brennan, K., Kelly, D., and Arguello, J. The effect of cognitive abilities on information search for tasks of varying levels of complexity. In Proceedings of the 5th Information Interaction in Context Symposium, IIiX '14, pages 165-174, New York, NY, USA, 2014.
[27]
Broder, A. A taxonomy of web search. SIGIR Forum, 36(2), September 2002.
[28]
Broder, A., Fontoura, M., Josifovski, V., and Riedel, L. A semantic approach to contextual advertising. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '07, pages 559-566, New York, NY, USA, 2007.
[29]
Bron, M., van Gorp, J., Nack, F., Baltussen, L. B., and de Rijke, M. Aggregated search interface preferences in multi-session search tasks. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '13, pages 123-132, New York, NY, USA, 2013.
[30]
Callan, J. and Connell, M. Query-based sampling of text databases. ACM Transactions on Information Systems, 19:97-130, 2001.
[31]
Capra, R., Arguello, J., and Scholer, F. Augmenting web search surrogates with images. In Proceedings of the 22nd ACM International Conference on Information and Knowledge Management, CIKM '13, pages 399-408, New York, NY, USA, 2013.
[32]
Carterette, B., Kanoulas, E., Hall, M., and Clough, P. Overview of the trec 2014 session track. In Proceedings of the 24th Text Retrieval Conference, TREC '14, NIST. 2014.
[33]
Caverlee, J., Liu, L., and Bae, J. Distributed query sampling: A quality-conscious approach. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '06, pages 340-347, New York, NY, USA, 2006.
[34]
Chapelle, O., Joachims, T., Radlinski, F., and Yue, Y. Large-scale validation and analysis of interleaved search evaluation. ACM Transactions of Information Systems, 30(1):6:1-6:41, 2012.
[35]
Chen, D., Chen, W., Wang, H., Chen, Z., and Yang, Q. Beyond ten blue links: Enabling user click modeling in federated web search. In Proceedings of the 5th ACM International Conference on Web Search and Data Mining, WSDM '12, pages 463-472, New York, NY, USA, 2012.
[36]
Chen, K., Bai, J., and Zheng, Z. Ranking function adaptation with boosting trees. ACM Transactions of Information Systems, 29(4):18:1-18:31, December 2011.
[37]
Chen, Y., Liu, Y., Zhou, K., Wang, M., Zhang, M., and Ma, S. Does vertical bring more satisfaction?: Predicting search satisfaction in a heterogeneous environment. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, CIKM '15, pages 1581-1590, New York, NY, USA, 2015.
[38]
ChildTrends. Home computer access and internet use. http://www.childtrends.org/?indicators=home-computer-access, 2015. Accessed: 2016-05-31.
[39]
Chuklin, A., Schuth, A., Hofmann, K., Serdyukov, P., and de Rijke, M. Evaluating aggregated search using interleaving. In Proceedings of the 22nd ACM International Conference on Conference on Information and Knowledge Management, CIKM '13, pages 669-678, New York, NY, USA, 2013a.
[40]
Chuklin, A., Serdyukov, P., and de Rijke, M. Click model-based information retrieval metrics. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '13, pages 493-502, New York, NY, USA, 2013b.
[41]
Cleverdon, C. W. The aslib cranfield research project on the comparative efficiency of indexing systems. Aslib Proceedings, 12(12):421-431, 1960.
[42]
Comscore. Digital Future in Focus U.S. 2015. Technical report, 2015.
[43]
Cronen-Townsend, S., Zhou, Y., and Croft, W. B. Predicting query performance. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '02, pages 299-306, New York, NY, USA, 2002.
[44]
Dean-Hall, A., Clarke, C. L. A., Kamps, J., Thomas, P., and Voorhees, E. Overview of the trec 2012 contextual suggestion track. In Proceedings of the 21st Text Retrieval Conference, TREC '12, NIST. 2012.
[45]
Dean-Hall, A., Clarke, C. L. A., Simone, N., Kamps, J., Thomas, P., and Voorhees, E. Overview of the trec 2013 contextual suggestion track. In Proceedings of the 22nd Text Retrieval Conference, TREC '13, NIST. 2013.
[46]
Dean-Hall, A., Clarke, C. L. A., Kamps, J., Thomas, P., and Voorhees, E. Overview of the trec 2014 contextual suggestion track. In Proceedings of the 23rd Text Retrieval Conference, TREC '14, NIST. 2014.
[47]
Dean-Hall, A., Clarke, C. L. A., Kamps, J., Kiseleva, J., and Voorhees, E. Overview of the trec 2014 contextual suggestion track. In Proceedings of the 24th Text Retrieval Conference, TREC '15, NIST. 2015.
[48]
Demeester, T., Trieschnigg, D., Nguyen, D., and Hiemstra, D. Overview of the trec 2013 federated web search track. In Proceedings of the 23rd Text Retrieval Conference, TREC '13, NIST. 2013.
[49]
Demeester, T., Trieschnigg, D., Nguyen, D., Hiemstra, D., and Zhou, K. Overview of the trec 2014 federated web search track. In Proceedings of the 23rd Text Retrieval Conference, TREC '14, NIST. 2014.
[50]
Diaz, F. Performance prediction using spatial autocorrelation. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '07, pages 583-590, New York, NY, USA, 2007.
[51]
Diaz, F. Integration of news content into web results. In Proceedings of the 2nd ACM International Conference on Web Search and Data Mining, WSDM '09, pages 182-191, New York, NY, USA, 2009.
[52]
Diaz, F. and Arguello, J. Adaptation of offline vertical selection predictions in the presence of user feedback. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '09, pages 323-330, New York, NY, USA, 2009.
[53]
Diaz, F., White, R., Buscher, G., and Liebling, D. Robust models of mouse movement on dynamic web search results pages. In Proceedings of the 22nd ACM International Conference on Conference on Information and Knowledge Management, CIKM '13, pages 1451-1460, New York, NY, USA, 2013.
[54]
Duarte-Torres, S. and Weber, I. What and how children search on the web. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management, CIKM '11, pages 393-402, New York, NY, USA, 2011.
[55]
Duarte-Torres, S., Hiemstra, D., and Serdyukov, P. Query log analysis in the context of information retrieval for children. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '10, pages 847-848, New York, NY, USA, 2010.
[56]
Duarte-Torres, S., Hiemstra, D., and Huibers, T. Vertical selection in the information domain of children. In Proceedings of the 13th ACM/IEEE-CS Joint Conference on Digital Libraries, JCDL '13, pages 57-66, New York, NY, USA, 2013.
[57]
Duarte-Torres, S., Weber, I., and Hiemstra, D. Analysis of search and browsing behavior of young users on the web. ACM Transactions on the Web, (2):7:1-7:54, 2014.
[58]
Dumais, S., Cutrell, E., and Chen, H. Optimizing search by showing results in context. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '01, pages 277-284, New York, NY, USA, 2001.
[59]
Dworman, G. and Rosenbaum, S. Helping users to use help: Improving interaction with help systems. In CHI '04 Extended Abstracts on Human Factors in Computing Systems, CHI EA '04, pages 1717-1718, New York, NY, USA, 2004.
[60]
Ekstrom, R., French, J., Harman, H., and Dermen, D. Kit of Factor-Referenced Cognitive Tests. Educational Testing Service, Princeton, NJ, USA, 1979.
[61]
Feng, Y. and Lapata, M. Topic models for image annotation and text illustration. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, HLT '10, pages 831-839, Association for Computational Linguistics, Stroudsburg, PA, USA, 2010.
[62]
Fleiss, J. Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5):378-382, 1971.
[63]
Friedman, J. H. Stochastic gradient boosting. Computational Statistics and Data Analysis, 38(4):367-378, February 2002.
[64]
Gravano, L., Chang, C.-C. K., García-Molina, H., and Paepcke, A. Starts: Stanford proposal for internet meta-searching. In Proceedings of the 1997 ACM SIGMOD International Conference on Management of Data, SIGMOD '97, pages 207-218, New York, NY, USA, 1997.
[65]
Guy, I. Searching by talking: Analysis of voice queries on mobile web search. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '16, pages 35-44, New York, NY, USA, 2016.
[66]
Hassan, A., White, R. W., Dumais, S. T., and Wang, Y.-M. Struggling or exploring?: Disambiguating long search sessions. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining, WSDM '14, pages 53-62, New York, NY, USA, 2014.
[67]
2010. Hauff, C. Predicting the Effectiveness of Queries and Retrieval Systems. dissertation, Univeristy of Twente, 2010.
[68]
Hofmann, K., Whiteson, S., and de Rijke, M. A probabilistic method for inferring preferences from clicks. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management, CIKM '11, pages 249-258, New York, NY, USA, 2011.
[69]
Hofmann, K., Behr, F., and Radlinski, F. On caption bias in interleaving experiments. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management, CIKM '12, pages 115-124, New York, NY, USA, 2012.
[70]
Hofmann, K., Whiteson, S., and Rijke, M. D. Fidelity, soundness, and efficiency of interleaved comparison methods. ACM Transactions of Information Systems, 31(4):1-43, November 2013.
[71]
Hofmann, K., Li, L. L., and Radlinski, F. Online evaluation for information retrieval. Foundations and Trends in Information Retrieval, 10:1-117, June 2016.
[72]
Hong, D., Si, L., Bracke, P., Witt, M., and Juchcinski, T. A joint probabilistic classification model for resource selection. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '10, pages 98-105, New York, NY, USA, 2010.
[73]
Hong, L., Dom, B., Gurumurthy, S., and Tsioutsiouliklis, K. A time-dependent topic model for multiple text streams. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '11, pages 832-840, New York, NY, USA, 2011.
[74]
Huang, J., White, R. W., and Dumais, S. No clicks, no problem: Using cursor movements to understand and improve search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '11, pages 1225-1234, New York, NY, USA, 2011.
[75]
Huang, J., White, R., and Buscher, G. User see, user point: Gaze and cursor alignment in web search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '12, pages 1341-1350, New York, NY, USA, 2012a.
[76]
Huang, J., White, R. W., Buscher, G., and Wang, K. Improving searcher models using mouse cursor activity. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '12, pages 195-204, New York, NY, USA, 2012b.
[77]
Jansen, B. J., Booth, D. L., and Spink, A. Determining the informational, navigational, and transactional intent of web queries. Information Processing and Management, 44(3):1251-1266, May 2008.
[78]
Jansen, B. J., Booth, D., and Smith, B. Using the taxonomy of cognitive learning to model online searching. Information Processing and Management, 45(6):643-663, November 2009.
[79]
Järvelin, K. and Kekäläinen, J. Cumulated gain-based evaluation of ir techniques. ACM Transactions of Informaton Systems, 20(4):422-446, 2002.
[80]
Jeon, J., Croft, W. B., Lee, J. H., and Park, S. A framework to predict the quality of answers with non-textual features. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '06, pages 228-235, New York, NY, USA, 2006.
[81]
Jiang, J., Hassan Awadallah, A., Jones, R., Ozertem, U., Zitouni, I., Gurunath Kulkarni, R., and Khan, O. Z. Automatic online evaluation of intelligent assistants. In Proceedings of the 24th International Conference on World Wide Web, WWW '15, New York, NY, USA, 2015.
[82]
Jie, L., Lamkhede, S., Sapra, R., Hsu, E., Song, H., and Chang, Y. A unified search federation system based on online user feedback. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '13, pages 1195-1203, New York, NY, USA, 2013.
[83]
Joachims, T. Optimizing search engines using clickthrough data. In Proceedings of the 8thh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '02, pages 133-142, New York, NY, USA, 2002.
[84]
Khelghati, M., Hiemstra, D., and van Keulen, M. Size estimation of noncooperative data collections. In Proceedings of the 14th International Conference on Information Integration and Web-based Applications and Services, IIWAS '12, pages 239-246, New York, NY, USA, 2012.
[85]
Kim, J. and Croft, W. B. Ranking using multiple document types in desktop search. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '10, pages 50-57, New York, NY, USA, 2010.
[86]
Kiseleva, J., Williams, K., Hassan Awadallah, A., Crook, A. C., Zitouni, I., and Anastasakos, T. Predicting user satisfaction with intelligent assistants. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '16, pages 45-54, New York, NY, USA, 2016a.
[87]
Kiseleva, J., Williams, K., Jiang, J., Hassan Awadallah, A., Crook, A. C., Zitouni, I., and Anastasakos, T. Understanding user satisfaction with intelligent assistants. In Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval, CHIIR '16, pages 121-130, New York, NY, USA, 2016b.
[88]
Koffka, K. Principles of Gestalt psychology. Harcourt, New York, 1935.
[89]
König, A. C., Gamon, M., and Wu, Q. Click-through prediction for news queries. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '09, pages 347-354, New York, NY, USA, 2009.
[90]
Koolen, M., Kazai, G., and Craswell, N. Wikipedia pages as entry points for book search. In Proceedings of the 2nd ACM International Conference on Web Search and Data Mining, WSDM '09, pages 44-53, New York, NY, USA, 2009.
[91]
Krakovsky, M. All the news that's fit for you. Communications of the ACM, 54(6):20-21, 2011.
[92]
Kulkarni, A. and Callan, J. Selective search: Efficient and effective search of large textual collections. ACM Transactions on Information Systems, 33 (4):1-33, 2015.
[93]
Kumar, R. and Vassilvitskii, S. Generalized distances between rankings. In Proceedings of the 19th International Conference on World Wide Web, WWW '10, pages 571-580, New York, NY, USA, 2010.
[94]
Lagun, D. and Agichtein, E. Inferring searcher attention by jointly modeling user interactions and content salience. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '15, pages 483-492, New York, NY, USA, 2015.
[95]
Lagun, D., Ageev, M., Guo, Q., and Agichtein, E. Discovering common motifs in cursor movement data for improving web search. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining, WSDM '14, pages 183-192, New York, NY, USA, 2014a.
[96]
Lagun, D., Hsieh, C.-H., Webster, D., and Navalpakkam, V. Towards better measurement of attention and satisfaction in mobile search. In Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '14, pages 113-122, New York, NY, USA, 2014b.
[97]
Lavrenko, V. and Croft, W. B. Relevance based language models. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '01, pages 120-127, New York, NY, USA, 2001.
[98]
Lawrence, S., Bollacker, K., and Giles, C. L. Indexing and retrieval of scientific literature. In Proceedings of the 8th International Conference on Information and Knowledge Management, CIKM '99, pages 139-146, New York, NY, USA, 1999.
[99]
Lee, C.-J., Croft, W. B., and Kim, J. Y. Evaluating search in personal social media collections. In Proceedings of the 5th ACM International Conference on Web Search and Data Mining, WSDM '12, pages 683-692, New York, NY, USA, 2012.
[100]
Li, J., Huffman, S., and Tokuda, A. Good abandonment in mobile and pc internet search. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '09, pages 43-50, New York, NY, USA, 2009.
[101]
Li, X., Wang, Y.-Y., and Acero, A. Learning query intent from regularized click graphs. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '08, pages 339-346, New York, NY, USA, 2008.
[102]
Liu, J., Cole, M. J., Liu, C., Bierig, R., Gwizdka, J., Belkin, N. J., Zhang, J., and Zhang, X. Search behaviors in different task types. In Proceedings of the 10th Annual Joint Conference on Digital Libraries, JCDL '10, pages 69-78, New York, NY, USA, 2010a.
[103]
Liu, J., Liu, C., Gwizdka, J., and Belkin, N. J. Can search systems detect users' task difficulty?: Some behavioral signals. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '10, pages 845-846, New York, NY, USA, 2010b.
[104]
Liu, J., Liu, C., Cole, M., Belkin, N. J., and Zhang, X. Exploring and predicting search task difficulty. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management, CIKM '12, pages 1313-1322, New York, NY, USA, 2012.
[105]
Liu, K.-L., Santoso, A., Yu, C., and Meng, W. Discovering the representative of a search engine. In Proceedings of the Tenth International Conference on Information and Knowledge Management, CIKM '01, pages 577-579, New York, NY, USA, 2001.
[106]
Liu, K.-L., Yu, C., and Meng, W. Discovering the representative of a search engine. In Proceedings of the Eleventh International Conference on Information and Knowledge Management, CIKM '02, pages 652-654, New York, NY, USA, 2002.
[107]
Liu, Y., Liu, Z., Zhou, K., Wang, M., Luan, H., Wang, C., Zhang, M., and Ma, S. Predicting search user examination with visual saliency. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '16, pages 619-628, New York, NY, USA, 2016.
[108]
Liu, Z., Liu, Y., Zhou, K., Zhang, M., and Ma, S. Influence of vertical result in web search examination. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '15, pages 193-202, New York, NY, USA, 2015.
[109]
Long, B. and Chang, Y. Relevance Ranking for Vertical Search Engines. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1st edition, 2014.
[110]
2007. Lu, J. Full-text Federated Search in Peer-to-peer Networks. PhD thesis, Pittsburgh, PA, USA, 2007.
[111]
Luo, C., Liu, Y., Zhang, M., and Ma, S. Query Ambiguity Identification Based on User Behavior Information, pages 36-47. AIRS 2014. Springer International Publishing, 2014.
[112]
Lv, Y., Moon, T., Kolari, P., Zheng, Z., Wang, X., and Chang, Y. Learning to model relatedness for news recommendation. In Proceedings of the 20th International Conference on World Wide Web, WWW '11, pages 57-66, New York, NY, USA, 2011.
[113]
Markov, I., Kharitonov, E., Nikulin, V., Serdyukov, P., de Rijke, M., and Crestani, F. Vertical-aware click model-based effectiveness metrics. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, CIKM '14, pages 1867-1870, New York, NY, USA, 2014.
[114]
McCreadie, R. and Macdonald, C. Relevance in microblogs: Enhancing tweet retrieval using hyperlinked documents. In Proceedings of the 10th Conference on Open Research Areas in Information Retrieval, OAIR '13, pages 189-196, Paris, France, France, 2013.
[115]
Metzler, D., Dumais, S., and Meek, C. Similarity measures for short segments of text. In Proceedings of the 29th European Conference on IR Research, ECIR'07, pages 16-27, Springer-Verlag, Berlin, Heidelberg, 2007.
[116]
Moffat, A. and Zobel, J. Rank-biased precision for measurement of retrieval effectiveness. ACM Transactions of Information Systems, 27(1):2:1-2:27, 2008.
[117]
Nanba, H., Sakai, T., Kando, N., ana Koji Eguchi, A. K., Hatano, K., Shimizu, T., Hirate, Y., and Fujii, A. Nexti at ntcir-12 imine-2 task. In Proceedings of the 12th NTCIR Conference on Evaluation of Information Access Technologies, NTCIR '16, National Institute of Informatics. 2016.
[118]
Navalpakkam, V., Jentzsch, L., Sayres, R., Ravi, S., Ahmed, A., and Smola, A. Measurement and modeling of eye-mouse behavior in the presence of nonlinear page layouts. In Proceedings of the 22nd International Conference on World Wide Web, WWW'13, pages 953-964, New York, NY, USA, 2013.
[119]
Nygren, E. Between the clicks: Skilled users scanning of pages. In Proceedings of Designing for the Web: Empirical Studies, 1996.
[120]
Palmer, S. E. Common region: A new principle of perceptual grouping. Cognitive Psychology, 24(3):436-447, 1992.
[121]
Ponnuswami, A. K., Pattabiraman, K., Brand, D., and Kanungo, T. Model characterization curves for federated search using click-logs: Predicting user engagement metrics for the span of feasible operating points. In Proceedings of the 20th International Conference on World Wide Web, WWW'11, pages 67-76, New York, NY, USA, 2011a.
[122]
Ponnuswami, A. K., Pattabiraman, K., Wu, Q., Gilad-Bachrach, R., and Kanungo, T. On composition of a federated web search result page: Using online users to provide pairwise preference for heterogeneous verticals. In Proceedings of the 4th ACM International Conference on Web Search and Data Mining, WSDM '11, pages 715-724, New York, NY, USA, 2011b.
[123]
Radlinski, F. and Joachims, T. Query chains: Learning to rank from implicit feedback. In Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, KDD '05, pages 239-248, New York, NY, USA, 2005.
[124]
Radlinski, F., Kurup, M., and Joachims, T. How does clickthrough data reflect retrieval quality? In Proceedings of the 17th ACM Conference on Information and Knowledge Management, CIKM '08, pages 43-52, New York, NY, USA, 2008.
[125]
Sahami, M. and Heilman, T. D. A web-based kernel function for measuring the similarity of short text snippets. In Proceedings of the 15th International Conference on World Wide Web, WWW '06, pages 377-386, New York, NY, USA, 2006.
[126]
Sanderson, M. Ambiguous queries: Test collections need more sense. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '08, pages 499- 506, New York, NY, USA, 2008.
[127]
Santos, R. L., Macdonald, C., and Ounis, I. Learning to rank query suggestions for adhoc and diversity search. Information Retrieval, 16(4):429-451, 2013.
[128]
Santos, R. L. T., Macdonald, C., and Ounis, I. Aggregated search result diversification. In Proceedings of the 3rd International Conference on Advances in Information Retrieval Theory, ICTIR '11, pages 250-261, Springer-Verlag, Berlin, Heidelberg, 2011.
[129]
Schulze, M. A new monotonic, clone-independent, reversal symmetric, and condorcet-consistent single-winner election method. Social Choice and Welfare, 36(2):267-303, 2011.
[130]
Seo, J., Croft, W. B., Kim, K. H., and Lee, J. H. Smoothing click counts for aggregated vertical search. In Proceedings of the 33rd European Conference on Advances in Information Retrieval, ECIR'11, pages 387-398, Springer-Verlag, Berlin, Heidelberg, 2011.
[131]
Shen, D., Pan, R., Sun, J.-T., Pan, J. J., Wu, K., Yin, J., and Yang, Q. Query enrichment for web-query classification. ACM Transactions of Information Systems, 24(3):320-352, 2006.
[132]
Shokouhi, M. Central-rank-based collection selection in uncooperative distributed information retrieval. In Proceedings of the 29th European Conference on IR Research, ECIR'07, pages 160-172, Springer-Verlag, Berlin, Heidelberg, 2007.
[133]
Shokouhi, M. Learning to personalize query auto-completion. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '13, pages 103-112, New York, NY, USA, 2013.
[134]
Shokouhi, M. and Si, L. Federated search. Foundations and Trends in Information Retrieval, 5(1):1-102, January 2011.
[135]
Shokouhi, M., Scholer, F., and Zobel, J. Sample sizes for query probing in uncooperative distributed information retrieval. In Proceedings of the 8th Asia-Pacific Web Conference on Frontiers of WWW Research and Development, APWeb'06, pages 63-75, Springer-Verlag, Berlin, Heidelberg, 2006a.
[136]
Shokouhi, M., Zobel, J., Scholer, F., and Tahaghoghi, S. M. M. Capturing collection size for distributed non-cooperative retrieval. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '06, pages 316-323, New York, NY, USA, 2006b.
[137]
Si, L. and Callan, J. Relevant document distribution estimation method for resource selection. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval, SIGIR '03, pages 298-305, New York, NY, USA, 2003a.
[138]
Si, L. and Callan, J. A semisupervised learning method to merge search engine results. ACM Transactions on Information Systems, 21(4):457-491, 2003b.
[139]
Si, L., Jin, R., Callan, J., and Ogilvie, P. A language modeling framework for resource selection and results merging. In Proceedings of the Eleventh International Conference on Information and Knowledge Management, CIKM '02, pages 391-397, New York, NY, USA, 2002.
[140]
Snow, R., O'Connor, B., Jurafsky, D., and Ng, A. Y. Cheap and fast--but is it good?: Evaluating non-expert annotations for natural language tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP '08, pages 254-263, Association for Computational Linguistics, Stroudsburg, PA, USA, 2008.
[141]
Sohn, T., Li, K. A., Griswold, W. G., and Hollan, J. D. A diary study of mobile information needs. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '08, pages 433-442, New York, NY, USA, 2008.
[142]
Sushmita, S., Joho, H., and Lalmas, M. A task-based evaluation of an aggregated search interface. In Proceedings of the 16th International Symposium on String Processing and Information Retrieval, SPIRE '09, pages 322-333, Springer-Verlag, Berlin, Heidelberg, 2009.
[143]
Sushmita, S., Joho, H., Lalmas, M., and Jose, J. M. Understanding domain relevance in web search. In WWW Workshop on Web Search Result Summarization and Presentation, 2010a.
[144]
Sushmita, S., Joho, H., Lalmas, M., and Villa, R. Factors affecting click-through behavior in aggregated search interfaces. In Proceedings of the 19th ACM International Conference on Information and Knowledge Management, CIKM '10, pages 519-528, New York, NY, USA, 2010b.
[145]
Sushmita, S., Piwowarski, B., and Lalmas, M. Dynamics of genre and domain intents. In Proceedings of the 6th Asia Information Retrieval Societies Conference, AAIRS '10, pages 399-409, Springer-Verlag, Berlin, Heidelberg, 2010c.
[146]
Sutton, R. S. and Barto, A. G. Introduction to Reinforcement Learning. MIT Press, Cambridge, MA, USA, 1st edition, 1998.
[147]
Thomas, P. and Shokouhi, M. Sushi: Scoring scaled samples for server selection. In Proceedings of the 32Nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '09, pages 419-426, New York, NY, USA, 2009.
[148]
Treeratpituk, P. and Callan, J. Automatically labeling hierarchical clusters. In Proceedings of the 2006 International Conference on Digital Government Research, DG.O '06, pages 167-176, Digital Government Society of North America. 2006.
[149]
Trippas, J. R. Spoken conversational search: Information retrieval over a speech-only communication channel. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '15, pages 1067-1067, New York, NY, USA, 2015.
[150]
Tsur, G., Pinter, Y., Szpektor, I., and Carmel, D. Identifying web queries with question intent. In Proceedings of the 25th International Conference on World Wide Web, WWW '16, pages 783-793, International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 2016.
[151]
Turpin, L., Kelly, D., and Arguello, J. To blend or not to blend? perceptual speed, visual memory and aggregated search. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '16, New York, NY, USA, 2016.
[152]
van Rijsbergen, C. J. Information Retrieval. Butterworth-Heinemann, Newton, MA, USA, 2nd edition, 1979.
[153]
Wang, C., Liu, Y., Zhang, M., Ma, S., Zheng, M., Qian, J., and Zhang, K. Incorporating vertical results into search click models. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '13, pages 503-512, New York, NY, USA, 2013.
[154]
Wang, Y., Yin, D., Jie, L., Wang, P., Yamada, M., Chang, Y., and Mei, Q. Beyond ranking: Optimizing whole-page presentation. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining, WSDM '16, pages 103-112, New York, NY, USA, 2016.
[155]
Wen, J.-R., Nie, J.-Y., and Zhang, H.-J. Clustering user queries of a search engine. In Proceedings of the 10th International Conference on World Wide Web, WWW '01, pages 162-168, New York, NY, USA, 2001.
[156]
Wu, W.-C., Kelly, D., Edwards, A., and Arguello, J. Grannies, tanning beds, tattoos and nascar: Evaluation of search tasks with varying levels of cognitive complexity. In Proceedings of the 4th Information Interaction in Context Symposium, IIIX '12, pages 254-257, New York, NY, USA, 2012.
[157]
Xu, J. and Li, H. Adarank: A boosting algorithm for information retrieval. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '07, pages 391-398, New York, NY, USA, 2007.
[158]
Xue, X.-B., Zhou, Z.-H., and Zhang, Z. M. Improving web search using image snippets. ACM Transactions of Internet Technology, 8(4):21:1-21:28, 2008.
[159]
Yamamoto, T., Liu, Y., Zhang, M., Dou, Z., Zhou, K., Markov, I., Kato, M. P., Ohshima, H., and Fujita, S. Overview of the ntcir-12 imine-2 task. In Proceedings of the 12th NTCIR Conference on Evaluation of Information Access Technologies, NTCIR '16, National Institute of Informatics. 2016.
[160]
Yue, Y., Patel, R., and Roehrig, H. Beyond position bias: Examining result attractiveness as a source of presentation bias in clickthrough data. In Proceedings of the 19th International Conference on World Wide Web, WWW '10, pages 1011-1018, New York, NY, USA, 2010.
[161]
Zhou, K., Cummins, R., Halvey, M., Lalmas, M., and Jose, J. M. Assessing and predicting vertical intent for web queries. In Proceedings of the 34th European Conference on Advances in Information Retrieval, ECIR'12, pages 499-502, Springer-Verlag, Berlin, Heidelberg, 2012a.
[162]
Zhou, K., Cummins, R., Lalmas, M., and Jose, J. M. Evaluating reward and risk for vertical selection. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management, CIKM '12, pages 2631-2634, New York, NY, USA, 2012b.
[163]
Zhou, K., Cummins, R., Lalmas, M., and Jose, J. M. Evaluating aggregated search pages. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '12, pages 115-124, New York, NY, USA, 2012c.
[164]
Zhou, K., Cummins, R., Lalmas, M., and Jose, J. M. Which vertical search engines are relevant? In Proceedings of the 22nd International Conference on World Wide Web, WWW '13, pages 1557-1568, New York, NY, USA, 2013a.
[165]
Zhou, K., Lalmas, M., Sakai, T., Cummins, R., and Jose, J. M. On the reliability and intuitiveness of aggregated search metrics. In Proceedings of the 22nd ACM International Conference on Conference on Information and Knowledge Management, CIKM '13, pages 689-698, New York, NY, USA, 2013b.
[166]
Zhou, K., Demeester, T., Nguyen, D., Hiemstra, D., and Trieschnigg, D. Aligning vertical collection relevance with user intent. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, CIKM '14, pages 1915-1918, New York, NY, USA, 2014.
[167]
Zhuang, J., Mei, T., Hoi, S. C., Xu, Y.-Q., and Li, S. When recommendation meets mobile: Contextual and personalized recommendation on the go. In Proceedings of the 13th International Conference on Ubiquitous Computing, UbiComp '11, pages 153-162, New York, NY, USA, 2011.

Cited By

View all
  1. Aggregated Search

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image Foundations and Trends in Information Retrieval
    Foundations and Trends in Information Retrieval  Volume 10, Issue 5
    6 3 2017
    141 pages
    ISSN:1554-0669
    EISSN:1554-0677
    Issue’s Table of Contents

    Publisher

    Now Publishers Inc.

    Hanover, MA, United States

    Publication History

    Published: 06 March 2017

    Qualifiers

    • Article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)0
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 16 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Visualization-Enhanced Aggregated Search InterfacesProceedings of the 2024 Conference on Human Information Interaction and Retrieval10.1145/3627508.3638336(461-464)Online publication date: 10-Mar-2024
    • (2023)Multi-agent-based hybrid peer-to-peer system for distributed information retrievalJournal of Information Science10.1177/0165551521101039249:2(529-543)Online publication date: 22-Mar-2023
    • (2023)Federated search techniques: an overview of the trends and state of the artKnowledge and Information Systems10.1007/s10115-023-01922-665:12(5065-5095)Online publication date: 10-Jul-2023
    • (2023)The influence of knowledge type and source reputation on preferences for website or video search resultsJournal of the Association for Information Science and Technology10.1002/asi.2477175:5(521-537)Online publication date: 17-May-2023
    • (2021)Context-aware Target Apps Selection and Recommendation for Enhancing Personal Mobile AssistantsACM Transactions on Information Systems10.1145/344767839:3(1-30)Online publication date: 5-May-2021
    • (2021)MIRRE approach: nonlinear and multimodal exploration of MIR aggregated search resultsMultimedia Tools and Applications10.1007/s11042-021-10603-x80:13(20217-20253)Online publication date: 1-May-2021
    • (2021)How do multilingual users search? An investigation of query and result list language choicesJournal of the Association for Information Science and Technology10.1002/asi.2444372:6(759-776)Online publication date: 14-May-2021
    • (2019)Vertical Search BlendingProceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3331184.3331345(1237-1240)Online publication date: 18-Jul-2019
    • (2019)The Effects of Working Memory, Perceptual Speed, and Inhibition in Aggregated SearchACM Transactions on Information Systems10.1145/332212837:3(1-34)Online publication date: 16-May-2019
    • (2019)First International Workshop on Professional SearchACM SIGIR Forum10.1145/3308774.330879952:2(153-162)Online publication date: 17-Jan-2019
    • Show More Cited By

    View Options

    View options

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media