research-article

I-REX: A Lucene Plugin for EXplainable IR

Authors:

Debasis GangulyAuthors Info & Claims

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Pages 2949 - 2952

https://doi.org/10.1145/3357384.3357859

Published: 03 November 2019 Publication History

Abstract

Providing high-level, intuitive explanations of the performance of IR systems is generally difficult due to their complexity, and the various low-level implementation details involved. We present I-REX, a tool built on top of Lucene, that is intended to provide a systematic view into the inner workings of retrieval models and methods (specifically query expansion). This should help researchers study, compare, understand and explain the performance of these models and methods. I-REX can be run either as a Web service accessible through a browser, or as a terminal-based tool with a shell-like interactive interface. In this article, we describe a session that illustrates how I-REX can be used to explain the observed difference in the performance of two variants of the Language Model.

References

[1]

L. Azzopardi, M. Crane, H. Fang, G. Ingersoll, J. Lin, Y. Moshfeghi, H. Scells, P. Yang, and G. Zuccon. 2017a. The Lucene for Information Access and Retrieval Research (LIARR) Workshop at SIGIR 2017. In Proc. SIGIR. 1429--1430.

[2]

L. Azzopardi, Y. Moshfeghi, M. Halvey, R. S. Alkhawaldeh, K. Balog, E. Di Buccio, D. Ceccarelli, J. M. Fernández-Luna, C. Hull, J. Mannix, and S. Palchowdhury. 2017b. Lucene4IR: Developing IR Evaluation Resources using Lucene . SIGIR Forum, Vol. 50, 2 (Feb. 2017), 58--75.

Digital Library

[3]

T. Bolukbasi, K. Chang, J. Y. Zou, V. Saligrama, and A. Kalai. 2016. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. In Proc. of NIPS 29 . 4349--4357.

[4]

C. Buckley. 1985. Implementation of the SMART Information Retrieval System . Technical Report. Ithaca, NY, USA.

[5]

S. Clinchant and É Gaussier. 2013. A Theoretical Analysis of PRF Models. In Proc. of ICTIR '13. 6--13.

Digital Library

[6]

L. Dixon, J. Li, J. Sorensen, N. Thain, and L. Vasserman. 2018. Measuring and mitigating unintended bias in text classification. In Proc. of 2018 AAAI/ACM Conference on AI, Ethics, and Society. ACM, 67--73.

[7]

H. Fang and C. Zhai. 2005. An exploration of axiomatic approaches to information retrieval. In Proc. of 28th SIGIR . ACM, 480--487.

[8]

Zeon Trevor Fernando, Jaspreet Singh, and Avishek Anand. 2019. A study on the Interpretability of Neural Retrieval Models using DeepSHAP. In Proc. of SIGIR'19 . 1005--1008.

Digital Library

[9]

D. Metzler and W. B. Croft. 2007. Linear Feature-based Models for Information Retrieval. Inf. Retr., Vol. 10, 3 (June 2007), 257--274.

Digital Library

[10]

D Roy, M. Mitra, and D. Ganguly. 2018. To Clean or Not to Clean: Document Preprocessing and Reproducibility. J. Data and Info. Quality, Vol. 10, 4, Article 18 (Oct. 2018), bibinfonumpages25 pages.

[11]

A. Singhal, C. Buckley, and M. Mitra. 1996. Pivoted Document Length Normalization. In Proc. of SIGIR (SIGIR '96). 21--29.

[12]

M. Smucker and J. Allan. 2005. An investigation of Dirichlet prior smoothing's performance advantage . Technical Report. CIIR, U. Mass., Amherst.

[13]

Manisha Verma and Debasis Ganguly. 2019. LIRME: Locally Interpretable Ranking Model Explanation. In Proc. of SIGIR'19 . 1281--1284.

Digital Library

[14]

P. Yang, H. Fang, and J. Lin. 2017. Anserini: Enabling the Use of Lucene for Information Retrieval Research. In Proc. of the 40th SIGIR (SIGIR '17). 1253--1256.

[15]

P. Yang, H. Fang, and J. Lin. 2018. Anserini: Reproducible Ranking Baselines Using Lucene. J. Data and Information Quality, Vol. 10, 4, Article 16 (Oct. 2018), bibinfonumpages20 pages.

Digital Library

Cited By

Ferraretto FLaitz TLotufo RNogueira RChen HDuh WHuang HKato MMothe JPoblete B(2023)ExaRanker: Synthetic Explanations Improve Neural RankersProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3592067(2409-2414)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3592067
Polley S(2022)Towards Explainable Search in Legal TextAdvances in Information Retrieval10.1007/978-3-030-99739-7_65(528-536)Online publication date: 5-Apr-2022
https://doi.org/10.1007/978-3-030-99739-7_65

Index Terms

I-REX: A Lucene Plugin for EXplainable IR
1. Information systems
  1. Information retrieval
    1. Evaluation of retrieval results
    2. Users and interactive retrieval
      1. Search interfaces

Recommendations

A Deep Analysis of an Explainable Retrieval Model for Precision Medicine Literature Search
Advances in Information Retrieval
Abstract
Professional search queries are often formulated in a structured manner, where multiple aspects are combined in a logical form. The information need is often fulfilled by an initial retrieval stage followed by a complex reranking algorithm. In ...
REX: explaining relationships between entity pairs

Knowledge bases of entities and relations (either constructed manually or automatically) are behind many real world search engines, including those at Yahoo!, Microsoft, and Google. Those knowledge bases can be viewed as graphs with nodes representing ...
BiTe-REx: An Explainable Bilingual Text Retrieval System in the Automotive Domain
SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

To satiate the comprehensive information need of users, retrieval systems surpassing the boundaries of language are inevitable in the present digital space in the wake of an ever-rising multilingualism. This work presents the first-of-its-kind Bilingual ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

November 2019

3373 pages

ISBN:9781450369763

DOI:10.1145/3357384

General Chairs:
Wenwu Zhu
Tsinghua University, China
,
Dacheng Tao
University of Massachusetts, USA
,
Xueqi Cheng
Institute of Computing Technology, CAS, China
,
Program Chairs:
Peng Cui
Tsinghua University, China
,
Elke Rundensteiner
Worcester Polytechnic Institute, USA
,
David Carmel
Amazon Research, USA
,
Qi He
LinkedIn, USA
,
Jeffrey Xu Yu
Chinese University of Hong Kong, China

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM '19

Sponsor:

CIKM '19: The 28th ACM International Conference on Information and Knowledge Management

November 3 - 7, 2019

Beijing, China

Acceptance Rates

CIKM '19 Paper Acceptance Rate 202 of 1,031 submissions, 20%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
182
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)1

Reflects downloads up to 18 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ferraretto FLaitz TLotufo RNogueira RChen HDuh WHuang HKato MMothe JPoblete B(2023)ExaRanker: Synthetic Explanations Improve Neural RankersProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3592067(2409-2414)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3592067
Polley S(2022)Towards Explainable Search in Legal TextAdvances in Information Retrieval10.1007/978-3-030-99739-7_65(528-536)Online publication date: 5-Apr-2022
https://doi.org/10.1007/978-3-030-99739-7_65

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents