With the introduction of new types of devices in our everyday lives such as intelligent assistants, the way people access information and their expectations have significantly changed. Most interactions now happen through conversations held over devices ...

The study of IR evaluation metrics through axiomatic analysis enables a better understanding of their numerical properties. Some works have modelled the effectiveness of retrieval metrics with axioms that capture desirable properties on the set of ...

In this work we presentWooIR, an open realistic benchmark for Page Stream Segmentation (PSS), the task of recovering document boundaries from aggregatedstreams of pages. Our dataset consists of over 200 streams of scanned in documents, 7K documents, 45K ...

Knowledge graph question answering (KGQA) is important to make structured information accessible without formal query language expertise on the part of the users. The semantic parsing (SP) flavor of this task maps a natural language question to a formal ...

Pairwise re-ranking models predict which of two documents is more relevant to a query and then aggregate a final ranking from such preferences. This is often more effective than pointwise re-ranking models that directly predict a relevance value for ...

Term-based ranking with pre-trained transformer-based language models has recently gained attention as they bring the contextualization power of transformer models into the highly efficient term-based retrieval. In this work, we examine the ...

Outlier data points are known to affect negatively the learning process of regression or classification models, yet their impact in the learning-to-rank scenario has not been thoroughly investigated so far. In this work we propose SOUR, a learning-to-...

Due to the sequential and interactive nature of conversations, the application of traditional Information Retrieval (IR) methods like the Cranfield paradigm require stronger assumptions. When building a test collection for Ad Hoc search, it is fair to ...

BCubed is a mathematically clean, elegant and intuitively well behaved external performance metric for clustering tasks. BCubed compares a predicted clustering to a known ground truth through elementwise precision and recall scores. For each element, ...