A framework for effective retrieval

Reviewer: Robert G Crawford

In their conclusion, the authors note four reasons for the mediocre retrieval performance of existing systems: “(i) Terms are not independent. (ii) Some relevant documents may not have any terms in common with a given query and therefore they cannot be retrieved. (iii) Term weights in documents are not assigned optimally. (iv) Previous parameter estimation techniques are incapable of estimating the parameter values correctly. . . . ” They propose a framework to address these four problems. The nonbinary independence model is introduced as a way of taking into consideration the frequency of occurrences of terms in documents. The authors also suggest methods to alleviate the term independence assumption, present an approach to estimating the parameters required by the model, and give a method to collect statistics, allowing for different usages of any particular term. The paper notes, “As in all other probabilistic models, parameters . . . need to be estimated from previously retrieved relevant and irrelevant documents” and “it is assumed that the document collection has been used for some time by a given user.” It is critical that these assumptions be understood in reading the work, and the authors have been careful in this regard. With these underlying requirements, the authors analyze results for small collections that have known queries and relevance assessments. The paper is clearly written. The mathematics is carefully done and is presented in a way that the reader can follow. Those interested in retrieval models, whether theoretically or for practical implementation, should read this solid piece of work.