Presenting results of experimental retrieval comparisons

摘要

Methods of calculating and presenting results from experimental retrieval comparisons are considered and illustrated by some new laboratory results. The measures used center on Recall and Precision. Topics of data calculation, single value measures, benchmark results, data aggregation, statistical significance, and the presentation of performance differences are discussed. User-oriented presentations can be used to simulate different needs such as high or low levels of Recall. Several methods of retrieval cutoff can be used as the control variable, but the document cutoff is the most useful. Two difficult performance comparisons are illustrated: Boolean versus Ranked output retrieval and non-iterative versus relevance feedback.