An information measure of retrieval performance

摘要

The classical measures of information retrieval quality, namely precision and recall, each have disadvantages depending on the circumstances. In judging the output of an automatic retrieval procedure it would be useful to have a single number which is practical to compute and summarizes the quality of the output. Such a measure could facilitate the study of retrieval methodology and clarify efforts to optimize retrieval. We show how such a measure may be defined for retrieval methods with ranked output. It is based on the change in the Shannon entropy of the distribution of relevant documents in the database that is produced by the act of retrieval. Properties of this new measure are derived and reveal that it conforms to intuitive expectations.