Experiments with document components for indexing and retrieval

摘要

The use of document components—such as a term, a sentence, or a whole document — for indexing and retrieval has been investigated employing two medium-size collections. A number of probabilistic similarity measures based on document components are studied, as well as a new method of handling probability estimates involving small sample sizes. It is seen that some of the new similarity measures can provide comparable performance to those methods studied by other investigators. In general, the term and sentence modes result in substantially equal performance, and both are superior to the document mode. However, it may be necessary to use longer documents in order to reveal fully the usefulness of the sentence mode, because documents in our databases may not have sufficient numbers of sentences.