Improving keyword based web image search with visual feature distribution and term expansion

摘要

This paper discusses techniques for improving the performance of keyword-based web image queries. Firstly, a web page is segmented into several text blocks based on semantic cohesion. The text blocks which contain web images are taken as the associated texts of corresponding images and TF*IDF model is initially used to index those web images. Then, for each keyword, both relevant web image set and irrelevant web image set are selected according to their TF*IDF values. And visual feature distributions of both positive image and negative image are modeled using Gaussian Mixture Model. An image’s relevance to the keyword with respect to visual feature is thus defined as the ratio of positive distribution density over negative distribution density. We combine the text-based relevance model with visual feature relevance model to improve the performance. Thirdly, a query expansion model is used to improve the performance further. Expansion terms are selected according to their cooccurrences with the query terms in the top-relevant set of the original query. Our experiments show that our approach yield significant improvement over the traditional keyword based query model.