Projective Matrix Factorization with unified embedding for social image tagging

摘要

This paper presents a general formulation, named ProJective Matrix Factorization with unified embedding (PJMF), by which social image retagging is transformed to the nearest tag-neighbor search for each image. We solve the proposed PJMF as an optimization problem mainly considering the following issues. First, we attempt to find two latent representations in a unified space for images and tags respectively and explore the two representations to reconstruct the observed image-tag correlation in a nonlinear manner. In this case, the relevance between an image and a tag can be directly modeled as the pair-wise similarity in the unified space. Second, the image latent representation is assumed to be projected from its original visual feature representation with an orthogonal transformation matrix. The projection makes convenient to embed any images including out-of-samples into the unified space, and naturally the image retagging problem can be solved by the nearest tag-neighbors search for those images in the unified space. Third, local geometry preservations of image space and tag space respectively are explored as constraints in order to make image similarity (and tag relevance) consistent in the original space and the corresponding latent space. Experimental results on two publicly available benchmarks validate the encouraging performance of our work over the state-of-the-arts.