Visual Topic Network: Building better image representations for images in social media

摘要

Topic models have demonstrated to be effective on building image representations for general images. Recently, how to build better image representations for images in social media becomes an interesting problem, where one key issue is how to leverage images’ social contextual cues, e.g., user tags associated with images. Nevertheless, most previous methods either just exploited image content and neglect user tags, or assumed there are exact correspondences between image content and tags, i.e., tags are closely related to image content. Thus, they cannot be applied to the realistic scenarios where the images are only weakly annotated with tags, i.e., tags are only loosely related to image content as already manifested in real-world social media data. In this paper, we address the problem of building better image representations in social media, where the images are weakly annotated with user tags. In particular, we organize a collection of images as an image network where the relations between images are modeled by user tags. To model such image network and build image representations, we further propose a network structured topic model, namely Visual Topic Network (VTN), where the image content and their relations are simultaneously modeled. In this way, the weakly annotated tags can be effectively leveraged as building image representations. The proposed VTN model is inspired by the Relational Topic Model (RTM) recently introduced in the document analysis literature. Different from the binary article relations in RTM, the proposed VTN can model the multiple-level image relations. Our extensive experiments on two social media datasets demonstrated the advantage of the proposed VTN model.