An exocentric look at egocentric actions and vice versa

摘要

In this work we address the task of relating action information across two drastically different visual domains, namely, first-person (egocentric) and third-person (exocentric). We investigate two different yet highly interconnected problems including cross-view action classification and action based video retrieval. First, we perform action classification in one domain using the knowledge transferred from the other domain. Second, given a video in one view, we retrieve videos from the same action class in the other view. In order to evaluate our models, we collect a new cross-domain dataset of egocentric-exocentric action videos containing 14 action classes and 3569 videos (1676 collected egocentric videos and 1893 exocentric videos borrowed from the UCF 101 dataset). Our results demonstrate the possibility of transferring action information across the two domains and suggest new directions in relating first and third person vision for other tasks.