Relational graph neural network for situation recognition

作者：

Highlights：

• We propose a novel relational graph neural network for situation recognition, which explicitly models the triplet relationships between the activity and the objects.

• We propose a progressive supervised learning method which incrementally adds the weights to the cross entropy loss.

• To harmonize the training and testing procedures, we use a policy-gradient method to directly optimize the non-differentiable value-all metric.

• We conduct extensive experiments to quantitatively evaluate our proposed method on the only available dataset Imsitu. Experimental results demonstrate that our proposed method performs much better than other state-of-the-art methods.

摘要

•We propose a novel relational graph neural network for situation recognition, which explicitly models the triplet relationships between the activity and the objects.•We propose a progressive supervised learning method which incrementally adds the weights to the cross entropy loss.•To harmonize the training and testing procedures, we use a policy-gradient method to directly optimize the non-differentiable value-all metric.•We conduct extensive experiments to quantitatively evaluate our proposed method on the only available dataset Imsitu. Experimental results demonstrate that our proposed method performs much better than other state-of-the-art methods.

论文关键词：Situation recognition,Relationship modeling,Graph neural network,Reinforcement learning

论文评审过程：Received 26 April 2019, Revised 30 September 2019, Accepted 11 July 2020, Available online 12 July 2020, Version of Record 24 July 2020.

论文官网地址：https://doi.org/10.1016/j.patcog.2020.107544