A comparative review of graph convolutional networks for human skeleton-based action recognition

摘要

Human action recognition is one of the hottest topics in the research field, so there are many relevant review papers illustrating the multi-modality of data, the selection of feature vectors, and the pros and cons of classification networks. With the continuous development of relational networks, graph convolutional networks (GCNs) have been applied to many different fields, including human action recognition. Although the graph convolutional networks have been demonstrated the powerful functionality in human action recognition, few literatures review GCN-based human action recognition. In this review, we not only give a detailed introduction to the structure of graph convolutional networks and data modalities used for human action recognition, but also focus on the application of GCNs in the field of human action recognition. Most importantly, we conduct experiments on five benchmark datasets for comparing the performance of seven state-of-the-art GCN-based algorithms for human skeleton-based action recognition. The five datasets selected in the experiments cover data of different scales (large-scale vs. small-scale) and different types (single-person, human-object interaction, and two-person-interaction) to explore the promising applicable scope of graph networks. We adopt the frequently used performance metrics such as accuracy, network parameters and loss function. Specifically, we analyze the impact of the multi-stream fusion strategies on improving the performance of the human action recognition schemes. To our best knowledge, it is the first time to survey human action recognition strategies related to GCNs, and to give a thorough experimental comparison of GCN-based human action recognition techniques with various datasets.