Learning multiscale hierarchical attention for video summarization

Highlights：

• Different from conventional works that employ either the hierarchical RNN for structure modeling or the self-attention mechanism for long-range dependencies, our method applies the intra-block attention and the inter-block attention to learn both the underlying structure and long-range representations.

• Our method learns multiscale features, where the intra-block attention module learns the attentive frame-level features inside each block while the inter-block attention module learns the attentive block-level features.

• Moreover, we further extend our method into a two-stream framework to leverage both the appearance and motion information.

• We conduct comprehensive experiments on two widely-used video summarization datasets. Our method achieves very competitive performance compared with the state-of-the-art methods.