Unsupervised learning of optical flow with patch consistency and occlusion estimation

作者：

Highlights：

• A patch-based census constancy loss is proposed by patch-based warping to further improve the performance of unsupervised optical flow estimation.

• A parallel decoder branch is devised for occlusion mask learning in an unsupervised way.

• The pseudo label generated from the forward-backward consistency check is used to derive a mask loss for occlusion mask learning.

• Our method achieves the-state-of-the-art results among the unsupervised learning methods that are using the FlowNet-liked network structure.

摘要

•A patch-based census constancy loss is proposed by patch-based warping to further improve the performance of unsupervised optical flow estimation.•A parallel decoder branch is devised for occlusion mask learning in an unsupervised way.•The pseudo label generated from the forward-backward consistency check is used to derive a mask loss for occlusion mask learning.•Our method achieves the-state-of-the-art results among the unsupervised learning methods that are using the FlowNet-liked network structure.

论文关键词：Patch consistency,Optical flow estimation,Occlusion estimation,Unsupervised learning,Deep learning

论文评审过程：Received 24 April 2019, Revised 23 November 2019, Accepted 24 December 2019, Available online 3 January 2020, Version of Record 22 February 2020.

论文官网地址：https://doi.org/10.1016/j.patcog.2019.107191