Learning spatial-temporal deformable networks for unconstrained face alignment and tracking in videos

Highlights：

• In our approach, we propose a differential transformer module, which typically learns the deformable offsets to adaptively augment the receptive field. By doing this, we achieve shape-informative and robust feature representation versus conventional fixed-length filters for high-performance face alignment.

• Extensively, we carefully develop a temporal relational reasoning module integrated in our DHGN. Consequently, it infers the temporal offsets to capture the meaningful ordinal relationship among frames, which reinforces the robustness of our model to temporal variations over time steps.

• Experimentally, our DHGN consistently outperforms most existing face alignment methods on 300-W challengingset and COFW including large variations. Moreover, our proposed T-DHGN achieves smoothing performance in the difficult face tracking dataset, i.e., 300-VW Category Three.