Domain adaptive representation learning for facial action unit recognition

作者：

Highlights：

• Facial action unit recognition system is improved by a novel approach that learns a coordinated representation through cross-modality supervision.

• The proposed method shows superior generalization capabilities of the trained latent representation when applied to datasets lacking thermal images, complementing the performance of the dataset-specific action unit recognizers.

• The transfer learning technique that was proposed in order to adapt the learned representation to the target domain improved performance on target domain datasets.

• Multimodal conditional feature enhancement method, which is presented, proves that creating a unified representation for fusion, may not be required if both source modalities are present.

摘要

•Facial action unit recognition system is improved by a novel approach that learns a coordinated representation through cross-modality supervision.•The proposed method shows superior generalization capabilities of the trained latent representation when applied to datasets lacking thermal images, complementing the performance of the dataset-specific action unit recognizers.•The transfer learning technique that was proposed in order to adapt the learned representation to the target domain improved performance on target domain datasets.•Multimodal conditional feature enhancement method, which is presented, proves that creating a unified representation for fusion, may not be required if both source modalities are present.

论文关键词：Feature fusion,Feature fine-tuning,Facial action unit recognition,Deep fusion,Multi-Modal representation learning

论文评审过程：Received 1 December 2018, Revised 28 October 2019, Accepted 21 November 2019, Available online 28 November 2019, Version of Record 30 January 2020.

论文官网地址：https://doi.org/10.1016/j.patcog.2019.107127