Hard negative generation for identity-disentangled facial expression recognition

Highlights：

• We extract identity-disentangled representations for facial expression recognition (FER) without requiring expressive-neutral pairs in a testing task.

• Proposing a novel recognition via generation scheme as a substitution of conventional hard sample mining.

• The distance comparisons are largely reduced from K3(triplet loss) to 2 K, where K is the number of sample in a training batch.

• Our new architecture achieves the state-of-the-art on 3 popular FER datasets, which do not have neutral expression samples.

• We alleviate the difficulty of threshold validation and anchor selection in conventional deep metric learning.

• We learn distance metrics with much fewer distance calculations and training iterations, without sacrificing the performance.

• We optimize the softmax loss and metric learning loss jointly based on their characteristics and tasks.

• A novel approach to generate photorealistic and identity-preserved normalized face image.

摘要

•We extract identity-disentangled representations for facial expression recognition (FER) without requiring expressive-neutral pairs in a testing task.•Proposing a novel recognition via generation scheme as a substitution of conventional hard sample mining.•The distance comparisons are largely reduced from K3(triplet loss) to 2 K, where K is the number of sample in a training batch.•Our new architecture achieves the state-of-the-art on 3 popular FER datasets, which do not have neutral expression samples.•We alleviate the difficulty of threshold validation and anchor selection in conventional deep metric learning.•We learn distance metrics with much fewer distance calculations and training iterations, without sacrificing the performance.•We optimize the softmax loss and metric learning loss jointly based on their characteristics and tasks.•A novel approach to generate photorealistic and identity-preserved normalized face image.