Toward learning a unified many-to-many mapping for diverse image translation

Highlights：

• A novel deep generative model is proposed for image-to-image translation. This model learns a many-to-many mapping function among multiple domains with unpaired training data.

• The proposed model unifies the generative adversarial network and variational autoencoder to explore the latent space, which is indicated by domain-specific features and unspecific random variations.

• A novel neural network structure is developed to combine the input images with latent variables. The input of the model is a combination of the observed image, domain-specific features, and unspecific variations. Within one unified framework, the trained model generates diverse samples in multiple domains.

• The proposed model is qualitatively and quantitatively evaluated on multiple datasets with respect to style transfer and the facial attribute transfer tasks. Its diverse generations with high quality reflect a superior performance over baseline models.

摘要

•A novel deep generative model is proposed for image-to-image translation. This model learns a many-to-many mapping function among multiple domains with unpaired training data.•The proposed model unifies the generative adversarial network and variational autoencoder to explore the latent space, which is indicated by domain-specific features and unspecific random variations.•A novel neural network structure is developed to combine the input images with latent variables. The input of the model is a combination of the observed image, domain-specific features, and unspecific variations. Within one unified framework, the trained model generates diverse samples in multiple domains.•The proposed model is qualitatively and quantitatively evaluated on multiple datasets with respect to style transfer and the facial attribute transfer tasks. Its diverse generations with high quality reflect a superior performance over baseline models.