IP-GAN: Learning Identity and Pose Disentanglement in Generative Adversarial Networks

Abstract

Synthesizing realistic multi-view face images from a single-view input is an effective and cheap way for data augmentation. In addition it is promising for more efficiently training deep pose-invariant models for large-scale unconstrained face recognition. It is a challenging generative learning problem due to the large pose discrepancy between the synthetic and real face images, and the need to preserve identity after generation. We propose IP-GAN, a framework based on Generative Adversarial Networks to disentangle the identity and pose of faces, such that we can generate face images of a specific person with a variety of poses, or images of different identities with a particular pose. To rotate a face, our framework requires one input image of that person to produce an identity vector, and any other input face image to extract a pose embedding vector. Then we recombine the identity vector and the pose vector to synthesize a new face of the person with the extracted pose. Two learning pathways are introduced, the generation and the transformation, where the generation path focuses on learning complete representation in the latent embedding space. While the transformation path focuses on synthesis of new face images with target poses. They collaborate and compete in a parameter-sharing manner, and in an unsupervised settings. The experimental results demonstrate the effectiveness of the proposed framework.