Multi-Process Training GAN for Identity-Preserving Face Synthesis

Zhiyong Tang, Jianbing Yang, Zhongcai Pei, Xiao Song, Baoshuang GeIEEE Access 2019 · OpenReview

Abstract

Recently, the advent of generative adversarial networks (GANs) in synthesizing identity-preserving faces has aroused the considerable interest of many scholars. However, face attribute representation learning, which is explicitly disentangled from identity feature and synthesizes identity-preserving face images with high diversity and quality in other datasets, still remains challenging. To cope with that, this paper proposes multi-process training GAN, or MP-GAN for short, which significantly improves the disentangled representation, diversity, and quality. Unlike other existing single-process models that map noise to a final output resolution image in a single training process, MP-GAN divides training into multiple processes. The main idea is to first generate lower resolution images that contain lower frequency feature information through competition and then extract their disentangled facial features to generate a higher resolution image. Furthermore, an identity-preserving image with real identity feature and disentangled facial feature could be generated at the final output resolution training process. The distinct benefits are not only getting diverse facial feature generation but also achieving disentangled representation from the lower resolution training processes and rendering a photo-realistic image that contains high diversity but preserves identity at the final output resolution training process. The high performance of this method is highlighted by quantitative and qualitative comparisons. We conclude that MP-GAN can generate face images featuring high diversity and quality while efficiently preserving identity, thereby significantly outperforming most modern advanced methods.

Introduction

In the field of face synthesis, researchers face a persistent challenge: the "tug-of-war" between identity (the unique features of a person) and attributes (variable factors like pose, expression, or lighting). The goal is to manipulate attributes—such as making a subject smile or look to the side—without altering their fundamental identity.

Traditional Generative Adversarial Networks (GANs) often struggle with this disentanglement. Most existing models utilize a single-process approach, attempting to map random noise directly to a high-resolution output. This frequently results in "tangled" features, where changing an expression inadvertently alters the person's bone structure or eye color, leading to a loss of identity preservation and photo-realism.

Traditional single-process training architecture, which often leads to feature entanglement.

Methodology

Multi-Process Training Framework

To address the limitations of single-stage models, the authors introduce MP-GAN. This framework departs from standard training by adopting a progressive, multi-process strategy. Instead of targeting high resolution immediately, MP-GAN begins at a base resolution of $8 \times 8$. Once the model masters the basic structural features at this scale, layers are added to double the resolution ($16 \times 16$, $32 \times 32$, up to $128 \times 128$).

This "foundation first" philosophy allows the model to learn low-frequency information, such as head pose and general shape, before attempting to render high-frequency details like skin texture or fine wrinkles. This hierarchical learning significantly improves the stability of the training process and the quality of the final output.

The MP-GAN multi-process training framework, illustrating the progressive resolution increase from $8 \times 8$ to $n \times n$.

Disentangled Feature Extraction and Supervision

MP-GAN achieves superior identity preservation through explicit feature-wise supervision. The model utilizes four specialized, pre-trained encoders to extract distinct vectors from real images:

  • Expression ($f_e$): Extracted via ExpNet.
  • Pose ($f_p$): Extracted via FacePoseNet.
  • Shape ($f_s$): Derived using a 3D Morphable Model (3DMM) network.
  • Identity ($f_{id}$): Extracted using the FaceNet recognition engine.

In the final training stages, the model does not rely solely on the Discriminator's binary judgment. It employs an $L_2$ loss to ensure the features of the generated image strictly match the features of the target identity. This forces the generator to remain "identity-aware" even as it manipulates other attributes.

Experimental Results

Datasets and Evaluation

The framework was trained on massive datasets including CelebA (202k images) and CASIA-WebFace (494k images). Evaluation was conducted on the LFW (Labeled Faces in the Wild) and IJB-A benchmarks to measure identity preservation performance.

Quantitative and Qualitative Performance

MP-GAN demonstrated significant improvements over state-of-the-art models such as FaceFeat-GAN and FaceID-GAN:

  • Verification Accuracy: On the LFW benchmark, MP-GAN achieved an identity verification accuracy of 97.94%, outperforming contemporary methods.
  • Robustness: Qualitative results confirm the model's ability to handle extreme pose rotations and varied illumination while keeping the subject recognizable.
  • Latent Space Manipulation: Because features are disentangled, the model supports smooth interpolation. For example, researchers can generate a neutral expression by finding the midpoint between "Happy" and "Sad" latent vectors for a specific identity.

Conclusion and Future Work

MP-GAN represents a significant step forward in identity-preserving face synthesis by utilizing a multi-process training regime and explicit feature supervision. While the model excels at standard expressions and poses, future work will focus on addressing unnatural distortions in extreme expressions (e.g., intense surprise) and mitigating dataset biases that favor neutral or smiling faces. Additionally, the authors aim to incorporate demographic factors like age and race more explicitly to further refine the precision of disentangled representations.

Summary updated

Abstract

Recently, the advent of generative adversarial networks (GANs) in synthesizing identity-preserving faces has aroused the considerable interest of many scholars. However, face attribute representation learning, which is explicitly disentangled from identity feature and synthesizes identity-preserving face images with high diversity and quality in other datasets, still remains challenging. To cope with that, this paper proposes multi-process training GAN, or MP-GAN for short, which significantly improves the disentangled representation, diversity, and quality. Unlike other existing single-process models that map noise to a final output resolution image in a single training process, MP-GAN divides training into multiple processes. The main idea is to first generate lower resolution images that contain lower frequency feature information through competition and then extract their disentangled facial features to generate a higher resolution image. Furthermore, an identity-preserving image with real identity feature and disentangled facial feature could be generated at the final output resolution training process. The distinct benefits are not only getting diverse facial feature generation but also achieving disentangled representation from the lower resolution training processes and rendering a photo-realistic image that contains high diversity but preserves identity at the final output resolution training process. The high performance of this method is highlighted by quantitative and qualitative comparisons. We conclude that MP-GAN can generate face images featuring high diversity and quality while efficiently preserving identity, thereby significantly outperforming most modern advanced methods.
BibTeX
@inproceedings{tang2019multi,
  author = {Zhiyong Tang and Jianbing Yang and Zhongcai Pei and Xiao Song and Baoshuang Ge},
  title = {{Multi-Process Training GAN for Identity-Preserving Face Synthesis}},
  booktitle = {IEEE Access},
  year = {2019},
  url = {https://openreview.net/forum?id=9eOQwyXZB9},
}
Figure