Abstract
Generating high-resolution facial images under extreme poses and expressions remains a significant challenge for existing generative models. We present a two-stage framework that utilizes dense boundary prediction and a novel Feature Threshold Loss to effectively disentangle facial structure from fine-grained texture. Our approach maintains high fidelity and identity preservation even at 90-degree profile views. Furthermore, we introduce the Multi-View Face High Quality (MVF-HQ) dataset, containing over 120,000 ultra-high-resolution images, to address the scarcity of diverse multi-view data.
Introduction
Face manipulation—the digital alteration of a subject's pose or expression—has seen remarkable progress through Generative Adversarial Networks (GANs). However, current state-of-the-art methods often fail when tasked with high-resolution ($1024 \times 1024$) synthesis involving extreme variations. The primary difficulty lies in the entanglement of facial structure (e.g., the shape of the nose) and texture (e.g., skin pores and hair). Standard disentanglement techniques often result in "blurry" outputs or a loss of the subject's unique identity. This work proposes a framework that overcomes these limitations by providing dense structural guidance and a flexible loss function that preserves high-frequency details.
Methodology
The proposed system decomposes the manipulation task into two correlated stages: boundary prediction and disentangled synthesis.
Boundary Prediction Stage
Rather than using sparse landmarks, which provide insufficient guidance for high-resolution tasks, our model predicts a dense "boundary image." This sketch-like map captures the underlying geometry of the target pose and expression, ensuring structural correctness before pixel synthesis begins.
Disentangled Face Synthesis and Feature Threshold Loss
The synthesis stage employs separate encoders for the predicted boundary (structure) and the source image (texture). To solve the texture-loss problem inherent in identity-preserving networks, we introduce the Feature Threshold Loss. Using a pre-trained face recognition network as a structural proxy, this loss only penalizes differences that exceed a specific margin ($m$). This allows the model to satisfy structural constraints while retaining the fine-grained textures that make a face appear realistic.
The MVF-HQ Dataset
A major bottleneck in training high-fidelity models is the lack of high-resolution datasets with extreme poses. Datasets like CelebA-HQ, while high-quality, are predominantly front-facing. To address this, we introduce MVF-HQ (Multi-View Face High Quality), featuring 120,283 images at $6000 \times 4000$ resolution. The dataset includes 13 camera views ranging from $-90^\circ$ to $+90^\circ$, providing a much-needed benchmark for extreme-angle synthesis.
Results
Our framework achieves state-of-the-art results on standard benchmarks. On the MultiPIE dataset, the model maintains a 99.9% Rank-1 recognition accuracy for $15^\circ$ poses and 87.4% at $90^\circ$, significantly outperforming previous models like CAPG-GAN (66% at $90^\circ$). Qualitative evaluations at $1024 \times 1024$ resolution show exceptional sharpness in fine details such as teeth and eyes, even during extreme expressions like screaming. Ablation studies confirm that the Feature Threshold Loss is critical for maintaining sharpness, while the identity-preserving loss is essential for recognizing the subject.
Conclusion
By integrating dense boundary guidance with a thresholded disentanglement loss, we enable high-fidelity face manipulation across extreme poses and expressions. The introduction of the MVF-HQ dataset further supports the development of robust, high-resolution multi-view synthesis models. Future research will focus on extending these capabilities to unconstrained, "in-the-wild" environments.
Summary updated