Abstract
Advances in face rotation, along with other face-based generative tasks, are more frequent as we advance further in topics of deep learning. Even as impressive milestones are achieved in synthesizing faces, the importance of preserving identity is needed in practice and should not be overlooked. Also, the difficulty should not be more for data with obscured faces, heavier poses, and lower quality. Existing methods tend to focus on samples with variation in pose, but with the assumption data is high in quality. We propose a generative adversarial network (GAN) -based model to generate high-quality, identity preserving frontal faces from one or multiple low-resolution (LR) faces with extreme poses. Specifically, we propose SuperFront-GAN (SF-GAN) to synthesize a high-resolution (HR), frontal face from one-to-many LR faces with various poses and with the identity-preserved. We integrate a super-resolution (SR) side-view module into SF-GAN to preserve identity information and fine details of the side-views in HR space, which helps model reconstruct high-frequency information of faces (i.e., periocular, nose, and mouth regions). Moreover, SF-GAN accepts multiple LR faces as input, and improves each added sample. We squeeze additional gain in performance with an orthogonal constraint in the generator to penalize redundant latent representations and, hence, diversify the learned features space. Quantitative and qualitative results demonstrate the superiority of SF-GAN over others.
Introduction
Face frontalization—the process of computationally "rotating" a profile view to a front-facing view—is a staple of modern computer vision, essential for applications ranging from security surveillance to digital photography. However, most existing models suffer from a "double whammy" problem. First, there is a significant pose discrepancy: when a face is turned 60° or 90° away, half the features are occluded, making the mapping to a frontal view a highly non-linear, ill-posed problem. Second, real-world images are often low-resolution (LR). Existing frontalization models usually assume high-quality input; when fed low-resolution faces, they lose the fine details necessary for recognition.
Solving these two problems sequentially (upscaling then rotating, or vice versa) often leads to cumulative errors. The proposed SuperFront framework aims to solve them collaboratively, generating high-resolution (HR) frontal faces directly from LR profile inputs.
Methods: The SF-GAN Architecture
The core of this work is the SuperFront Generative Adversarial Network (SF-GAN). Its architecture and training regime introduce several novel elements designed to preserve identity and recover high-frequency details.
SR-Integrated Decoder
Unlike standard GANs that output only a frontal face, SF-GAN’s decoder features two branches. While the main branch performs frontalization, a parallel Side-view Super-Resolution (SR) branch reconstructs the original profile in high resolution. This ensures the model "remembers" fine details—such as the specific shape of a nose or eye—from the input before attempting the rotation.
Multi-Level Loss Functions
To achieve high fidelity, SF-GAN employs a three-level loss structure:
- Pixel-level ($L_1$): Ensures basic color and shape consistency.
- Patch-level (SSIM): Compares small patches using the Structural Similarity Index (SSIM), forcing the GAN to focus on local structures like the periocular region and mouth.
- Global-level: Combines adversarial loss for realism and an identity-preserving loss (using a pre-trained Light CNN) to ensure the subject's unique facial "fingerprint" is maintained.
Multi-Image (MI) Fusion and Orthogonal Regularization
SF-GAN can accept multiple LR profile shots. To maximize the utility of multiple inputs, the authors implement Orthogonal Regularization. This constraint penalizes redundant features, forcing the model to extract unique, complementary information from each distinct camera angle.
Results and Discussion
SF-GAN consistently outperformed previous state-of-the-art methods, such as TP-GAN and M2FPA, particularly on low-resolution inputs.
Quantitative Performance
On the Multi-PIE dataset, SF-GAN achieved a Rank-1 recognition accuracy of 92.81% on LR inputs, significantly higher than M2FPA's 79.55% under identical conditions. Furthermore, the multi-image (MI) version demonstrated that accuracy scales with the number of inputs, climbing from ~91% with one image to over 99% with four, validating the effectiveness of the fusion strategy.
Qualitative Fidelity
Visually, SF-GAN produces much sharper images than its predecessors. While other models often generate "ghostly" or blurry features when starting from LR profiles, SF-GAN successfully recovers distinct textures in critical areas like the eyes and mouth. This success stems from treating super-resolution as a "feature anchor" during the rotation process.
Conclusion
SuperFront addresses the critical challenge of synthesizing high-quality frontal faces from low-quality, extreme-pose images. By integrating super-resolution directly into the generative process and utilizing orthogonal constraints for multi-view fusion, it preserves identity far better than sequential approaches. Future work may extend this multi-view fusion to video-based recognition, where continuous frames provide a rich source of data for high-fidelity synthesis.
Summary updated