Diffusion Prior Interpolation for Flexibility Real-World Face Super-Resolution

Jiarui Yang, Tao Dai, Yufei Zhu, Naiqi Li, Jinmin Li, Shu-Tao Xia

Introduction

Face Super-Resolution (FSR) aims to restore high-resolution (HR) details from degraded low-resolution (LR) inputs. A fundamental challenge in FSR is the "consistency vs. diversity" trade-off: models must maintain pixel-level fidelity to the original identity (consistency) while generating sharp, realistic textures like skin pores and hair (diversity/fidelity). While discriminative models often produce over-smoothed results, generative models like Diffusion Models can hallucinate details that deviate from the subject's identity.

Methodology: Diffusion Prior Interpolation (DPI)

The proposed Diffusion Prior Interpolation (DPI) framework addresses these challenges by refining the diffusion sampling process through two primary mechanisms: a two-stage masking strategy and a neural Condition Corrector.

Two-Stage Masking Strategy

DPI divides the reverse diffusion process into two distinct phases based on a time-step threshold $\tau$:

  • Stage 1 (Coarse Generation): Employs a Fixed Condition Mask (FCM) to enforce strict global structure and identity preservation during the early stages of sampling.
  • Stage 2 (Fine Detail Interpolation): Utilizes a Randomly Adaptive Condition Mask (RACM). This probabilistic mask focuses on high-frequency regions (edges), allowing the model to interpolate realistic textures without compromising the underlying facial structure.

Condition Corrector (CRT)

To mitigate artifacts caused by injecting LR conditions into pre-trained diffusion models, the authors introduce the Condition Corrector (CRT). This lightweight module acts as a "pull-back" mechanism, ensuring that intermediate conditions remain aligned with the pre-trained model's manifold. This creates a reciprocal refinement loop where the condition guides the sample, and the evolving sample helps refine the condition.

Experimental Results

The DPI framework was evaluated on synthetic (FFHQ1000, CelebA1000) and real-world (LFW, WebPhoto, WIDER) datasets.

Comparison of DPI against state-of-the-art methods. The bottom row illustrates the flexible interpolation between consistency and diversity.

As shown in Figure 3, DPI achieves superior visual quality compared to existing methods like CodeFormer and PGDiff. Quantitatively, on the CelebA1000 ($\times16$) task, DPI reached an LPIPS of 0.1927 and an FID of 27.33. Crucially, DPI maintains high face recognition consistency, achieving 72.8% accuracy at $\times16$ upscaling, significantly outperforming other diffusion-based approaches like PGDiff (59.8%).

Conclusion and Limitations

DPI provides a flexible framework for face super-resolution by balancing identity preservation with high-fidelity generation. While the method demonstrates state-of-the-art performance, it relies on hyperparameter tuning ($\tau, s, \omega$) for optimal results across different degradation levels. Additionally, the iterative nature of the diffusion process and the CRT module add computational overhead compared to simpler discriminative models. Future work may extend this masking and correction strategy to general image restoration and video super-resolution.

Summary updated

BibTeX
@inproceedings{yang2025diffusion,
  author = {Jiarui Yang and Tao Dai and Yufei Zhu and Naiqi Li and Jinmin Li and Shu-Tao Xia},
  title = {{Diffusion Prior Interpolation for Flexibility Real-World Face Super-Resolution}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i9.32997},
}
Figure