Abstract
Blind Face Restoration (BFR) aims to recover high-quality facial images from inputs with unknown degradations. While Diffusion Probabilistic Models (DDPMs) have set new standards for generative quality, their high computational cost and slow inference speeds limit practical application. This paper introduces TD-BFR, a truncated, three-stage diffusion pipeline designed to bridge the gap between the efficiency of GAN-based methods and the superior generative priors of diffusion models. By leveraging a multi-resolution framework and an adaptive degradation removal module, TD-BFR achieves an average 4.75× speedup over state-of-the-art diffusion methods, delivering high-fidelity results in approximately 3.5 seconds per image.
1. Introduction
Blind Face Restoration is a challenging task due to the complex and unknown nature of real-world degradations, such as blur, noise, and compression artifacts. Recent advancements have seen a shift from Generative Adversarial Networks (GANs)—which are fast but often produce "plastic" or over-smoothed results—to Diffusion Models. However, standard diffusion models typically require hundreds of sampling steps, often taking 10–25 seconds to process a single face. Furthermore, these models can occasionally introduce semantic artifacts or struggle with fine-grained detail recovery. TD-BFR addresses these limitations through a truncated sampling strategy that avoids starting from pure Gaussian noise, significantly reducing the number of required iterations.
2. Methodology
The core of TD-BFR is a multi-resolution framework that "stitches" together partial diffusion processes across different scales. This approach is supported by the authors' discovery of Resolution-Noise Duality: intermediate variables at different resolutions exhibit similar noise distributions at specific timesteps, allowing the model to transition between resolutions without restarting the denoising process.
2.1 Low-resolution Startup (LRS)
The pipeline begins at a 64×64 resolution. Instead of the standard full reverse diffusion, LRS uses a truncated sampling method. By integrating the global structure of the low-quality (LQ) input early in the process at a low resolution, the model saves significant computational time that would otherwise be spent on high-resolution initialization.
2.2 Adaptive Degradation Remover (ADR)
Transitioning to 256×256 resolution, the ADR stage acts as a bridge to strip away unknown degradations. It employs a dynamic low-pass filter that adaptively tunes its cutoff frequency based on a Mean Squared Error (MSE) loss calculated between the intermediate result and the input. This feedback loop ensures the restoration remains faithful to the original identity without requiring manual parameter tuning for different blur levels.
2.3 Generative Detail Boost (GDB)
The final stage utilizes a fine-tuned Stable Diffusion model to inject realistic textures and fine details, such as iris patterns and skin wrinkles. By leveraging the strong generative priors of large-scale text-to-image models, GDB performs a final truncated denoising process to produce the high-quality (HQ) output.
3. Experiments and Results
3.1 Datasets and Evaluation
The model was trained on the FFHQ dataset. Quantitative evaluations were performed on a synthetic test set derived from CelebA-HQ, measuring PSNR, SSIM, and LPIPS. To ensure real-world robustness, the authors also tested on the WIDER-Test, CelebChild, and LFW-Test datasets.
3.2 Performance and Speed
TD-BFR demonstrates a significant leap in efficiency. It achieves an inference speed of 3.53s per image, compared to 25.04s for DifFace and 18.33s for StableSR. On the CelebA-Test dataset, TD-BFR achieved a PSNR of 25.21 and a NIQE of 3.99, outperforming existing methods in both pixel-level accuracy and natural visual quality.
3.3 Qualitative Analysis
Visual results indicate that TD-BFR preserves identity more effectively than GAN-based architectures, which can appear artificial. It also avoids the "hazy" appearance or semantic distortions sometimes found in other diffusion-based restoration models, excelling particularly in the restoration of natural skin textures and eye details.
4. Conclusion and Future Work
TD-BFR successfully reduces the computational overhead of diffusion-based face restoration while maintaining high generative fidelity. While it represents a 4.75× speedup over current diffusion benchmarks, it remains slower than near real-time GAN-based methods. Future research will focus on optimizing the Generative Detail Boost stage and refining the MSE thresholding in the Adaptive Degradation Remover to handle even more extreme degradation scenarios.
Summary updated