InfoBFR: Real-World Blind Face Restoration via Information Bottleneck

Nan Gao, Jia Li, Huaibo Huang, Ke Shang, Ran He

TL;DR

InfoBFR is a plug-and-play framework designed to "polish" the outputs of existing blind face restoration (BFR) models. By combining a manifold information bottleneck for noise compression and a one-step diffusion LoRA for detail compensation, it effectively eliminates AI-generated artifacts—or "neural degradations"—to produce high-fidelity, realistic faces with minimal computational overhead.

What Problem Does This Paper Address?

Blind Face Restoration (BFR) aims to recover high-quality (HQ) faces from low-quality (LQ) images plagued by blur, noise, and compression. While current state-of-the-art models (like CodeFormer or GPEN) produce impressive results, they often introduce what the authors call neural degradations.

These aren't physical image defects, but rather artifacts created by the neural networks themselves:

  • Topology Distortion: Weirdly shaped facial features.
  • Prior Bias: The model forces a face to look like its training data rather than the original person.
  • Artifact Residues: Unnatural textures or "hallucinations" in sensitive areas like teeth or eyes.

As shown in Fig. 1, existing models often struggle with these issues in complex real-world scenarios. InfoBFR acts as a universal "cleaner" to absorb the essence of these pre-trained models while discarding their mistakes.

Fig. 1. InfoBFR helps pre-trained BFR models (CodeFormer [30], Diff- BIR [36], GPEN [20]) fight against neural degradations, e.g., topology distortion (col 1), prior bias (col 2), texture distortion (col 3), and artifact residues (col 4). Note that pre-trained BFR models and our proposed In- foBFR are both trained on the FFHQ [21] dataset. More visual examples are shown in Fig. 5.

Methods

The InfoBFR framework follows a two-step "Information Optimization" strategy:

  1. Information Compression (MIB): The output of a pre-trained BFR model is mapped to a latent manifold. The authors introduce a Manifold Information Bottleneck (MIB) that uses a Transformer block to gain a spatial-perception view of the face. The MIB acts as a filter, maximizing the mutual information between the restored manifold and the ground truth while minimizing the information inherited from the noisy, artifact-prone BFR output.
  2. Information Compensation (Diffusion LoRA): To restore the details lost during compression, the framework uses a pre-trained Stable Diffusion model. Crucially, they employ LoRA (Low-Rank Adaptation) to fine-tune the model, allowing it to synthesize high-quality facial details in just one denoising step. This avoids the massive computational cost and uncertainty of standard multi-step diffusion processes.

The entire architecture is summarized in Fig. 4, highlighting its plug-and-play nature.

Fig. 4. InfoBFR framework adopts a diffusion finetuning strategy to tackle the challenging real-world BFR task, novelly encompassing transformer, MIB, and diffusion LoRA. Fig. 7 and Table 3 demonstrate the indispensability of transformer and MIB. We implement manifold information compression from neurally distorted manifold EXBF R followed by a QKV attention. Based on compressed feature Z in Algorithm 1, diffusion LoRA is then used to control feature transformations for real-world BFR adaptively. QC means quant conv layer of the encoder of pre-trained VAE. Overall, the InfoBFR framework effectively achieves high-quality BFR results with robustness in diverse domains. Powerful supporting experiments are shown in Table 2 and figures of the paper and supplementary material.

Datasets

  • Training: The model was trained on the FFHQ dataset, a standard benchmark for high-quality human faces.
  • Evaluation: To test real-world generalization, the authors used several challenging "in-the-wild" datasets:
  • Wider-Test: Severely degraded faces from the WIDER Face dataset.
  • FOS: Faces with side views or heavy occlusions.
  • CelebChild: Real-world photos of children.
  • WebPhoto: Low-quality faces collected from the internet.

Key Insights and Ideas

  • Neural vs. Data Degradation: The core insight is that the "noise" in modern BFR isn't just in the input; it's generated by the restorer. Treating AI hallucinations as a form of degradation that can be filtered via an Information Bottleneck is a novel shift in perspective.
  • The "Essence" Filter: The MIB uses an information filter ($\lambda$) to decide how much of the original BFR output to keep. If $\lambda$ is too low, the result is random noise; if it's too high, the neural artifacts remain. Finding the sweet spot allows for controllable restoration.
  • Efficiency via One-Step Diffusion: By starting the diffusion process from a point already close to the "finish line" (the BFR model's output), the authors prove that a single denoising step is sufficient for high-quality detail compensation.

Key Results

InfoBFR demonstrates significant quantitative and qualitative improvements across the board:

  • Performance Boost: The framework achieved a "BFR-boosting" effect in nearly 85% of evaluation events. As seen in Fig. 3, it significantly improved FID (Frechet Inception Distance) and MUSIQ (image quality) scores for a wide variety of base models.
  • Speed and Scale: Despite using a diffusion backbone, InfoBFR runs in approximately 70ms and uses only 16.4M trainable parameters, making it much lighter than standalone diffusion restorers.
  • Visual Fidelity: Qualitative results in Fig. 6 show that InfoBFR produces cleaner teeth, more natural eyes, and better-preserved identities compared to original SOTA models.
Fig. 3. Comprehensive FID-boosting and MUSIQ-boosting visualization of BFR* models equipped with plug-and-play InfoBFR, compared with the original BFR in real-world datasets with heavy degradations (Wider- Test [30], FOS [38], CelebChild [19] and WebPhoto [19]). InfoBFR facil- itates BFR quality by effectively harnessing the information bottleneck. More quantitative evaluations are shown in Tab. 1 and Tab. 2.
Fig. 6. Qualitative comparison with state-of-the-art methods (CodeFormer [30], GPEN [20], DiffBIR [36], GFPGAN [19], DifFace [35], DFDNet [16], VQFR [31], FeMaSR [37], StableSR [9] and GCFSR [8]) in severely degraded photorealistic scenes including side, large-pose, occluded, extreme-expression faces. InfoBFR is capable of higher fidelity and more realistic face restoration. Zoom in for better observation.

Limitations and Future Work

While highly effective, InfoBFR is not a magic bullet. The authors note that if the underlying BFR model produces extremely severe neural degradations (where the face is essentially unrecognizable), InfoBFR may still struggle to recover the correct structure. Future work could involve tighter integration between the base model and the bottleneck to prevent these extreme failures earlier in the pipeline. Additionally, while 70ms is fast, it is still slower than pure GAN-based methods, leaving room for further optimization for real-time applications.

Summary updated

Abstract

Blind face restoration (BFR) is a highly challenging problem due to the uncertainty of data degradation patterns. Current BFR methods have realized certain restored productions but with inherent neural degradations that limit real-world generalization in complicated scenarios. In this paper, we propose a plug-and-play framework InfoBFR to tackle neural degradations, e.g., prior bias, topological distortion, textural distortion, and artifact residues, which achieves high-generalization face restoration in diverse wild and heterogeneous scenes. Specifically, based on the results from pre-trained BFR models, InfoBFR considers information compression using manifold information bottleneck (MIB) and information compensation with efficient diffusion LoRA to conduct information optimization. InfoBFR effectively synthesizes high-fidelity faces without attribute and identity distortions. Comprehensive experimental results demonstrate the superiority of InfoBFR over state-of-the-art GAN-based and diffusion-based BFR methods, with around 70ms consumption, 16M trainable parameters, and nearly 85% BFR-boosting. It is promising that InfoBFR will be the first plug-and-play restorer universally employed by diverse BFR models to conquer neural degradations.
BibTeX
@article{gao2025infobfr,
  author = {Nan Gao and Jia Li and Huaibo Huang and Ke Shang and Ran He},
  title = {{InfoBFR: Real-World Blind Face Restoration via Information Bottleneck}},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2025},
  doi = {10.1109/tcsvt.2025.3637231},
}
Figure